When designing Engrafo it was very important to have a way to automate creation of data catalogs and data usage documentation.
The goal is to minimize the manual efforts needed to document data and data usages.
So, we created an API in Engrafo that allows you to extract all kinds of information, place that in a folder and just let Engrafo import and store the information. In that way, you can just schedule your extracts and let Engrafo fetch, structure and storage all the information.
But, the extract you make and store must match tha structure of the API. We have explanied how this is done in this section.
On this page of the Engrafo Guide you will find information about:
Generating Data Catalog via .csv load
Mandatory columns
Load a csv file which has the following mandatory columns:
Kategori
Database
Skema
Tabel
Kolonne
In addition, the csv file can have the following optional columns:
If the columns are to be sorted other than alphabetically
KolonneSortering
If importing to custom fields
Database_[fieldname]
Skema_[fieldname]
Tabel_[fieldname]
Kolonne_[fieldname]
IMPORTANT
The [fieldname] fields must exist as Custom fields before it is possible to load content into them.
Below is an example of how a CSV file might look with the ¤ character selected as the separator:
Kategori¤Database¤Skema¤Tabel¤Kolonne¤Database_Beskrivelse¤Skema_Beskrivelse¤Tabel_Tabelansvarlig¤Kolonne_Beskrivelse¤Kolonne_Kontaktperson Stage¤DB01¤SKEMA_A¤TABEL01¤KOLONNE01¤Vigtig database¤Skema-beskrivelse¤Knus¤Den 1. kolonne-beskrivelse¤Peter Poulsen Stage¤DB01¤SKEMA_A¤TABEL01¤KOLONNE02¤Vigtig database¤Skema-beskrivelse¤Jens¤Den 2. kolonne-beskrivelse¤Maria Hansen Stage¤DB01¤SKEMA_A¤TABEL01¤KOLONNE03¤Vigtig database¤Skema-beskrivelse¤Mads¤Min 3. kolonne-beskrivelse¤Ole Jensen Stage¤DB01¤SKEMA_A¤TABEL01¤KOLONNE04¤Vigtig database¤Skema-beskrivelse¤Ebbe¤Min 4. kolonne-beskrivelse¤Inge Madsen Stage¤DB01¤SKEMA_A¤TABEL01¤KOLONNE05¤Vigtig database¤Skema-beskrivelse¤Gorm¤Min 5. kolonne-beskrivelse¤Jim Jensen
Updating Data Catalog via .csv load
Mandatory columns
Load a csv file which has the following mandatory columns:
KategoriID
DatabaseID
SkemaID
TabelID
KolonneID
In addition, the csv file can have the following optional columns:
If names should be updated
Kategori
Database
Skema
Tabel
Kolonne
If the columns are to be sorted other than alphabetically
KolonneSortering
If custom fields are to be updated
Database_id_[fieldname]
Database_[fieldname]
Skema_id_[fieldname]
Skema_[fieldname]
Tabel_id_[fieldname]
Tabel_[fieldname]
Kolonne_id_[fieldname]
Kolonne_[fieldname]
IMPORTANT
The [fieldname] fields must exist as Custom fields before it is possible to load content into them.
Below is an example of how a CSV file might look with the ¤ character selected as the separator:
KategoriID¤DatabaseID¤SkemaID¤TabelID¤KolonneID¤Kolonne 5¤3¤9¤7¤6¤CVR_NUMMER
Generating Data Usage documentation via .csv load
Mandatory columns
Load a csv file which has the following mandatory columns:
Kategorihierarki
Skabelonnavn
Unitnavn
Ejer
Dokumentationsnavn
In addition, the csv file can have the following optional columns for load of Text fields:
Customfelt_text_name
Customfelt_text_value
In addition, the csv file can have the following optional columns for load of links:
Customfelt_link_name
Customfelt_link_href
Customfelt_link_description
In addition, the csv file can have the following optional columns for load of inputdata:
Inputdata_customfelt_name
Inputdata_kategori
Inputdata_database
Inputdata_skema
Inputdata_tabel
Inputdata_kolonne (* can be used if ALL columns are to be loaded)
In addition, the csv file can have the following optional columns for load of outputdata:
Outputdata_customfelt_name
Outputdata_kategori
Outputdata_database
Outputdata_skema
Outputdata_tabel
Outputdata_kolonne (* can be used if ALL columns are to be loaded)
VIGTIGT
The category hierarchy must already exist before it is possible to load into it
The category elements must be specified with ¤ as a separator, for example: Level1¤Level2¤Level3
The templates specified in [Skabelonnavn] must exist before loading to them is possible
Units specified in [Unitnavn] must exist before loading to them is possible
The fields specified in [Customfelt_text_name] must exist as Custom fields before it is possible to load content into them
The fields specified in [Customfelt_link_name] must exist as Custom Fields before it is possible to load content into them
The fields specified in [Inputdata_customfelt_name] must exist as Custom fields before it is possible to load content into them
The fields specified in [Outputdata_customfelt_name] must exist as Custom fields before it is possible to load content into them
The column specified in [Inputdata_kolonne] must exist before it can be loaded as input data
The column specified in [Outputdata_kolonne] must exist before it can be loaded as output data
Below is an example of how a CSV file can look for loading both text fields and links, where the character ; is selected as separator:
Kategorihierarki;Skabelonnavn;Unitnavn;Ejer;Dokumentationsnavn;Customfelt_text_name;Customfelt_text_value;Customfelt_link_name;Customfelt_link_href;Customfelt_link_description Level1;ETL;Oversigt;Ole;Datarens;Beskrivelse;Dette er en kort beskrivelse...;Link til info;https://tv2.dk/;TV2 Level1;ETL;Detaljer;Peter;Datarens;Vigtige fakta;Her står fakta omkring datarens;;; Level1¤Level2;ETL;Oversigt;Ole;Solvensberegning;Beskrivelse;Dette er en beskrivelse af solvensberegningen...;Link til info;https://www.dr.dk/;Danmarks Radio Level1¤Level2;ETL;Detaljer;Peter;Solvensberegning;Vigtige fakta;Fakta om solvens er angivet her...;;;
Below is an example of how a CSV file can look for loading input data, where the character ; is selected as separator:
Kategorihierarki;Skabelonnavn;Unitnavn;Ejer;Dokumentationsnavn;Inputdata_customfelt_name;Inputdata_kategori;Inputdata_database;Inputdata_skema;Inputdata_tabel;Inputdata_kolonne Level1¤Level2;ETL;Oversigt;Ole;Datarens;Inputdata;Kategori 1;Database A;Skema A;Tabel A;Kolonne A Level1¤Level2;ETL;Oversigt;Ole;Datarens;Inputdata;Kategori 1;Database A;Skema A;Tabel A;Kolonne B Level1¤Level2;ETL;Oversigt;Ole;Datarens;Inputdata;Kategori 1;Database A;Skema A;Tabel A;Kolonne C Level1¤Level2;ETL;Oversigt;Ole;Datarens;Inputdata;Kategori 1;Database A;Skema A;Tabel B;*
Generating Data Catalog SQL Server connection string
TBC - sorry
Generating sample space via .csv load
Load a csv file that has the following required columns:
Udfaldsrum
Værdi
Beskrivelse
The following is an example of what a CSV file might look like where the character ¤ is selected as the separator:
Udfaldsrum¤Værdi¤Beskrivelse Postnr¤1301¤København K Postnr¤2000¤Frederiksberg Postnr¤2100¤København Ø Postnr¤2200¤København N Postnr¤2300¤København S Postnr¤2400¤København NV Postnr¤2450¤København SV Postnr¤2450¤Valby Postnr¤2600¤Glostrup