Importing the Data Catalog

To actually start cataloging, we first need to load some data. This can be done via .csv files.

For loading a Data Catalog we need to following columns

Category
Database
Schema
Table
Column

Below is a file containing this. Note that this file also contains additional columns including: “Column_Description” and “Table_UpdatedBy”. Theese are custom fields where we can add additional data to our catalog.

An important detail is that these fields must exist already in engrafo. This can be done in the section “Data Catalog Template”. More about that here: Configuring Templates for the Data Catalog However for this demo, those custom fields already exist in the Engrafo Installation.

To actually load a file, head over to Load Metadata → Meta load (API)

The process for uploading and loading a file is simple: Upload → Validate → Load.

First we upload our chosen file(s) and add a separator. Note that all files used in this demo are semicolon);) separated…

…Validate the file…

And finally load the file.

Now you should see that your Data Catalog contains a bunch of schemas, tables and columns. Most also include descriptions and other information we added using custom fields.

Now that we have our Catalog we can start using it to create lineage diagrams. But first we need to load some additional data to do this: Importing Documentations