Creating Data Lineage
- Elias Helgren Dahl
Now that we have created our data documentations, we can start messing around with Data Lineage Diagrams.
If we head to the Data Usage segment and click on one of our Data Documentations, we will see a bunch of info including information about input and output. This is what is used to create lineage flowcharts based on this documentation.
If we now click on “Lineage” in the top right we will be redirected to a page with a lineage diagram using the chosen documentation as the starting point.
Here we can customize various parameters to personalize our flowchart.
The graph itself is interactive, so you can click on it to jump to any part of the “data journey” you want. This goes for other data documentations but also any included table or columns hereunder.
Some data journeys are quite long, so one option is to simply limit the amount of steps backwards and forwards in the lineage.
Other parameters include showing variables, external or temporary data. Using this you can easily create some one of a kind awesome looking charts.
Another way to create Data Lineage is directly from the Data Catalog. Here we can click on a table and see how it is used as input and output for our Data Usage Documentations.
For instance if you navigate to an element, you will be able to see lineage in the top right. A list of documentations will be shown that holds information on the usage of that specific table.
You can follow these links to get to the Data Usage documentation that uses this table.
You can also create the lineage diagrams themselves directly from the Data Catalog using the given table as the starting point. You can create either Impact lineage or Reverse Impact Lineage. These diagram will, likewise, show what Input or Output is used for the selected element. Impact Lineage will show what Output is from the table whereas the reverse will show the Input.
Below you can see more about navigating your Lineage Diagrams or learn more about Data Usage Documentations: