Configuring Templates for Data Usage
- Elias Helgren Dahl
Organizing your Data Usage is vital when it comes to navigating Documentations and creating Lineage. This is also important for making sure uploading metadata goes without issues.
On this page you will find information about:
The information pages for the data usage part of Engrafo consists of templates with several units which contain fields as to organize the information. This is all configured in the section under “Data Usage Templates“.
Here you will see options to create and edit categories, templates, units and fields in Engrafo Data Usage
Creating Categories and Hierarchies
A way to organize your various data usage entities like ETL-jobs, ML algorithms, Analytics, Data Visualizations, etc. is to create a categorial hierarchy in Engrafo.
This can be done in the section under “Data Usage Categories”. Here you can then easily create a new one and assign a parent category if applicable…
… and will reflect immediately in the Data usage section of Engrafo
NB! Important: Note that when automatically generating data usage documentations via import, these hierarchies must be created first. The import can (unlike with the data catalog) not be created by the import.
Creating Templates, Units and Fields
Before you can create and configure templates, it is important to first understand what a unit and a field is. In broad terms it can be described as the following:
Template is the overall documentation structure for a given area. It can be “ETL jobs in SAS”, “Finance analytics”, “Finance analytics in Python”, “Power Bi VA in Norway”. Pretty much anything where a specific unique structure is needed.
Units are the building block of your templates. You can have one unit or several, that is up to you. Three typical units within a single template could be “Business background”, “Technical documentation” and “Data”.
Fields are the specific entities within a unit. So fields within the unit “Data” could be “Source data”, “Output data” and within Technical documentation it could be “Special calculations”, “Schedule” and “Lead developer”
See Below for an illustration containing the complete structure for the Data Usage section:
Although this can very easily be changed at a later state it is highly advisable to do a workshop with hands on stakeholders, to make sure the needs of everyone is met AND that you for starters follow the golden principal “less is more”
Creating templates, units and fields pretty much all follow the same procedure. Simply click what you want to create in the overview (e.g. Data Usage Fields), if you wish to edit an existing field select the corresponding template unit and find the field you wish to edit, otherwise simply press “Create new”.
From here, assign the template unit, field type, a name for it, how it should be sorted relative to other fields in the unit, give an explanation if wanted and assign properties as needed. Finally click “Create” and your new data usage field will be created.
In the Engrafo free trial version for download: https://engrafo.atlassian.net/wiki/spaces/EDV/pages/258179083 there are several examples of data usage templates to play around with!
Types of Fields and Properties for Fields
Below is a pretty boring, but important list of the types of fields you can make as metadata attributes for Data Usage, as well as the properties you can give them for plugging in the right data.
Field Types
Inputdata: This is a special field where input/source data for the specific data usage is mapped towards the data catalog. What goes into the field after it has been created preferably flows automatically through integration, but can quite easily be selected manually via the integrated data catalog browser.
Outputdata: This is a special field where output data for the specific data usage is mapped towards the data catalog. What goes into the field after it has been created preferably flows automatically through integration, but can quite easily be selected manually via the integrated data catalog browser.
Link: Link to a URL with optional display text
Link in Modal View: Link to a URL with target website opening in a frame inside Engrafo
Metadata-Load: SAS (scaproc): This is a special field for automatically generating code visualization and analytics around SAS-code.
Metadata-Load: SQL:
Text: Simple text display
Textarea: Larger area for simple text
Rich-text editor: Allows for rich-text editing (i.e. text size, font and other properties)
Checkbox: Adds a simple checkbox with the name of the field
Date: Adds a date - dd/mm/yyyy
Datetime: Adds date and time - dd/mm/yyy
Image: Upload an image and optional image text
Static File: Adds a static link to an external file. (eg. docx, xlsx, pptx, html, pdf, etc.)
Properties
Note that some field types cannot have all properties.
Mandatory: Makes the field require information represented with a “*”
Hide: Makes the field hidden in Data Usage
Read Only: The field becomes non-editable
Multiple: Allows for multiple of the same field under one unit
Attribute: Makes the field show as an attribute for the given item in the Data Catalog??
Show at search: Makes this field included in overview search results. Read more about the Engrafo search function here: https://engrafo.atlassian.net/wiki/spaces/EDV/pages/258015247