Adding a manual datasource lineage relationship allows for precise control over the connections between datasets, ensuring accurate data mapping and enhancing data traceability and management.
Relationships at the datasource level affect the entire data source, which may include multiple datasets and fields. Typically used to define how different datasources relate to each other, often through common keys or identifiers. Changes here can have a broad impact, potentially affecting all datasets and fields within the datasource.
To add a manual lineage relationship for a datasource:
- Navigate to and select the specific datasource to which you wish to add manual lineage.
- Go to the Lineage tab to view the origin card containing information about the selected datasource.
- Enable Edit-Mode to start building lineage relationships manually.
- You can hover over the card, field, or dataset information within the card. The card itself cannot establish a relationship with another card; only the items within the card can form relationships.
- Two arrows or + (Add) buttons will
appear in both directions. Hover over the dataset in the
card to select relationship direction, either downstream or
upstream. Note: If the card is in a selected state and no options appear when hovering, it indicates that a relationship cannot be established at the datasource level.
- Select the relationship direction to open the Add Lineage Relationships side panel.
- A list of available datasets will appear. Select the dataset you want to link to the current dataset. After selecting the desired dataset, click Next to proceed.
- You will be prompted to select the Relationship Type. Choose the appropriate type (for example, upstream or downstream) based on how the datasets interact.
- Map the columns that correspond to the selected datasets according to the established relationship types and click Add.
- The selected fields will get added to the canvas in the chosen direction. Review the lineage diagram to ensure that the new relationship is accurately represented.