dbt - Automatic Data Lineage

What to Expect

dbt models

Dataedo shows column-level lineage for every model that dbt materializes (table, view, incremental, ...). Each model is treated as a processor that links its upstream objects to the objects it creates in the warehouse (see Processors and processes).

• Ephemeral models do not appear in lineage. When an ephemeral model feeds another model, lineage is created from upstream sources of the ephemeral model.
• source objects themselves do not appear in lineage even as processors; their materialized counterparts do.
• Column-level links come from SQL parsing.
• Additional object-level links come from dbt metadata (in case SQL parsing is not possible).

Same flow in dbt Docs

How it looks inside warehouse documentation (not dbt documentation)

• If you hide the processors (so-called business view), you will see a clean source-to-target flow without dbt objects.

• If you enable the processors (technical view), you will see how each object was created.

.csv file → dbt seed (processor) → materialized table/view

Dataedo creates object-level data lineage from a .csv file to its materialization in the warehouse with dbt seed (structure) as a processor.

Limitations

• Warehouse must be documented – automatic lineage appears only if the warehouse targeted by your dbt project has also been imported into Dataedo.
• Column-level lineage depends on SQL dialect – Dataedo can break lineage down to columns only when the Dataedo parser supports the warehouse's SQL dialect. See the SQL parsing overview for the current list of dialects.
• Materialized objects only – lineage is generated for models that dbt materializes. Raw source tables and ephemeral models themselves do not get lineage, although flows feeding into an ephemeral model are still traced.
• Linked sources must be mapped manually – any linked source left unassigned will produce no lineage.

Troubleshooting

Lineage missing right after the first import

See “After the first import — activate automatic lineage” in the dbt Cloud guide or the dbt Core guide. After mapping the linked sources, remember to run Import changes on the dbt documentation to populate the lineage.

No object-level data lineage

1. Check dbt Docs – If object-level lineage is absent in dbt Docs, it cannot be imported. See the dbt guide on viewing lineage.
2. Automatic lineage – Confirm the Automatic lineage checkbox is ticked in the connection form.
3. Warehouse imported – The target warehouse must also be in Dataedo.
4. Linked source mapped – Map the warehouse to its linked source (see Map each linked source to the right database in the Cloud/Core guides).
5. Run Import changes – Once mapping is correct, run Import changes to refresh lineage.
6. Contact support – If lineage is still missing, open a support ticket.

how to verify lineage is present

Open any non-ephemeral model and switch to the Data lineage tab.

Desktop: make sure you are viewing the dbt object, not the warehouse table.
Web: if you open a warehouse object, confirm it belongs to your dbt project and enable Show processors in the lineage tab.

Column-level lineage missing for some columns/objects

If you see the object-level lineage but the column-level only for some columns/objects, check if features used in the SQL code are supported by Dataedo SQL parser - SQL parsing overview. If all features are supported, but you still do not see the column-level lineage, please contact Dataedo support and provide the compiled script of the object you are having issues with.

You see object-level lineage, but column-level lineage appears only for part of the model.

• Check whether Dataedo's parser supports the SQL features used in those scripts (see the SQL parsing overview).
• If the features are supported yet lineage is still missing, please contact support and attach the compiled SQL for the problematic object.

No column-level data lineage

1. Parse SQL – make sure the Parse SQL checkbox is ticked in the connection form.
2. Dialect match – verify that each warehouse-linked source uses the correct SQL dialect. If Dataedo auto-assigns a similar but not identical dialect (e.g., PL/SQL for Db2), column-level lineage may be incomplete. See the SQL parsing overview for supported dialects.
3. Default schema – confirm the Default schema is set on the warehouse connections mapped to linked sources.
4. Contact support – if column-level lineage is still missing, open a support ticket.