Skip to main content

Qlik Sense SaaS - Automatic data lineage

A Qlik app rarely loads data straight from its final source — instead data is pulled from a database, staged in a QVD file, reshaped by another app, staged again, and only then used in a sheet.

Dataedo traces that chain with Data Lineage built using Qlik Cloud's own lineage API. Nothing extra needs to be enabled in Qlik — the API key used by the connection is enough. Qlik computes the flows while it reloads an app, and Dataedo reads the result.

What to expect​

Inside a Qlik app​

App tables — shown as datasets in Dataedo — are the tables an app builds during a reload. Dataedo creates column-level lineage from them to the sheets and visualizations that use them, so you can see which columns feed a given chart.

Measures and dimensions are read as expressions, so a field built from several columns produces a flow from each one. For example: Sum([Price] * [Quantity]) would link to both Price and Quantity.

A visualization that renames a field keeps the link to its origin under the new label.

warning

Lineage inside an app is derived from field expressions. Field names must be referenced in square brackets ([Sales]) to be recognized.

Column-level lineage from a Qlik app table to visualizations on a sheet

From databases​

When an app loads from a database connection, Dataedo traces the source to the specific table and links it through a Linked Source. Matching is based on the identity of the Qlik data connection and the table's full name, including its schema, not on text found in the load script.

Column flows come from the detailed column-level lineage Qlik provides, so a column that is renamed or computed in the app's load script keeps its link to the source columns it came from.

For this to work, you must document the database in Dataedo and match its linked source during import — if you have not, see Troubleshooting.

Column-level lineage from a SQL Server table through a Qlik app table to a QVD file

From files​

Files that an app loads are documented as datasets and connected to the app tables that read them. This covers QVD and CSV files as well as spreadsheets.

Column-level lineage follows as long as the Qlik catalog knows the file's column names — see Known limitations for the case where it does not.

Column-level lineage from a CSV dataset in the Qlik catalog to a Qlik app table

Through QVD files​

QVD files are the usual way to stage data in Qlik, and a single chart often sits at the end of a long chain of them. Dataedo follows those chains end to end.

A typical one looks like this:

  1. An extract app reads a database table and stores a QVD.
  2. A transform app reads that QVD, reshapes the data, and stores another QVD.
  3. A reporting app reads the second QVD and builds the sheets people look at.

Every step appears separately on the diagram, the intermediate apps included, and column lineage propagates through all of them. Processes are named after the operation that produced the object — LOAD Employee Extract.Employees, STORE Employees.qvd — so the direction of the chain is readable at a glance. The same mechanism connects one app to another whenever they exchange data through a QVD.

Multi-step Qlik lineage chain from a database table through two QVD files to a reporting app, with column lines

To see which app performs a given step, switch Lineage view to via processors in the diagram settings. Each step then appears as a process owned by the app that ran it — here Department Summary reads Employees.qvd with LOAD Department Summary.DeptSummary and writes the next file with STORE DeptSummary.qvd.

Qlik lineage with the via processors view: a QVD file, the LOAD process of the Department Summary app, its app table, the STORE process, and the resulting QVD file

Lineage support at a glance​

info

This page covers Qlik Sense SaaS. Qlik Sense On-Prem builds lineage from the app load script instead and behaves differently — see Qlik Sense On-Prem data lineage.

FlowObject-levelColumn-levelNotes
Database table, through a linked sourcerequires a matched linked source
QVD fileincluding multi-step chains
CSV file
Excel spreadsheet
Any file scanned without treating the first row as column headersfix the schema in the Qlik catalog, see Troubleshooting
Connection-based dataset (a database table added to the Qlik catalog)the app's lineage points at the database table through the linked source, not at the catalog entry — see From databases
Another Qlik appthrough a QVD file; intermediate apps appear as steps
App table to sheet or visualization ⚠️built from expressions, not from the lineage API — see Known limitations

Known limitations​

  • Transformation formulas from load scripts are not imported. Qlik's lineage data reports which columns a value depends on, not the expression that combines them — the flow Salary → AnnualSalary is shown, the expression behind it is not. Measure and dimension expressions of visualizations are imported from the app itself.
  • Lineage to sheets and visualizations requires bracketed field names. This part of the lineage is derived from field expressions, and only fields referenced in square brackets ([Sales]) are recognized — see Inside a Qlik app.
  • Files scanned without treating the first row as column headers get object-level lineage only. When the Qlik catalog describes a file as columns A, B, C, those names match nothing on the app side and the column mapping is not created. This most often happens to spreadsheets, but can affect any file type. The fix is on the Qlik side — see Troubleshooting.
  • A QVD file written by an app shows its producer only when another imported app reads it. Dataedo fetches lineage per app, tracing only back to its sources, not forward to what reads it, so a STORE into a file that no imported app loads is not visible. Import the app that reads it, or check the file's lineage directly in Qlik Cloud.
  • Lineage to a database depends on a matched linked source. If the linked source is missing or points at the wrong database, the chain stops at the app table — see Troubleshooting.
  • Importing lineage takes time. Qlik allows only about 20 lineage requests per minute and Dataedo makes one call per app, so a tenant with many apps spends noticeably longer in the lineage step. Dataedo retries with the delay Qlik requests; an app whose lineage still cannot be fetched is reported in the import errors, and the rest of the import continues.

Troubleshooting​

I don't see lineage from a database source​

  1. Confirm the source database is documented in Dataedo. Without its tables in the catalog there is nothing for the flow to point at.

  2. On the connection's Linked Sources tab, check that the linked source points at that database — assign it by hand if needed.

    Linked Sources tab of a Qlik Sense SaaS connection with a SQL Server source matched to a documented database
  3. Rerun the import — the documentation may come from an older version or an import run with different settings.

I don't see column-level lineage from a file​

  1. Open the dataset in the Qlik catalog and look at its column names. If they are A, B, C…, Qlik scanned the file without treating the first row as column headers.
  2. Edit the dataset's schema in Qlik so that the first row is used as field names.
  3. Rerun the import — the column mapping appears on the next run.

I don't see any lineage for an app​

  1. Check the import errors for that app. A rate-limited or unavailable lineage endpoint is reported per app and does not stop the rest of the import.
  2. Rerun the import for that app.

Lineage stops at a QVD file​

  1. Find the QVD file where the chain breaks — it is shown on the diagram, but nothing is shown writing or reading it — and check in Qlik which app loads or stores it.
  2. If that app exists but is not part of the import (for example, it was filtered out), include it and rerun the import.
  3. If the file is written by an imported app but no imported app reads it, the STORE step is not visible in this version — see Known limitations.
Dataedo is an end-to-end data governance solution for mid-sized organizations.
Data Lineage • Data Quality • Data Catalog