Data stored in Dataedo

Dataedo is an on-premise application. This means that it runs locally, and that the data is stored in a repository on your server. This approach helps to ensure the security of your data. It is still, however, important to understand what data is actually read and stored in Dataedo, and how it can be used.

On-prem architecture

Dataedo is an on-premise software. There is no data encryption involved when your data is processed, as everything happens locally. To ensure the highest level of safety, we advise following certain precautions when setting up your local repository.

Metadata

When connecting to a Data Source, Dataedo extracts metadata — information relating to your data's organization, and characteristics. Depending on a source, it can be read from the stored information schema, application logs, or using other comparable methods. Only the metadata is stored in your local Dataedo Repository, no actual data (involving potentially sensitive data) is kept in Dataedo during the metadata import. You can read more about metadata extracted from each supported Data Source in the data sources section.

The stored Metadata is the basis of all other processes and operations in Dataedo with the exception of Data Quality, Data Profiling, and Data Classification which use samples of real data. Learn more in the dedicated sections below

Data Profiling and Data Quality

Data Profiling

Data Profiling is concerned with quantitative information regarding your data — the mean length of strings in a column, the most common value, the number of null values, distinct values, value distribution etc. As such, it requires access to your data. During profiling, Dataedo queries your data source, analyzes the returned rows, and stores the results of that analysis in your repository so they can be presented in the catalog. The queried rows themselves are not kept in the repository.

What is read from the source

Profiling is performed column by column, and only for supported data types (TEXT and CLOB columns, for example, are skipped and listed as Unsupported). Strings longer than 250 characters are truncated to the first 250 characters for processing.

There are also size limits: by default, a column is not profiled if its table exceeds a row-count limit or is larger than 5 GB. Both limits are controlled by the DATA_PROFILING_TABLE_ROW_COUNT_LIMIT and DATA_PROFILING_TABLE_DATA_SIZE_LIMIT_GB parameters in the dbo.configuration table.

What values from the source are saved

caution

Random values come from a separate query run against the source table, not from the top values. This means that rare values that never appear among the most frequent ones can be saved in Dataedo.

Two kinds of information end up in the repository:

• Aggregated statistics — numbers such as row count, number and share of null and empty values, number of distinct values (cardinality), mean/minimum/maximum string length, and detected formatting patterns. These values do not contain individual rows of your data.

• Actual values from your source — optionally, Dataedo can store samples from your original data source. This covers:

  • Top values — up to 100 of a column's most frequent values, together with their counts.
  • Random values — up to 100 non-null values picked at random from your tables.
  • The minimum and maximum values, and the mean value (or, for non-numerical data, the most common value).

  These are stored in the dbo.data_profiling_column_datasets table of your repository.

A record of every profiling run is also kept as profiling history, so the statistics above are stored not only for the latest run but for previous runs as well.

Dataedo does not store your full column data, instead profiling keeps aggregates plus a bounded sample of representative values.

Profiling modes and what each one stores

How much of the above is kept for a given column depends on its profiling mode:

In Statistical mode, users see only derived information such as mean length, completeness, or data patterns. Data Samples from your actual data source are never stored These are kept only for profiling in full mode.

You can configure these modes globally, per confidentiality level, and per classification in the Profiling Configuration settings, or review and adjust the mode of individual columns before publishing the results to your users.

Deleting stored profiling data

Lowering a column's profiling mode deletes data from your repository permanently:

• Changing Full to Statistical removes the stored values from your source (top values, random values). Switching back to Full does not bring them back — they reappear only after the next profiling run.
• Changing a mode to None permanently removes all profiling information for that column, including its profiling history. Re-enabling profiling recreates the statistics on the next run, but the history is lost for good.

Visibility of stored results

Storing profiling results is separate from exposing them. Unpublished results are visible only to (Power) Data Stewards; once published, they become part of the catalog and are visible to all user types with relevant permissions. Unpublishing results does not delete them from the repository — only setting the mode to None does.

Data Quality

Data Quality on the other hand, needs access to your Data to check all of the Data Quality rule instances you configured in Dataedo.

If a rule is satisfied (meaning that your data is healthy and compliant with the standards you defined), the data will not be held in Dataedo after the check finishes. However, when a column's row fails the Data Quality check its details are stored in the repository. You can choose to disable storing failed rows, but this feature is necessary to display the failed rows report in the application.

The failed rows information is stored in a [dbo].[dq_failed_rows] table. Its row_values column contains a JSON file with the failed rows' rowIDs. The maximum number of failing rows that can be saved, as well as any extra information saved beside the rowsID are defined when setting up a Data Quality rule instance. Only the fields you explicitly select during the rule instance configuration are saved — never the entire row. Every other column of a failing row stays in your data source.

Once the failing value is amended, during the next Data Quality check Dataedo will recognize it as fixed and automatically delete the stored information.

Each failing row can also be ignored by the User to prevent it from getting flagged in subsequent Data Quality checks. Ignored rows are stored in the [dbo].[dq_failed_rows_ignored]. Only the row_id is stored.

So to summarize: Data Quality does not store the entire data of checked columns by default — only the failing cells and whatever extra fields the rule's author chose to save. Two things are worth keeping in mind:

• If the entire column fails, every one of its cells will be stored, leaving the option to reconstruct that column's contents.
• Because the extra saved fields can be any columns of the failing row, a failed rows report can end up holding names, surnames, or any other sensitive data the rule's author decided to include. It is therefore worth reviewing what your rule instances actually store.

Disabling Data Access

Data Quality and Data Profiling require access to your data. You can decide to disable those functions (and subsequently, Dataedo's access to Data) in the General Tab of System Settings. Find the Enable Data Quality and Profiling toggle, and turn it off. Remember to click save to apply this change.

Data Classification

Data classification reads a sample of data from each checked columns — upwards of 1000 data points per column. This sample is then checked against rules governing the Semantic Type assignment (Semantic Types are used to classify data, learn more here). This happens after a Metadata Import, if Classification is enabled.

The sample data is not stored by Dataedo afterwards.

Disabling Data Access

You can prevent Dataedo from using Data Samples to run Data Classification. To do so head to Settings>System settings and click the General tab. Find and disable the Enable Data-Based Classification toggle. Click Save to remember your choice.

Doing this will not prevent automatic Data Classification from running, but can seriously lower its accuracy since it will be based solely on your column's names (learn more here). Therefore we advise turning this option off only if your organization operates in an extremely data-sensitive field.

Cell-by-cell analysis

Data Profiling, and Data Classification all look at the values of a single column in isolation — they never combine a cell with the other cells of its row. A rule checking a first name, for example, considers that name on its own, not together with the middle name, last name, or date of birth.

In Data Quality, rule instances can analyze multiple cells in one row. While some rule instances are built-in (but their use is optional), users in your repository can define their own rules using SQL. Those "multi-cell" rules help to satisfy complex rules of data well-formedness, by aggregating info from multiple cells (e.g: event_name and parameters both meeting certain conditions in order to catch specific host addresses, or combining name and last_name to find unique users).

Custom SQL rules, in general have the widest access to your data, since they query your data directly and can look over multiple cells. Because of that, the option to create them is limited to users with Power Data Steward permissions only.

Data sent to Dataedo

Dataedo tracks launch logs, crash reports and application usage. The stored information is minimal, and contains only what is necessary for troubleshooting and product analysis purposes.

Launch logs and usage records are periodically sent to Dataedo. As an Administrator, you can choose to opt out of this by blocking the transfer via a firewall, or editing application data.

Crash reports are not forwarded to us automatically, but you can choose to send them to our support. Keep in mind that they can contain sensitive information within the stack trace. Always make sure to review the crash reports before sending them to us.

AI Usage

Dataedo does not offer its own AI model. However, your organization can choose to connect a custom AI model to the repository via API. We offer support for selected OpenAI models. Dataedo does not perform any fine-tuning on your models, we use them as-is.

Currently AI models are used to enable the autodocumentation of your metadata and glossary terms.

Dataedo supplies pre-defined queries, supplied with the following information from your repository:

Object to Generate Description For	Metadata Sent to OpenAI
Column	Id, Name, DataType, IsNullable, IsUniqueKey, PkConstraint, FkTargetName, DefaultValue, CalculationFormula, ParentName, ParentObjectType
Table/View	Id, Name, ObjectType, Description, Title, Schema, List of linked terms
Term/KPI	Id, DisplayName, Code, Description, Title, List of related terms

Only the metadata specified above is sent to the model, the AI autodocumentation feature does not read any of your actual data. The query is then sent to the AI model via API, and the suggested description is returned to the user.

MCP

If you configure model access via MCP, your selected AI agent will receive read only access to information stored in your Dataedo repository. This includes extracted metadata, as well as any documentation (descriptions, Custom Fields, etc.) you created in Dataedo. Models connected through MCP do not get direct access to Data in your original data sources.

Encryption

Dataedo stores the following secrets in your repository:

• all passwords, tokens, keys etc. used in Credentials for connecting to external Data Sources.
• secrets used for configuration of authentication methods
• SMTP configuration passwords

Secrets stored in Dataedo are encrypted using the AES-256 encryption protocol.

info

Dataedo does not store any secrets used to login to Dataedo Portal

MCP

Personal Access Tokens are stored during MCP configuration. They are not encrypted, but hashed using the SHA-256 protocol. Hashing works only one way, so Dataedo can verify a token you present, but never recover it — which is why a token's secret is displayed only once, right after you create it.

PATs are removed automatically after they expire, or when a user's account is deleted.

UI

Most encrypted details are never shown to users after they have been saved. They are stored only internally, and used for repeated connections and other similar actions. The only information visible in Dataedo's user interface, is whether a given secret has been set or not.

The only exception to this rule is a connection string, which is returned in its encrypted form to Admin users. This string is necessary for Dataedo Agent configuration. It is kept in a file on your application server, rather than in the repository database.

Public API

You can choose to integrate Dataedo with other tools your organization is using, to streamline data fetching or creation of certain objects. Keep in mind that this creates a connection between your local repository and an external tool. Find our API documentation here.