Scheduling Tasks in Dataedo Portal

This guide outlines how to manage task scheduling using the Scheduler, Connections, and Credentials views available under the Connectors section in Portal.

Note that only users with the Connection Manager role can access the Connectors section and perform the actions described here.

Connectors Overview

Under the Connectors section in Portal, you can manage:

• Connections – connection configurations for Data Sources
• Credentials – reusable authentication details shared across Connections
• Schedule – task scheduling interface with logs and calendar

Connections

Connections group shared configuration settings like Credentials, host, and port, making it easier to manage and reuse across environments. You can also share one connection across multiple sources.

Accessing Connections

Navigate to Connectors > Connections to see all configured Connections. You can:

• view Connections as tiles or in a grid
• filter by connector type, category, or Connection/Data Source name
• search for specific Connections
• check scheduled tasks in the Scheduled tasks column
• select a Connection name to view more details

Connection Details

When you select a Connection, you can access two tabs:

• Task Planner – manage tasks for that Connection (same interface as in Scheduler)
• Settings – view/edit Connection details and manage Credentials

In the Settings tab, you can:

• update Connection parameters
• assign or create Credentials
• test the Connection to verify access

Adding a new data source

Dataedo checks whether the host your connection points to has some data sources that are not yet imported into that connection. If any are found, the Connection view will let you add those sources to the existing connection using the Add new Data Source button.

Using this button takes you through the same interface as in Importing Metadata, with the difference being that the first two steps (host details and credentials) are skipped, since those details are already stored in the connection.

Since the import steps differ slightly between data sources, you might see a different number of steps; however, you will always be asked to select the source(s) you want to import and the assets whose metadata you want to store in your repository.

Duplicated connections

To avoid redundancies, Dataedo automatically flags potentially duplicated connections. If multiple connections to the same host with the same credentials exist, you will see a warning on the Connections page.

When you click the Clean up connections button, a pop-up will appear. The duplicated connections will be grouped according to their host details and credentials. This results in a list of lists, where each item represents a group of connections that can be merged. For each item you can:

• choose the name of the Connection where all the duplicates will be merged [a] (by default it is the name of one of the existing connections with the most sources linked to it)
• add an optional description to the connection [b] (by default, it will be pre-filled with the descriptions from all merged connections)
• choose to not merge the connections [c]

Finalize your choices using the Merge all button. The duplicated connections will be merged into one, selected connection and then deleted. This leaves you with a single connection holding all your data sources from a given database.

Keep in mind that if the duplicate connections were associated with assets that you documented (e.g., by adding a description, Badges or Custom Fields), the assets' documentation will be retained after the merge.

You can also merge duplicated connections directly from a Connection's page. In this case, the process will be the same, but the merge pop-up will contain only connections that link to the same host as the currently viewed one. So, if you were viewing a Connection to SQL Server, and you were to click the merge connections button there, you would be shown only other SQL Server connections pointing to that same host.

Change database name

Sometimes you have to change the name of a data source in its source database. If such a data source has been previously imported to Dataedo, it will not be recognized after the name change — so during subsequent re-imports, any changes will not be recorded.

To fix this problem, the Connections page allows you to modify the name of an imported data source.

Head to Connectors > Connections and open a connection you want to modify. Then find the data source that changed. Click the three dots icon and select Edit Parameters.

This opens a popup. Use the text box to provide the new name of your data source.

Make sure that the database with the new name has the same schema. Dataedo will test if the data source with the new name exists in your database, and terminate changes if it does not. However, if you accidentally change the name to a data source with a different schema, the connection will still succeed, overwriting the previous data.

Scheduler

The Scheduler is available under Connectors > Schedule and consists of two main tabs:

• Task Planner – create and manage scheduled tasks.
• Calendar – view task execution history and upcoming executions.

Task Planner

Use the Task Planner tab to schedule, edit, delete, or run tasks for your Data Sources.

Features

• tasks are grouped by Data Source
• filter tasks by connector or task type
• use the search bar to find tasks

Task Status

• Active – the task will be executed according to its schedule.
• Draft – the task will not be executed until manually activated.

To Schedule a Task

1. In the task planner grid, find a row corresponding to the Data Source you want to schedule for
2. Find the button corresponding to the type of task you want to schedule
3. Click the schedule button.

You can specify:

• execution frequency:
  • daily at a specific time
  • specific days of the week
  • specific days of the month
• status: Draft or Active

Note: Tasks use the creator's local time zone for scheduling, but are displayed in UTC in the interface.

Limitation: Tasks requiring Browser login cannot be scheduled.

To Edit a Task

1. Locate the task you want to edit in a scheduler. Hover over it
2. Click the Edit icon in the task row.
3. Modify task settings and save.

To Immediately Execute a Task

1. Find the task.
2. Click the Trigger now icon.

3. You will see a pop-up where you can either run the task immediately [A] or schedule a singular individual run later [B]

Note: The task will be added to the queue immediately, but actual execution may be delayed. A single Agent can only process one task at a time and must finish the current task before starting a new one.

To Remove a Task

1. Find the task.
2. Click the Remove icon.

Calendar

The Calendar tab displays task execution logs, including:

• past executions
• ongoing executions
• future (planned) executions

Search and Filter

You can filter executions by:

• Progress status: Past, In Progress, Upcoming.
• Execution status: All, Failed.
• Connector.
• Task type.

Tasks are grouped by execution ID. You can also search by Data Source name.

Note: Future tasks are based on estimated execution time. Actual times may vary depending on system load and Agent capacity.

Metadata Import Timeline

Metadata Imports are divided into 7 steps, giving you better insights regarding which process is actually taking place. This can also help with troubleshooting. If your Import fails, you can check at which step the crash happened.

These steps are:

• Schema Extraction
• Schema Loading
• SQL Parsing
• Lineage Extraction
• Building Lineage
• Extracting Tables Statistics
• Finalizing

Each step provides an overview of statistics — current level of completion, time elapsed on the current step, and number of scanned objects. The status can be viewed from the Calendar tab of the scheduler.

Schema Extraction and Loading, as well as Finalizing cannot be interrupted, as they are necessary for the metadata to appear in your repository.

All other steps can be cancelled. Use the Skip button to cancel a step in the future, or Cancel to terminate the currently running one. The step's status will change to cancelled. After that, the Import simply goes to the next stage. Metadata will still be extracted, but features related to the cancelled step will not be extracted or updated.

When an error happens, you can click the Error status in the relevant import stage, or the Check error message button, to see more details.

Execution History

Each execution includes a log history. If an error occurs, detailed information and a stack trace are available to help with troubleshooting.

To view details for a failed task, click the error icon in the Calendar tab. A popup will show the error message and stack trace, which can be sent to support if needed.

Downloading the Execution log

You can also choose to download the execution log directly to your device. To do so, in the Calendar tab, find the past execution you are interested in, open the menu using the three dots icon and select Download log.

The execution can end with either a success or a failure. Failure happens if the log does not exist in the system. Dataedo will show you information regarding both outcomes.

Execution log download status

The downloaded log's title is formatted as follows: date of execution + execution ID + data source name.

Import Summary

Whenever an import task is finished, you can review its summary in the Scheduler's Calendar tab. The summary is meant as a quick overview of the number and types of assets that were successfully imported, rather than a troubleshooting method. You can use it to verify whether the assets you wanted to document were read properly.

You can view the summary by:

• clicking the three dots icon next to a completed task and selecting import summary [A]
• expanding a task and selecting import summary in the bottom part of the task view [B]

The Import Summary has a few tabs, some of them data-source dependent (for example, the Lineage tab will be visible only for sources supporting Automatic Lineage Extraction).

The first tab you will see for all Import Summaries is the Overview. Overview contains summarized information regarding every import category like objects, Lineage information, statistics, etc. For each category you can see the number of imported assets and the import's status. There is also a Warning column, containing information regarding asset types that caused issues during the import. Clicking the warning will take you directly to the relevant tab, where you can read exact warning messages in the form of short, actionable descriptions; for a more detailed warning and error summary, you would have to look at the log.

The Objects tab shows a breakdown of imported object types. Each object type has its separate section, with its other sub-objects (like columns, parameters, charts, etc.) listed as well. The number of imported objects per type is displayed in the Imported column. The Status column, on the other hand, shows whether these types could be properly imported, or if an error prevented them from being read.

In the Data Lineage tab, you can see a numeric overview of imported objects that provide information about the data flow in your repository, together with the status of each object type's import.

The Linked sources tab displays similar information about imported linked sources. There is an additional column there — Warnings — which lets you know if any linked sources present in your original data source are not properly mapped in Dataedo.

The Basic Statistics and Dependencies tabs are available only for selected objects. They display a numeric overview of query statistics and of stored functions, procedures, and triggers, respectively.

Other tabs

At the bottom of the Import Summary, you can either Download log [A], or export the summary as a JSON file using the Export as JSON button. The resulting JSON file contains all the information from your report summary, in a format that can easily be shared and used outside of Dataedo's context.

Scheduled tasks notifications

useful tip

Enabling notifications from the Scheduler simply serves as a shortcut. Following the object normally will still subscribe you to notifications about its task, provided your account settings allow for that.

You can enable notifications for your scheduled tasks directly from the Scheduler. Click the three dots icon next to an object with a scheduled task and select notifications.

You will see a popup, where you can decide which notifications you want to receive.

If your notification settings allow that, you will receive notifications about that task's errors or failures via email and Portal. The notification system and configuring your notifications is explained in more depth in our dedicated article.