Connecting to Google BigQuery
This article explains how to connect Dataedo to Google BigQuery and import metadata - tables & columns.
Prerequisites
Service Account
Dataedo connects to Google BigQuery with Service Account. If you don't have one, please read carefully the following Google Cloud docs:
During the creation (or modification) of the service account, you need to specify roles which will allow Dataedo to document the BigQuery database. Dataedo reads INFORMATION_SCHEMA views to collect metadata, hence minimum permissions (roles) that the service account needs to have are:
- BigQuery Metadata Viewer - to read metadata,
- BigQuery Job User - to run a query that reads metadata.
Service account key
To authenticate to Google BigQuery with a service account, you need a Service Account Key. To get one, you need to find the Service Account resource, open the Service Account that will be used to connect from Dataedo, go to the Keys tab and click Add Key -> Create new Key button. Then select JSON format and download the key.
Cloud Resource Manager API
In order to document the BigQuery database, Dataedo needs to access the Cloud Resource Manager API which needs to be enabled first. Most likely this feature will be enabled, however, if it is disabled, you can activate it by searching Cloud Resource Manager API in Google Marketplace and then clicking the Enable button.
Add new connection
To connect to Google BigQuery create new documentation by clicking Add documentation and choosing Database connection.
On the Add documentation window choose Google BigQuery:
Connection details
Provide connection details:
- Service Access Key - path to Service Account Key. You can open file explorer to browse for the file by clicking the [...] button,
- Project - select the Google Cloud project under which your BigQuery database was created. You can expand the list of available projects by clicking the [...] button,
- Dataset - select one or more datasets to document. You can expand the list of datasets by clicking the [...] button.
Importing metadata
When the connection is successful, Dataedo will read objects and show a list of objects found. You can choose which objects to import. You can also use advanced filter to narrow down the list of objects.
Confirm the list of objects to import by clicking Next.
The next screen allows you to change the default name of the documentation under which it will be visible in the Dataedo repository.
Click Import to start the import.
When done, close the import window with the Finish button.
Outcome
Your database schema has been imported to new documentation in the repository.
