Connect to BigQuery
BigQuery is Google's serverless data warehouse solution designed for analytics.
In this document, we will walk you through the steps to
- Create a service account for Holistics to interact with your BigQuery data
- Connect BigQuery to Holistics as a data source.
Permissions required for the Service Accounts
You can check this official docs from Google for the full list of BigQuery roles and permissions.
Holistics needs some of these permissions to run on your BigQuery data warehouse. For a quick start, we recommend you to create a Google service account with the following roles:
- BigQuery User (to provide permissions to run jobs)
- BigQuery Job User (to execute queries)
- BigQuery Data Viewer (to get table metadata)
Or you can create a custom role that includes the following permissions: bigquery.tables.getData, bigquery.jobs.create, and bigquery.tables.get.
- If you also want to import, transform data/persist your models to your BigQuery, Holistics will need permission to create/edit tables. One of these roles will suffice:
- BigQuery Data Editor
- BigQuery Data Owner
Steps
Step 1: Create a Service Account for Holistics
Go to Google Cloud Console page. Create a new project if you are starting from scratch, or select an existing project that already contains data you need to model.
From the left navigation bar, navigate to IAM & Admin > Service Accounts and click CREATE SERVICE ACCOUNT.
The service account allows you to control the features that Holistics have access to. Fill in the account details and click Create.
Next, click Select a role > BigQuery User to grant permission to the account.
More details about roles in Google's official docs.
After that, click Done to finish the account setup. You now have a service account for Holistics to interact with your project.
Next, at the service account you have just created, click on Actions > Create key to create a private key for the account. Key type should be JSON.
Download and store the JSON file in a safe location. This file will be used to connect your BigQuery account to Holistics.
Step 2: Add Google BigQuery data source to Holistics
From Holistics app's header bar, open Organization Settings menu -> Data Sources.
Click New Data Source button, and select Google BigQuery as the database type. Fill in the display name for the data source, the project ID of your BigQuery project. The project ID can be found in the JSON key file.
In the JSON credentials box, paste in the content of the JSON file downloaded in the previous step.
Click on Test connection to ensure the connection works, and then Save to finish.
Now you can start using Holistics to import, transform and model data in your BigQuery data warehouse.
Settings
Use Storage API
This feature is currently in development and will be released soon!
Using BigQuery Storage API allows Holistics to fetch query results upto 3 times faster.