BigQuery

BigQuery is a Bigdata solution created by Google that allows the storage and analysis of data on a large scale. The data integration from BigQuery to the Data Warehouse created by Kondado allows you to access your BigQuery data in your analytical cloud.

Adding the data source

To automate the ETL from BigQuery with Kondado to your database, follow the steps below:

  1. Log in to the Kondado platform, go to the page to add data sources and select the Google BigQuery data source
  2. Click on AUTHORIZE
  3. Select the account you will use
  4. In the next screen, check ALL the permissions that are requested and click Continue
  5. You will be redirected to the Kondado page with access already authorized. Follow the rest of the step-by-step to find the rest of the necessary information
  6. Access your BigQuery console in GCP and copy the project IDs (in red in the figure below) and the dataset (in green)
  7. Again on the page from step (5), paste the project and dataset values obtained in step (6) and give a name to your data source

bq_ds_project.jpeg

ATTENTION: Make sure that the Dataset ID does NOT contain the project ID separated by ":". For example, "project_id:dataset_id" is wrong. The correct way is for the project ID and Dataset ID to be each separately in their respective fields

Now just save the data source and start integrating your BigQuery data into the Data Lake or Data Warehouse.

Don't forget to click on "TEST CONNECTION" to check if everything is right!

Tables and Views

With our integration, you will be able to integrate tables and also views

If your table has a datetime/timestamp type column that marks when a record was changed/inserted, your integration can be incremental

It will be necessary to define the primary key, which can be defined by several columns and refers to the column (or set of columns) that define a record as being unique

 

Add BigQuery as a Data Source on Kondado

Connect your Google BigQuery project to Kondado's data warehouse for automated ETL and analytics.

1
Log in and select BigQuery source

Access the Kondado platform, navigate to the data sources page, and choose the Google BigQuery connector to begin setup.

2
Authorize Google account access

Click AUTHORIZE, select your Google account, then grant ALL requested permissions on the consent screen and click Continue to proceed.

3
Copy project and dataset IDs from GCP

Open your BigQuery console in Google Cloud Platform, then copy the project ID and dataset ID values shown in your console interface.

4
Paste IDs and name your source

Return to Kondado, paste the project ID and dataset ID into their separate fields (never combine with ":"), give your source a descriptive name, then save.

5
Test connection and start integrating

Click "TEST CONNECTION" to verify everything works correctly, then begin integrating your BigQuery tables and views into your data integration pipeline.

Frequently asked questions

What is BigQuery and why connect it to Kondado?
BigQuery is Google's large-scale data storage and analytics solution. Connecting it to Kondado lets you automate ETL and access your BigQuery data in Kondado's analytical cloud for unified analysis.
Can I integrate both tables and views from BigQuery?
Yes, Kondado's BigQuery integration supports both tables and views, giving you flexibility in how you structure your data pipeline.
How do I make my BigQuery integration incremental?
Your table needs a datetime/timestamp column that records when data was changed or inserted. You'll also need to define a primary key, which can span multiple columns to uniquely identify each record.
What permissions do I need to grant during authorization?
You must check ALL permissions requested on the Google consent screen. These allow Kondado to securely access and sync your BigQuery datasets on your behalf.
Why is my connection failing when I paste the dataset ID?
Make sure the Dataset ID does NOT contain the project ID separated by ":". The project ID and dataset ID must each go in their own separate fields—"project_id:dataset_id" format will cause errors.
How can I verify my BigQuery source is configured correctly?
Always click "TEST CONNECTION" after saving your data source. This validates that Kondado can successfully reach your BigQuery project with the provided credentials.

Written by·Published 2023-08-02·Updated 2026-04-25