How to connect apache superset directly with google BigQuery?

Question

I am running Apache superset on GCP instance and it works fine with Sqlite database which is default in superset and I don't need to configure so many things. But my requirement is that I need superset to connect directly with BigQuery instead of Sqlite and I don't have developer background. So, is there an easy way to do that without heavy codes?

Answer 1

Following the steps mentioned at the official Google Cloud page here, you need to do the following

1. Install pybigquery

pip install pybigquery

2. Download your Google Cloud authorization json key file
3. From your terminal instance, set GOOGLE_APPLICATION_CREDENTIALS env. var to the path of your json key file

export GOOGLE_APPLICATION_CREDENTIALS="/path/to/[json_file].json"

Answer 2

Connecting to BigQuery is very well documented here in Preset's Superset user documentation https://docs.preset.io/docs/big-query-database

Answer 3

Elbehery is right. I don't have enough rep to comment, but I wanted to note that Apache has created docs for this.

FWIW, I couldn't use the UI for importing credentials.json so I set it as an env var in my Docker image. Here are the commands and steps I run locally:

# Setup virtual environment (exit by typing "deactivate")
pip3 install virtualenv
python3 -m virtualenv ./.venv
source ./.venv/bin/activate
?
# Download Superset
git clone https://github.com/apache/superset.git
cd superset/
?
# Create a copy of your credentials for docker to use
cp ~/.config/gcloud/application_default_credentials.json docker/credentials.json
echo "GOOGLE_APPLICATION_CREDENTIALS=docker/credentials.json" >> docker/.env-non-dev
?
# Run Superset
docker-compose -f docker-compose-non-dev.yml pull
docker-compose -f docker-compose-non-dev.yml up
?

Now that Superset is running locally:

• visit http://0.0.0.0:8088/ in your web browser
• In the top right of UI, click the +DATABASE button (or + > Data > Connect Database)
• In popup window Click Supported Databases then (at the bottom) Other
• Set DISPLAY NAME: BigQuery
• Set SQLALCHEMY URI: bigquery://my_project_id
• Click Test Connection
• Click Connect ? Now that BigQuery is integrated into Superset:
• In the top of page Click SQL Lab > SQL Editor