Error Setup Google Cloud Credentials in Dockerfile

Question

i have error like this (i assume this error bcs iam not properly set the google credentials in my dockerfile and .py file)

#12 3.678   File "/app/test_docker/pipeline_v1_test.py", line 334, in <module>                                                                                                                 
#12 3.678     pipeline_roboguru_question_clustering()
#12 3.678   File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1128, in __call__
#12 3.678     return self.main(*args, **kwargs)
#12 3.678   File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1053, in main
#12 3.678     rv = self.invoke(ctx)
#12 3.678   File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1395, in invoke
#12 3.678     return ctx.invoke(self.callback, **ctx.params)
#12 3.678   File "/usr/local/lib/python3.7/site-packages/click/core.py", line 754, in invoke
#12 3.678     return __callback(*args, **kwargs)
#12 3.678   File "/app/test_docker/pipeline_v1_test.py", line 42, in pipeline_roboguru_question_clustering
#12 3.678     df_raw = util.get_data_from_bq(project_id=config['PROJECT_ID_PROD'], query=query.query_data_raw)
#12 3.678   File "/app/test_docker/util.py", line 21, in get_data_from_bq
#12 3.678     client = bigquery.Client(project=project_id)
#12 3.678   File "/usr/local/lib/python3.7/site-packages/google/cloud/bigquery/client.py", line 235, in __init__
#12 3.678     _http=_http,
#12 3.678   File "/usr/local/lib/python3.7/site-packages/google/cloud/client/__init__.py", line 320, in __init__
#12 3.678     self, credentials=credentials, client_options=client_options, _http=_http
#12 3.678   File "/usr/local/lib/python3.7/site-packages/google/cloud/client/__init__.py", line 178, in __init__
#12 3.678     credentials, _ = google.auth.default(scopes=scopes)
#12 3.678   File "/usr/local/lib/python3.7/site-packages/google/auth/_default.py", line 544, in default
#12 3.678     credentials, project_id = checker()
#12 3.678   File "/usr/local/lib/python3.7/site-packages/google/auth/_default.py", line 537, in <lambda>
#12 3.678     lambda: _get_explicit_environ_credentials(quota_project_id=quota_project_id),
#12 3.678   File "/usr/local/lib/python3.7/site-packages/google/auth/_default.py", line 219, in _get_explicit_environ_credentials
#12 3.678     os.environ[environment_vars.CREDENTIALS], quota_project_id=quota_project_id
#12 3.678   File "/usr/local/lib/python3.7/site-packages/google/auth/_default.py", line 118, in load_credentials_from_file
#12 3.678     "File {} was not found.".format(filename)
#12 3.678 google.auth.exceptions.DefaultCredentialsError: File key.json was not found.

my code in dockerfile is like this

FROM python:3.7.11

WORKDIR /app
COPY . /app/test_docker

RUN pip3 install --upgrade pip
RUN pip3 install -qr /app/test_docker/requirements.txt

RUN export GOOGLE_APPLICATION_CREDENTIALS=key.json

RUN python3 /app/test_docker/pipeline_v1_test.py --env DEV

and i have setup the credentials like this in my .py file

import os
os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = 'key.json'

please help :D

Answer 1

The RUN export does not work in this case you should use "ENV", the rest should be ok!

WORKDIR /app
COPY . /app/test_docker

RUN pip3 install --upgrade pip
RUN pip3 install -qr /app/test_docker/requirements.txt

---> ENV GOOGLE_APPLICATION_CREDENTIALS=key.json

RUN python3 /app/test_docker/pipeline_v1_test.py --env DEV