Google Cloud Datastore Authentication on Windows 10

Question

I'm setting up a windows 10 vm that has internet access (I browse the internet just fine on chrome inside the vm). I downloaded and installed the google cloud SDK. I ran gcloud init, and set up my account and other options. It said it completed successfully. I type gcloud info and it shows the correct information. The problem is, is when I run a python script I wrote that includes from google.cloud import datastore. It has an issue with account authentication. This error only occurs when I try to instantiate the datastore client as so: client = datastore.Client(). Which makes sense of course. However, because I already performed the gcloud init and verified it with gcloud info, what is not set up here that needs to be?

Additionally, it looks like the google auth library is not able to make a request to a certain server properly. Here's the log output:

2020-09-27 20:26:30 [google.auth._default] DEBUG: Checking None for explicit credentials as part of auth process...
2020-09-27 20:26:30 [google.auth._default] DEBUG: Checking Cloud SDK credentials as part of auth process...
2020-09-27 20:26:30 [google.auth._default] DEBUG: Cloud SDK credentials not found on disk; not using them
2020-09-27 20:26:30 [google.auth._default] DEBUG: Checking for App Engine runtime as part of auth process...
2020-09-27 20:26:30 [google.auth._default] DEBUG: No App Engine library was found so cannot authentication via App Engine Identity Credentials.
2020-09-27 20:26:30 [google.auth.transport._http_client] DEBUG: Making request: GET http://169.254.169.254
2020-09-27 20:26:33 [google.auth.compute_engine._metadata] WARNING: Compute Engine Metadata server unavailable onattempt 1 of 3. Reason: timed out
2020-09-27 20:26:33 [google.auth.transport._http_client] DEBUG: Making request: GET http://169.254.169.254
2020-09-27 20:26:36 [google.auth.compute_engine._metadata] WARNING: Compute Engine Metadata server unavailable onattempt 2 of 3. Reason: timed out
2020-09-27 20:26:36 [google.auth.transport._http_client] DEBUG: Making request: GET http://169.254.169.254
2020-09-27 20:26:39 [google.auth.compute_engine._metadata] WARNING: Compute Engine Metadata server unavailable onattempt 3 of 3. Reason: timed out
2020-09-27 20:26:39 [google.auth._default] WARNING: Authentication failed using Compute Engine authentication due to unavailable metadata server.

The actual error at the end of the callstack is:

  File "C:\Users\<user>\anaconda3\lib\site-packages\google\auth\_default.py", line 354, in default
    raise exceptions.DefaultCredentialsError(_HELP_MESSAGE)
google.auth.exceptions.DefaultCredentialsError: Could not automatically determine credentials. Please set GOOGLE_APPLICATION_CREDENTIALS or explicitly create credentials and re-run the application. For more information, please see https://cloud.google.com/docs/authentication/getting-started

Edit: Note that this IP address listed in the log is mentioned here: https://serverfault.com/questions/427018/what-is-this-ip-address-169-254-169-254/427022 It looks like its a local only ip address. But that's strange because this VM has a DHCP assigned IP address and isn't even the same as the host machine IP. I'm using virtualbox. I've tried two network type settings: Bridged Adapter and NAT. Both don't work here. Is this the fault of VM networking or something else I don't realize?

Answer 1

I think in this situation you should follow the error message and add GOOGLE_APPLICATION_CREDENTIALS with path to your private key. How to get the key is in the link of the error.

The same approach you may find in Datastore documentation Accessing your database from another platform.

As well as in How-to guide Getting started with the Firestore in Datastore mode API, that contain similar example to yours you will find:

For this example, you'll fetch Application Default Credentials from the environment, and pass it as an argument to create the service object.

The fact that you are logged to gcloud does not mean that you are always authenticated on the particular machine. When you are running python script, you are running different app than gcloud. This means that you have to do authenticate.

Answer 2

For me, (as this was a new machine, and I already had all of the python code and gcp account settings working on a previous machine). The problem was after the gcloud init I also needed to run gcloud auth application-default login as well before my stuff started to work.