The behavior you observed with your code using the Google Python SDK is implemented by Application Default Credentials (ADC).
There's another mechanism by which the credentials are discovered (which isn't covered in the ADC documentation) that's when gcloud auth application-default login
is used and when GOOGLE_APPLICATION_CREDENTIALS
is not used (and is thus unset) too.
This is likely the behavior you observed with the Python code. You'd gcloud auth application-default login
and not set GOOGLE_APPLICATION_CREDENTIALS
.
This behavior should be consistent across SDKs but using gcloud auth application-default login
is increasingly discouraged. It (possibly why it's not covered by the ADC documentation):
- (as you experience) can work "miraculously"
- the credentials are often too broad matching your user account's
On Linux, gcloud application-default login
creates ${HOME}/.config/gcloud/application_default_credentials.json
and this is the key that's used by the SDK.
I encourage you to create a Service Account to function as the identity for your code and to grant this the minimal set of IAM roles/permissions that your code needs.
When your code runs on a Google compute service (e.g. Compute Engine, Cloud Run, Kubernetes Engine...), the credentials are obtained automatically using Metadata service.
When your code runs off-GCP (e.g. locally), you can create a Service Account key and export GOOGLE_APPLICATION_CREDENTIALS=/path/to/key.json
and SDK-based code will use these automatically.
NOTE For completeness, creating Service Account keys is also discouraged. Keys are secrets and should be treated judiciously. A recent, powerful mechanism that provides access to GCP resources for code running off-GCP is to use Workload Identity Federation