0

I am trying to use the workflow orchestrator Snakemake with a Google Kubernetes Engine (GKE) autopilot cluster. My workflow needs to upload files to Google Cloud Storage buckets within the same project. Both Snakemake and GKE documentations specify that a Google Storage read-write scope should be specified when the cluster is created, via --scopes storage-rw.

I tried doing so with an autopilot cluster:

gcloud container --project "project-name" clusters create-auto "default-cluster" \
    --region "asia-southeast2" --release-channel "regular" \
    --network "projects/project-name/global/networks/default" \
    --subnetwork "projects/project-name/regions/asia-southeast2/subnetworks/default" \
    --cluster-ipv4-cidr "/17" --services-ipv4-cidr "/22" --scopes storage-rw

(I generated this command using the Google Cloud web console, then added the --scopes storage-rw argument at the end.)

But I still get a Snakemake error message related to lack of permission for Google Storage read-write:

google.api_core.exceptions.Forbidden: 403 GET https://storage.googleapis.com/storage/v1/b/kubernetes-nccs-npc2/o/whoami.txt?fields=name&prettyPrint=false: Caller does not have storage.objects.get access to the Google Cloud Storage object.

It seems --scopes storage-rw does not work for the creation of autopilot clusters. Is this true? If so, how can I grant Google Storage read-write access to an autopilot cluster?

1
  • were you able to find a solution?
    – Mohsin
    Nov 23, 2022 at 22:44

1 Answer 1

0

I think the error has more to do with the service account permissions running your cluster. I'd make sure all the permissions are included, like storage.ojjects.get. The error seems very permissions related.

Caller does not have storage.objects.get access to the Google Cloud Storage object.

You might be hitting this issue Google has reported:
https://cloud.google.com/kubernetes-engine/docs/how-to/access-scopes

I'd listen to their recommendations over mine.

When managing IAM roles, you can treat a service account either as a resource or as an identity. This command is to get the iam policy of a service account resource. There are other gcloud commands to manage IAM policies for other types of resources. For example, to manage IAM policies on a project, use the $ gcloud projects commands and for services use $ gcloud iam service-accounts.

This will return your service running the cluster w/ role assignment (it's the robot one :) )

gcloud projects get-iam-policy <PROJECT-NAME> 

The problem is sometimes running these commands requires special permissions too, so it becomes fools progress. I think this link may be helpful. it shows how to set permissions on a service-account.

https://cloud.google.com/sdk/gcloud/reference/iam/service-accounts/add-iam-policy-binding

Also here is Google dictionary of IAM permissions including storage.ojjects.get.

https://cloud.google.com/iam/docs/understanding-roles

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.