What is the proper AWS IAM policy for access to a single bucket that allows stacking?

Question

I am working on a buildout where I am doing the following. Eventually I will need to make this scalable so I am not doing this manually each time.

I have several clients dumping data into S3 buckets. Each client needs R/W access to just that bucket. For that, I can use this policy

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "BucketOperations",
            "Effect": "Allow",
            "Action": "s3:ListBucket*",
            "Resource": "arn:aws:s3:::<bucketname>"
        },
        {
            "Sid": "ObjectOperations",
            "Effect": "Allow",
            "Action": [
               "s3:AbortMultipartUpload",
               "s3:DeleteObject*",
               "s3:GetObject*",
               "s3:PutObject*"
            ],
            "Resource": "arn:aws:s3:::<bucketname>/*"
        },
        {
            "Sid": "DenyAllOthers",
            "Effect": "Deny",
            "Action": "s3:*",
            "NotResource": [
               "arn:aws:s3:::<bucketname>",
               "arn:aws:s3:::<bucketname>/*"
            ]
        }
    ] 
}

On my end, I have several analysts that are using Athena to query the data via Tableau and other tools. I have a Glue job to index the source bucket, and each client dataset will need a Athena results bucket so I can maintain appropriate access.

My problem is that I don't know how to write the S3 policy so that I can have one-per-bucket, and then attach them to the IAM users. When I use the above policy to create

• bucket1-policy
• bucket2-policy

And then add them to an IAM user, eg

• bucket1-policy
• bucket2-policy
• my-athena-policy
• ...

it looks like DenyAllOthers parts are causing the user to not have S3 access.

Is there a better way to write the S3 access policy, so I can have one per bucket, or do I need to have a separate policy for my Athena users to grant access to the two buckets and then deny the rest?

Answer 1

Let's start at the end: it looks like DenyAllOthers parts are causing the user to not have S3 access.

Indeed:

By default, all requests are denied. (In general, requests made using the AWS account root user credentials for resources in the account are always allowed.)

An explicit allow in any permissions policy (identity-based or resource-based) overrides this default.

The existence of an Organizations SCP, IAM permissions boundary, or a session policy overrides the allow. If one or more of these policy types exists, they must all allow the request. Otherwise, it is implicitly denied.

An explicit deny in any policy overrides any allows.

https://docs.aws.amazon.com/IAM/latest/UserGuide/intro-structure.html

That means that it is OK to not have any Deny rule. That would work for the customers.

I would suggest Deny s3:PutObjectAcl with anything else then bucket-owner-full-control - see an example here https://docs.aws.amazon.com/AmazonS3/latest/userguide/about-object-ownership.html

An alternative - just disable public access on the whole bucket with PublicAccessBlockConfiguration https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-s3-bucket-publicaccessblockconfiguration.html

Speaking of that - creating accounts for the users via CloudFormation template allows you to pass the bucket name as a parameter. That way you can manage many users even with just PowerShell / unix sh or similar. You would have one template in a repo and you can change the permissions on a list of users with a simple loop. Terraform (suggested by one of the comments) or any other automation tool is also an option, usually the tools do use CloudFormation under the hood, so it may be still a good idea to understand CloudFormation just in case somethings does not work and you have to investigate. Or to made sure that some configuration option of your automation tool did not allow much broader permissions that you assume.

I also suggest to name the buckets in a clever way. If you name the upload buckets for example upload- you can allow read access to all the upload buckets by one rule with arn:aws:s3:::upload-*/*.

For your your analysts - create a group for them, add the permissions to the group.

It is good to know that you can put the IAM policy either on a bucket or on an identity (role/user/group...). Sometimes it does not matter, sometimes it does. If you ever need to allow access to a customer based on their AWS account, you will need to use a resource policy on the bucket. On the other side - maintaining a complex resource policy with many updates may be challenging. As an alternative you can also allow to assume roles.