I am creating a design proposal for a data lake hosted in S3. My goal is to create a flat data lake that will host a variety objects within it. The majority of these objects will be made available to the general public, but some will be for internal use only. My proposal is to have a bucket policy that allows only public access for items with a public=true tag implemented. However, this tag should only be able to be changed by users with the correct privileges. Is this something that is possible in S3 with IAM, and if it isn't, what other options that can be utilized to control private/public objects in an S3 data lake?

  • 1
    An "IAM role that allows only public access" is a contradiction in terms. In S3, "public" means no authentication is required. Perhaps instead of an IAM role, you're thinking of a bucket policy? – Michael - sqlbot May 7 '18 at 16:34
  • You are correct. Thanks @Michael-sqlbot! I've updated the question accordingly – PrestonM May 7 '18 at 21:33
  • 1
    That should be possible, though objects that have their object ACL set to public are automatically public, even if the bucket itself is not, and access to that permission can also be set through policy. Have you overlooked this option or decided it isn't what you want? – Michael - sqlbot May 7 '18 at 22:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.