Questions tagged [amazon-s3]
Amazon Simple Storage Service. For questions about storing and retrieving DevOps-related data in S3, or using S3 for building a code pipeline.
51
questions
0
votes
0answers
7 views
why is a PHP client trying to do unauthorized REST.POST.OBJECT to my S3 bucket?
I've written a logfile analysis program for my S3 logs and I'm now looking in detail at every line. I've seen some odd behavior lately. I'm trying to figure out
...
0
votes
1answer
19 views
AWS S3, as an organizer see empty bucket
I am an owner of organizer in on my aws account. When I enter S3 I see one folder, which is empty. But its being billed 8$ per month, so smth should be there.
Is it possible that one account, of my ...
0
votes
1answer
36 views
Migrating to AWS with Devops [closed]
What are they key stages of migrating to AWS with Devops ?
Can anyone list a use case or workflow?
0
votes
0answers
11 views
How would I automate the deployment of an S3 Bucket file to a remote external server which exists on the other side of an AWS VPN Tunnel?
So my current AWS VPC consists of an EC2 jump server which is on the cloud side of a site-to-site VPN tunnel and the other side is connected to a customer server. So I am trying to find out how I can ...
3
votes
1answer
57 views
auto delete aws S3 backups
I need to automate deletion of aws S3 backups ... evidently if I tag the backup with one of daily, weekly, monthly, yearly then aws will delete based on my desired retention counts per those periods ...
1
vote
1answer
395 views
Troubleshooting VPC flow logs with an S3 bucket using SSE-KMS encryption with CMK
I'm using Terraform and trying to set up automatic export of VPC flow logs into an S3 bucket in the same AWS account and region (ca-central-1) that has default encryption turned on with AWS-KMS (using ...
1
vote
2answers
26 views
Is there a policy that can limit the growth of an AWS S3 buckets total size?
We have an AWS S3 bucket that accepts uploads from untrusted sources which are then processed and moved out of the bucket.
Is there a Policy that can limit the growth of the bucket to some ...
1
vote
1answer
24 views
How to make different permissions for different subdirectories of storage bucket
On Google Cloud Storage or Amazon S3, is there a way to give users a key to only a specific subdirectory of a storage bucket, for example:
gs://images.eng.cm.com/avatars/<user-uuid>/xyz
so the ...
1
vote
1answer
340 views
Error when creating new recource using backend AWS s3 [closed]
When I try create a new resource using terraform apply (Terraform init was success) received an error:
data.terraform_remote_state.environment_data: Refreshing state...
Error: Unable to find remote ...
1
vote
0answers
54 views
Variables in Amazon S3 redirect rules
I'm trying to configure path redirect for Amazon S3 bucket static content ( → Properties → Static website hosting → Redirection rules).
Simple rules, such as in instruction, work properly.
Is it ...
1
vote
0answers
15 views
What is the term for releasing all builds to S3 - and then a 'release' changes the version configuration pointer?
We have a JS build system where every build is released to S3. There is a separate 'bootstrap configuration' file in a different S3 folder that points to the current version in use for that ...
1
vote
2answers
281 views
s3fs to mount S3 bucket with iamrole on non-aws machine
I am trying to mount S3 bucket on my on-premises CentOS-6 server using s3fs as below:
[roots@myhost ~]# s3fs bucket-name /mnt/s3mnt -o uid=1001,gid=1001,allow_other,umask=022,iam_role=iam-role,...
2
votes
0answers
278 views
Allowing all apps/users access to folder - S3FS-FUSE
I'm running an application on AWS EC2 Ubuntu instance, here's what I've done:
- I've mounted S3 bucket as folder with s3fs-fuse using this command (unfortunately as root : sudo):
sudo s3fs ...
1
vote
1answer
281 views
How to run aws cli with www-data?
I can successfully upload a file to my S3 bucket using the AWS cli version 1 with my normal linux user. However, I'm having trouble doing the same with Apache2's www-data.
I have a personal project ...
2
votes
1answer
361 views
For storing avatars in s3, which approach has the lowest cost?
On my site I have the ability for users to upload avatars (which they upload to s3). They can only have one avatar at the moment, so I currently just have the filename set to their user id.
However, ...
0
votes
1answer
295 views
S3 bucket policy errors
I have the following bucket policy for Amazon S3, which is giving the following error:
Error: Invalid bucket policy syntax.
Either it is a JSON syntax error or a Policy error.
This is the policy ...
4
votes
1answer
135 views
Triggering a SNS notification when an object is accessed (downloaded) in AWS S3
I have managed to trigger events on my bucket when objects are uploaded (PUT) but I am surprised that there are no triggers called when objects are downloaded (GETs), as can be seen in this screenshot ...
1
vote
3answers
74 views
How to make an AWS S3 bucket accessible over HTTP?
Does AWS S3 provide a way to make bucket's files available over HTTP preferably protected with BASIC_AUTH? I mean listing the files (index) and make it possible to download them over HTTP.
0
votes
1answer
14 views
s3 static hosting not redirecting my request
I followed these instructions to redirect:
platform.demo.interos.net
to
platform.demo.interos.io
The instructions use s3 for static site hosting and use the redirect option: https://aws.amazon....
2
votes
4answers
5k views
Is it possible to mount an s3 bucket as a point in a docker container?
I have a Java EE packaged as war file stored in an AWS s3 bucket. I will like to mount the folder containing the .war file as a point in my docker container. I have managed to do this on my local ...
1
vote
1answer
146 views
How can I 'aws s3 sync' two buckets, which are located in different accounts
I'm trying to use 'aws s3 sync' on the awscli between two accounts.
Account A, I own.
Account B, Owned by a third party.
Account B has given a user:jon on account A permission to a bucket through ...
1
vote
1answer
2k views
Amazon S3 bucket does not redirect folder URLs to index.html
I have uploaded my HUGO static site into AWS S3 Bucket. I have also added the domain in cloudrfront.
The home index.html page works fine, but I cannot access my resources.
For example https://www....
1
vote
1answer
143 views
Why isn't my S3 Cross Region Replication working for some files?
I've followed the documentation to set up cross-region replication to another AWS account. It looked like it was working correctly until I noticed an entire prefix was missing.
Here is my ...
1
vote
1answer
116 views
How to override default aws region credentials for Terraform?
I am using Terraform to create aws resources. I practiced with a single region and with my aws cli credentials being used. worked great!
however when I tried to override the region in the Terraform ....
1
vote
1answer
831 views
Artifacts mentioned in secondary-artifacts section not getting uploaded to s3 bucket
I have created a codePipeline project through which my application get build using maven and it's artifacts are uploaded to s3. I'm trying to upload multiple artifacts as a single zip file to s3 ...
0
votes
1answer
131 views
compiled opencv copy to another host
Every time I spawn a new host I need to compile OpenCV on Ubuntu following official manual. The problem is that it takes about 20-30 minutes depends on the VM type. Is it somehow possible to store the ...
0
votes
1answer
463 views
Can Terraform S3 remote state be kept in a different AWS account?
I am currently using Terraform with S3 remote state. The IAM user that I'm using to run Terraform exists in the same AWS account as the S3 bucket holding the remote state files. This is all working ...
3
votes
1answer
2k views
Is it necessary to specify resource in S3 Bucket Policies?
Is it necessary to specify a resource attribute within the statements that make up an S3 bucket policy? The Access Policy Language Overview seems to suggest that the resource attribute is always ...
4
votes
2answers
503 views
Impact on policy after deleting IAM user
I have created one IAM User named "test" with S3 and Glacier Permission.
Now with user "test" i created one life cycle policy to move data from S3 to Glacier at every 5 days.
Now, i delete IAM user "...
1
vote
1answer
316 views
How to Allow AWS S3 Access From only Specific IPs
Recently we were testing with AWS VPC, and a requirement for our project was that we needed to allow nodes within a VPC access to S3 buckets, but deny access from any other IP address.
Is there ...
2
votes
1answer
4k views
Access S3 files from Datacenter over VPN to VPC
I am trying to access S3 files from non-EC2 machine. non-EC2 machine is connected to VPC over VPN, also can reach to other hosts in VPC.
Created VPC Endpoint for S3
Main routing table of VPC has ...
5
votes
3answers
1k views
How to monitor AWS S3 cross-region replication process?
Amazon S3 has a cross-region replication which will handle copy of new/updated objects to additional region.
The problem is that solution does not provide visibility on state for replication process, ...
3
votes
1answer
5k views
How to “inject” AWS Lambda code into a CloudFormation?
I'm exercising an idea of not using S3 bucket to refer to the code of a lambda function.
The usual example of a lambda CloudFormation code might look like:
MyLambda:
Type: AWS::Lambda::...
1
vote
1answer
2k views
Different download speeds from AWS S3 CLI vs Console/Websites
I'm experiencing much slower traffic for the exact same files when downloading (or uploading) them via the AWS website vs. downloading them via the CLI.
Is there a known reason for this?
Is there ...
1
vote
0answers
25 views
Converting s3 URL to the desired one [closed]
I have created a s3 bucket with all permissions and user access through IAM policies. I give the below URL to the customer to access that bucket
https://s3.console.aws.amazon.com/s3/buckets/s3-upload-...
0
votes
1answer
51 views
Copying a file with special character & spaces
I have been trying to copy a file from my remote server to AWS s3, but the file name is quite weird and I am not able to figure it out what can I do. I tried using escape characters but that did not ...
-2
votes
1answer
255 views
Amazon AWS - Amazon Connect IAM policies
I was trying to lock down user account just to Amazon Connect, Lambda and S3. I was able to give permissions to S3 and Lambda but I cannot find permissions for Amazon Connect. I have went through the ...
2
votes
1answer
42 views
Is it possible to load an image into S3 via an URL?
I'm going through some steps documented in How Do I Upload Files and Folders to an S3 Bucket? to get my ova into S3. Seems like a waste of time for me to download it and then upload it, is there a ...
5
votes
1answer
5k views
How to apply using Terraform to launch multiple EC2 Resource with different configs (VPC not maintained by TF)
We are planning to use Terraform for provisioning EC2 instance which will host Oracle DB. While we are able to achieve this, there are some questions for which I am searching for suggestions / ...
1
vote
1answer
38 views
AWS S3 ACLs across accounts
I have two AWS accounts: A and B (viewable from https://console.aws.amazon.com/organizations). I have given an S3 bucket in account A a policy that allows account B to do everything in that bucket. ...
7
votes
3answers
2k views
Best practices for external backups for S3 buckets?
I would like to know how everyone is making external backups of S3 buckets and best practices/recommendations for this. Regardless of how good AWS is it is still a single point of failure.
Especially ...
2
votes
1answer
450 views
AWS Redshift UNLOAD not firing S3 PUT event
I am trying to use Redshift to UNLOAD data to S3 and then pick that data up with a Lambda function and process it. All of the constituent parts (the UNLOAD, the Lambda function) now work, however the ...
12
votes
4answers
28k views
View all AWS S3 buckets and list each buckets storage used
I have multiple AWS accounts and I need to list all S3 buckets per account and then view each buckets total size.
Currently, I can only view the storage size of a single S3 bucket with:
aws s3 ls ...
8
votes
5answers
5k views
How to provide a user to access only a particular bucket in AWS S3?
I have a list of buckets in AWS S3. I have created an IAM user. I have an option to provide S3 full or read only access for a user using groups. Is there any options to provide access only to a ...
4
votes
1answer
542 views
AWS Storage Cost for a custom AMI
How does AWS calculate the cost of a custom AMI?
If I would create an AMI which is based on Ubuntu 16.04, do I need to pay for the complete size of AMI (e.g. 4GB) or just the differences of my custom ...
10
votes
2answers
2k views
Can you use `aws-sdk` to store objects in Google Storage?
Google Storage claims they support interoperability in their solution with Amazon AWS S3 and Eucalyptus Systems APIs.
source: https://cloud.google.com/storage/docs/interoperability
Can the aws-sdk ...
9
votes
1answer
727 views
Is there any way I can rotate AWS KMS keys every time interval < 1 year?
According AWS KMS, keys can be rotated once every year. However, there is no way I can define a custom time period for that.
However, I can keep changing the same in the bucket's policy manually.
So,...
19
votes
2answers
3k views
Is cross-region replication 100% fool-proof for S3 region outages? [duplicate]
Amazon S3 has an option of cross-region replication which should be pretty fault-tolerant against region/zone outages.
Does that mean those who are ranting about the outage did not make use of this ...
21
votes
1answer
481 views
How do I work around the fact that AWS SQS is not HIPAA compliant?
I have a use case where data from S3 is queued into AWS SQS, which is in turn connected to CloudWatch, whose metrics will be triggering AWS Lambda.
However, I want the architecture to be HIPAA ...
33
votes
2answers
610 views
What is a good strategy to keep my site online when S3 goes offline?
What is a good strategy to keep my site online when S3 goes offline?
If S3 US East 1 goes offline, how should I have my app configured/structured to prevent that taking my entire site offline?
What ...
