29
votes
Accepted
What is a good strategy to keep my site online when S3 goes offline?
In March 2015, Amazon AWS announced they support S3 replication across regions. When a certain region in S3 goes offline, you can serve files from your mirror in another region.
source: https://aws....
- 8,963
20
votes
Accepted
View all AWS S3 buckets and list each buckets storage used
Resolution 1
So I solved this with the following script. I originally posted the question just in case there was an easier way that I was not aware of.
#!/bin/bash
aws_profile=('profile1' 'profile2' '...
- 1,152
19
votes
Accepted
How do I work around the fact that AWS SQS is not HIPAA compliant?
According to Amazon AWS
Customers may use any AWS service in an account designated as a HIPAA
account, but they should only process, store and transmit PHI in the
HIPAA-eligible services ...
- 8,963
12
votes
Accepted
How to order physical backup of S3 data?
AWS Import/Export is a service that accelerates transferring data into and out of AWS using physical storage appliances, bypassing the Internet. AWS Import/Export Disk was originally the only service ...
- 8,963
11
votes
What is a good strategy to keep my site online when S3 goes offline?
What you are asking for is, basically, high-availability. In order to make a system highly available, you need three things:
Eliminate single points of failure
A mechanism to switch from an endpoint ...
- 1,955
11
votes
Accepted
Is cross-region replication 100% fool-proof for S3 region outages?
The drawback when there's replication come from the note below:
Amazon S3 routes any virtual hosted–style requests to the US East
(N. Virginia) region by default if you use the US East (N. ...
- 11.3k
10
votes
Accepted
How to provide a user to access only a particular bucket in AWS S3?
Amazon's IAM roles generally grant a role access to a particular ARN (Amazon Resource Name). Amazon notes on their pages that for S3 a resource
...can be a bucket-name or a bucket-name/object-key.
...
- 3,724
10
votes
Is cross-region replication 100% fool-proof for S3 region outages?
Many big companies would be at fault for not using this feature. It does add additional cost, and historically any kind of real disaster recovery solution is untested even if implemented.
Other than ...
- 8,963
8
votes
Accepted
Is there any way I can rotate AWS KMS keys every time interval < 1 year?
The AWS documentation is clear that automated key rotation is only supported by Amazon on a yearly basis. However, you can use the CLI to manually create new keys, and then use an automation technique ...
- 1,279
8
votes
Can you use `aws-sdk` to store objects in Google Storage?
Yes, you can use the AWS SDK to store objects within google storage.
From Simple migration:
In a simple migration from Amazon S3 to Google Cloud Storage, you can
use your existing tools and ...
- 91
7
votes
Can you use `aws-sdk` to store objects in Google Storage?
Most of the AWS SDKs mentioned support constructing a custom endpoint, for example as described here for Node.JS.
Setting the endpoint to storage.googleapis.com and using your GCP access credentials ...
- 366
6
votes
Accepted
How to apply using Terraform to launch multiple EC2 Resource with different configs (VPC not maintained by TF)
You can use a remote backend as a data source. That is working well for us thus far but this project is not very mature and we will likely refactor a time or ten.
data "terraform_remote_state" "...
- 215
6
votes
How to order physical backup of S3 data?
I think you want Amazon Snowball, it's a HD shipping service by Amazon to/from AWS.
Snowball is a petabyte-scale data transport solution that uses secure appliances to transfer large amounts of data ...
- 1,241
5
votes
View all AWS S3 buckets and list each buckets storage used
You will need to write a script that parses and queries this data because as far as I'm aware, there is not a tool or cli function that performs this. Luckily, you can gather all of this information ...
- 3,218
5
votes
Accepted
Is it necessary to specify resource in S3 Bucket Policies?
Yes, Resource is a required element on S3 bucket policies. If the resource is omitted, the API will respond with an error resembling this one below.
status code: 400
Error putting S3 policy: ...
- 351
4
votes
Accepted
Access S3 files from Datacenter over VPN to VPC
You need to use...
the Internet, from your data center, to access S3 directly (not via the endpoint) or
an AWS DirectConnect connection to access S3 directly (not via the endpoint) or
an HTTP proxy ...
- 805
4
votes
Impact on policy after deleting IAM user
The policy will still exist.
The documentation lists what is deleted when a user is deleted, any resources that the user has created will persist.
- 1,112
4
votes
Accepted
Impact on policy after deleting IAM user
S3 Life cycle policy will remain and working as it is.
When you use the AWS Management Console to delete an IAM user, IAM automatically deletes the following information for user:
The user Any ...
- 1,536
4
votes
Is it possible to mount an s3 bucket as a point in a docker container?
Yes , you can ( and in swarm mode you should )
rexray example
s3vol example
in fact with volume plugins you may attach many things
also for registry
- 141
4
votes
Accepted
auto delete aws S3 backups
I think you're on the right track. Just need a tiny bit more logic on the backup process, then S3 can handle the rest.
Consider this:
A backup is uploaded every day.
If today is January 1st (for ...
- 1,338
3
votes
AWS S3 ACLs across accounts
No. The account that uploads an object into a bucket owns the object. All the bucket owner account can do, unless granted permission by the object owner account, is delete the object.
- 805
3
votes
Accepted
Amazon AWS - Amazon Connect IAM policies
There's a generator for policies in IAM (Into IAM -> Policies -> Create), choosing connect gives me the following policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid":...
- 11.3k
3
votes
Is it possible to load an image into S3 via an URL?
Nope, credit goes to this SO answer, still valid today: How to upload files directly to Amazon S3 from a remote server?.
But if you have a server in the cloud, ideally in AWS, it'll probably go ...
- 6,730
3
votes
Accepted
How to "inject" AWS Lambda code into a CloudFormation?
You've got a couple of options:
1. Embed the source code to the CloudFormation template
I do just that in my ec2-start-stop demo. You'll see file ec2-start-stop.template.yml with these lines:
...
- 1,001
3
votes
AWS Storage Cost for a custom AMI
AWS stores whole AMI as a single non-differential snapshot thus they charge for a full size of files in an image.
- 547
3
votes
View all AWS S3 buckets and list each buckets storage used
Easiest way to see this information nowadays is using S3 Storage Lens.
S3 dashboard > S3 storage lens (left navigation pane) > default-account-dashboard > Buckets tab
Scroll down to the "...
- 133
3
votes
Accepted
Artifacts mentioned in secondary-artifacts section not getting uploaded to s3 bucket
In the CodeBuild "Artifacts" configuration, there is an "Add Artifact" button.
You need to click that and reference your secondary artifact to get it to upload to S3 as well.
- 146
3
votes
Amazon S3 bucket does not redirect folder URLs to index.html
CloudFront does not actually have this feature. You can redirect the root object https://www.example.com/ to https://www.example.com/index.html by setting DefaultRootObject, but there's nothing for ...
- 1,395
3
votes
Accepted
Is it possible to mount an s3 bucket as a point in a docker container?
No you can't.
S3 is an object storage, accessed over HTTP or REST for example. Just as you can't mount an HTTP address as a directory you can't mount a S3 bucket as a directory.
Having said that ...
- 1,001
3
votes
Accepted
For storing avatars in s3, which approach has the lowest cost?
For the best performance naming many small files in an s3 bucket may not be the best idea unless they user_id is a random string. See how the S3 hashing algorithm works https://aws.amazon.com/blogs/...
- 381
Only top scored, non community-wiki answers of a minimum length are eligible