Questions tagged [amazon-web-services]

For questions about Amazon Web Services (AWS) in general. If your question is about a specific Amazon web service please use that tag, e.g. [amazon-s3], [amazon-ec2], [amazon-ecs], [aws-codedeploy]

Filter by
Sorted by
Tagged with
35
votes
5answers
8k views

How to avoid interactive dialogs when running “apt-get upgrade -y” in Ubuntu 16.04 when packaging with Packer?

I'm using Packer to create an AWS AMI based on an Ubuntu 16.04 image. In the beginning, I'm doing an upgrade: sudo apt-get update sudo apt-get upgrade -y Here is the relevant part of my provisioners ...
33
votes
2answers
613 views

What is a good strategy to keep my site online when S3 goes offline?

What is a good strategy to keep my site online when S3 goes offline? If S3 US East 1 goes offline, how should I have my app configured/structured to prevent that taking my entire site offline? What ...
31
votes
4answers
10k views

Why is the AWS EC2's spot price greater than the on-demand price?

I was trying to provision spot instances via Ansible yesterday, and almost all my requests failed, even when I put my spot price == the on-demand price of that instance. So, when I had a look at the ...
31
votes
3answers
4k views

Who is this Chaos Monkey and why did he crash my server?

I had a perfect server, it was so pretty and rock solid and so I named it Petra. It was perfect in every way, everything was configured and tuned just right, it had perfect 100% service record and 753 ...
25
votes
5answers
8k views

How to clean up orphaned AWS EC2 snapshots?

We end up with a fair amount of AWS EC2 snapshots where the AMI has been deleted, but the snapshot is left behind to rot. I'd like a non-manual way of identifying and deleting these orphans to save us ...
24
votes
4answers
16k views

What is the purpose of AssumeRolePolicyDocument in IAM?

I'm confused as to what the purpose of AssumeRolePolicyDocument is. It seems like to create a role all you need is a parent name that you can attach policies to. It seems like AssumeRolePolicyDocument ...
21
votes
5answers
4k views

How can I experiment with Cloud (Azure, AWS, Google, etc) without going broke?

Cloud technologies are very hot right now, but they can be expensive. What are the best strategies for learning/trying cloud services without racking up a huge bill?
21
votes
1answer
485 views

How do I work around the fact that AWS SQS is not HIPAA compliant?

I have a use case where data from S3 is queued into AWS SQS, which is in turn connected to CloudWatch, whose metrics will be triggering AWS Lambda. However, I want the architecture to be HIPAA ...
19
votes
2answers
6k views

Is it possible to set CloudWatch alarm for a combination of metrics?

For example, I want to trigger a CloudWatch alarm for the following: Number of messages sent - Number of messages deleted >= 6 The above is for AWS SQS metrics. And I want that metric to trigger AWS ...
16
votes
2answers
311 views

How to order physical backup of S3 data?

I'd like to order backup of data to be copied out from S3 to a physical drive (SATA HD) and to be shipped to the address. I know it's possible using Amazon Glacier, however I cannot find such option ...
16
votes
1answer
5k views

How to create a centralised dashboard to monitor multiple cloud (Amazon and Google)?

I'm looking for a single product or service which can be used as a centralized dashboard to monitor cloud infrastructure provide by Amazon Web Services and Google Cloud Platform in addition to the ...
15
votes
3answers
2k views

How to check AWS spot pricing automatically?

I've been thinking about a Jenkins job to check the pricing for the agent boxes we spin up; every so often the price spikes and it'll be an hour or two before anyone notices that no new agents are ...
13
votes
1answer
1k views

How to communicate queue-based processing delays to non-technical team members?

I am responsible for a set of SQS queue-processing jobs with a scaling policy on the ApproximateNumberOfMessagesVisible CloudWatch metric. These jobs can fail to keep up with the amount of messages ...
13
votes
1answer
654 views

How do you manage your inventory of spot instances on AWS with Ansible

I am new to Ansible, with some experience in Chef. I am considering learn and use Ansible to manage environments. I was wondering, what would be the best practice to manage inventory of AWS spot ...
13
votes
4answers
6k views

CodeBuild - multiple input artifacts in AWS Pipeline

I am building an AWS CodePipeline which depends on two source repositories. If the 'CodeBuild' step is specified with multiple Input artifacts, then you get the error: .. declares 2 input artifacts ...
12
votes
3answers
9k views

How to backup Amazon Cognito?

Amazon Cognito is a user management and authentication service provided by Amazon through Amazon Web Services: Amazon Cognito lets you easily add user sign-up and sign-in to your mobile and web ...
11
votes
1answer
884 views

How to set up AWS Trusted Advisor alert for Service Limit warnings?

All I want in life is for AWS to shoot me an email when any of Trusted Advisor's Service Limits go to WARN or ERROR status. I've checked their guides, tried chatting with support and no one can get ...
11
votes
1answer
2k views

How to automatically remove a dead node from RabbitMQ cluster

I am planning to create RabbitMQ cluster using Ansible on AWS VPC with Amazon internal load balancer as the frontend to point connections to it. Any suggestion how to remove a dead node from RabbitMQ ...
11
votes
1answer
706 views

Is Elastic Beanstalk suitable for enterprise-grade CD?

I'm working with a project that's using Jenkins to build and deploy microservices to Elastic Beanstalk. We deploy an integration branch to a test environment, release branches to a staging environment,...
10
votes
2answers
229 views

How to track changes to AWS settings?

Is there a way to track changes we make to the AWS system? For example, changes to a subnet setting, from using nat to iwg - these display a message, and then disappear. Is there a way to get AWS to ...
10
votes
3answers
7k views

How do I upload my zipped lambda code into AWS Lambda?

So I've read through the docs at: https://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html I've created the zip file, how / where do I actually upload / ...
9
votes
1answer
2k views

Terraform: How to support releasing new versions of the code, running in parallel with the last stable release?

I have one service in production (on AWS), which follows the immutable server pattern. Its deployment looks like this: Create a new AMI with Packer Create a new CloudFormation stack, starting with an ...
9
votes
4answers
6k views

How to track unused resources in AWS?

I have been using AWS for a while now. I always have the difficulty tracking AWS resources and how they are interconnected. Obviously, I am using Terraform but still, there is always ad-hoc operations ...
9
votes
6answers
942 views

How to prevent hug of death on EC2 Instance?

I've a iOS app on the app store and recently I've received a huge surge of traffic to my landing page hosted on EC2 and resulted in the page not responding, luckily I managed to recover it by ...
9
votes
1answer
594 views

How can I manage changes in desired capacity due to scaling policies in AutoScaling Group state?

How can I manage changes in desired capacity due to scaling policies in AutoScaling Group state in terraform? Specifically, suppose I provision an aws_autoscaling_group resource with terraform with a ...
8
votes
5answers
6k views

How to provide a user to access only a particular bucket in AWS S3?

I have a list of buckets in AWS S3. I have created an IAM user. I have an option to provide S3 full or read only access for a user using groups. Is there any options to provide access only to a ...
8
votes
1answer
3k views

How do I detach an existing volume and attach a new volume with Terraform?

Is there a way to detach EBS volume from existing EC2 instance and attach a new EBS volume using Terraform v0.9.2 ? I prefer to replace the volume rather than trying to destroy the instance and ...
8
votes
2answers
11k views

Jenkins installed on aws unable to launch on browser

I have a fresh aws instance with Ubuntu 14 installed. I am trying to install and run jenkins on it. I did the following steps. Installed java8 wget -q -O - https://pkg.jenkins.io/debian/jenkins-...
8
votes
1answer
129 views

Debug BackendConnectionErrors from Kubernetes Service LoadBalancer sesrvice

We recently moved some of our production infrastructure to Kubernetes. Many pods are exposed through a LoadBalancer service on AWS. This creates an ELB, registers each node in the cluster with the ELB,...
8
votes
1answer
8k views

How to get the ARN for an arbitrary Amazon Resource?

I am creating an AWS IAM policy for a Serverless Framework project, I am applying the Principle of Least Privilege to ensure the policy is as tight as possible. To restrict the scope of the policy I ...
8
votes
2answers
948 views

How to run database migrations in an auto-scaled group

We are just about to move from a monolithic architecture to an auto-scaled group and I'm not sure how I should be running my database migrations (Laravel). I am thinking that a script will run when a ...
7
votes
5answers
204 views

What are best practices and/or tools for updating AMIs on a regular basis?

I have a Master-Workers style computation model. And, I launch the workers via an AMI through an Ansible playbook. The workers' AMI has all the necessary configs needed for doing the computation (...
7
votes
3answers
2k views

How to ensure equitable distribution of SQS messages in a distributed systems setting?

I have multiple servers, each having a script polling an SQS queue [all polling the same queue]. So, is there any way I can ensure an equitable distribution of messages to all those clients [i.e. my ...
7
votes
3answers
197 views

Who should write Docker files?

Once again company debate rages on who should write Docker files (including compose, interchangeable for this post). The scenario is common in my work (developer and consultant) and I wanted to know ...
7
votes
2answers
8k views

How to only allow API Gateway requests to reach our EC2 instances

We have multiple EC2 instances hosting our micro-services. The autoscaling group of servers has an ELB. All traffic is routed via the AWS API Gateway. The problem is that the ELB has its HTTPS port ...
7
votes
1answer
101 views

How to use tags in Amazon Web Services and Azure

I am currently building a solution on Azure (though the question should relate to AWS equally) and I am using a UniqueId tag to locate and use objects. For instance, we have a storage account that ...
7
votes
2answers
3k views

Application calling AWS internal load balancer in same subnet is timing out

Some background: I've created a moderately complex network using Amazon's vpc. It's a three-tiered network across two availability zones. Each layer has a subnet in zone-a and zone-b. The ...
7
votes
1answer
2k views

How can I autoscale instances based on Kafka topic lag?

I have autoscaling groups that I want to dynamically scale up/down based on the amount of lag showing in certain Kafka topics. There's a 1:1 relationship between topic and autoscaling group that needs ...
7
votes
1answer
969 views

AWS ECS/EKS: EC2 vs Fargate mode

Amazon offers the option of launching their container management services ECS and EKS in two modes: EC2 and Fargate. In the former, normal VMs (EC2 instances) are used and ECS or EKS (depending on ...
7
votes
1answer
186 views

How to automate submission to Amazon Marketplace?

Is anyone aware of a way to automate the submission of an AMI to Amazon's Marketplace? I want to take a public AMI and share it to the AWS Marketplace so it can be scanned for listing and then submit ...
7
votes
2answers
587 views

Best way to launch 100 identical Docker machines on AWS?

I need to do some large scale testing on a system. The system consists of 2 main parts: Manager and Nodes. There will be only one manager server, but 100 nodes. Both the manager and nodes are Docker ...
7
votes
3answers
322 views

When your micro service in production needs a secret, where is the secret stored and retrieved from?

The context of the question is Secrets management. How do you manage storing and retrieving secrets for your micro service which is in production?
7
votes
2answers
902 views

How to add shards to a Redis cluster managed by Terraform without loosing all state?

I have set up a Redis cluster with Terraform. The setup looks like this: resource "aws_elasticache_replication_group" "instance" { replication_group_id = "test" node_type = "...
6
votes
1answer
143 views

Should AWS arn values be treated as secrets?

I have a configuration file that contains AWS resource arn's stored in a Github repo. This includes some arn's of AWS IAM role resources. Should these be treated as if they are a secret value, and not ...
6
votes
3answers
271 views

Continuously getting locked out from EC2 instance. No supported authentication methods available

I am getting continously locked out from my ec2 instance time and again. I have to take its root volume and attach it to some other instance (mostly new instance) to prevent data loss. I am not able ...
6
votes
1answer
3k views

How can I use a local file for a nested Cloudformation stack?

How and is it possible to use a local file for a CloudFormation stack besides an S3 bucket?
6
votes
2answers
681 views

Terraform AWS AZ error - manually specify AZ?

I'm fairly new to terraform and when creating AWS infrastructure I use the following to fetch the 'available' availability zones for my region (us-east-1) where I'm trying to create an EKS cluster. ...
5
votes
2answers
158 views

AWS Spot instances price

While trying to create spot instances I notices that the Price for almost all instance types (from m3 , c3, c4) are always higher than the On-Demand price of corresponding variants in us-east-1c and ...
5
votes
2answers
211 views

What should I use for configuration management on AWS?

I am trying to find a solution for configuration management using AWS OpsWorks. What I can see is AWS offers three services for OpsWorks Chef Automate Puppet AWS stacks I have read basics of all ...
5
votes
1answer
2k views

How to send SMS through AWS?

Can we send SMS through AWS for the below case? I have an application installed on Ec2 and that application runs few rules and may send SMS to end users if the rule conditions are met. I wanted to ...

1
2 3 4 5
9