This is actually a great example of people slightly abusing spot. People are saying 'Our workload is really important but we don't want to pay full on demand price', so they set a bid price higher than on-demand on the assumption that it is very unlikely to be terminated, but still want to get the 'cheapest possible' spot price on offer.
There have been ...
Spotted an open source tool called autospotting that just might help with this:
Once enabled on an existing on-demand AutoScaling group, it launches an EC2 spot instance that is cheaper, at least as large and configured identically to your current on-demand instances. As soon as the new instance is ready, it is added to the group and an on-demand ...
Largely inspired by the blog posts and gist already linked in the other answers, here is my take to the problem.
I did use some convoluted JMESpath functions to get a list of snapshots and not require tr.
Disclaimer: Use at your own risks, I did my best to avoid any problem and keep sane defaults, but I won't take any blame if it cause problem to you.
Amazon AWS just released (on 13th February 2017) a new feature that allows to change a size of an EBS volume.
This allows to increase the size of an EBS volume on an existing instance, while it is running.
It is important to note that changing the volume size,...
To run ipconfig from the AWS Systems Manager Run Command:
$ aws ssm send-command --document-name "AWS-RunPowerShellScript" --instance-ids "<your instance id>" --parameters commands=ipconfig
Note: If you've got the error, consider specifying the right --region.
This assumes you have your AWS credentials and CLI configured properly. See Systems ...
There are 2 reasons for this:
A lot of users are using the spot instance at some times (Think about batch processing, boot up 100 machines as spot instance and crunch away).
For a spot instance you don't pay the bidding price, you pay the current spot price. The bidding price is the cutoff point. If the current spot price exceeds that of the bidding price ...
If you want something quick to get this sorted without much more knowledge I'd recommend elastic beanstalk. It's another AWS application which will handle the load balancer configuration and instance scaling for you.
There's no extra cost on top of the load balancer and instances so you can keep using t2 type instances but let elastic beanstalk scale as ...
So what you have actually is this:
As your API ELB is in a private zone it can't be accessed from the internet.
Your frontend in React.js just run in User's browser and not on the UI servers, those server just serve static files.
You have two options, configure your frontend servers to redirect API calls to the API ELB or just update the API ELB to be ...
For a given instance, you would first use aws ec2 describe-instances to get the information JSON for your instance.
The information also contains the keypair name used to create that instance.
E.g. for an instance i-0e2x8xd7xxx (Note: I use the awesome tool jq to do JSON parsing but you can use any other solution)
aws ec2 describe-instances --instance-...
When creating an AMI, you are making a copy of your EBS volumes. This utilizes block storage and will only need to process the diffs of your EBS snapshots. If you changed a lot of data in your EBS or don't have a previous snapshot, it will take longer to create your AMI.
Here is a quote from the documentation at https://docs.aws.amazon.com/AWSEC2/latest/...
Boto3 has a function that allows you to create volume permissions, which is what AMI Sharing with AWS Marketplace requires you to do. snapshot.modify_attribute will allow you to share your AMI with the marketplace account like so (you can also use a JSON representation if you prefer, it's in the docs):
response = snapshot....
A useful bit of information for understanding why someone would bid over the on-demand price can be found in the Introduction to Spot Instances:
Spot Instances can be used to help you meet occasional needs
for large amounts of compute capacity (note that the default limit for Spot Instances is 100 versus the
default limit of 20 for On-Demand Instances....
I would personally consider a model like this:
Timed Lambdas -> Checks spot price -> Push to ElastiCache
Then when you need instances:
Timed lambdas -> Pulls spot price from ElastiCache, sets it as environment variable on your Machine where you spin up IaC from -> This is parsed as argument to IaC code and pushes out the spot price
You could ...
You notice that you now have access to 6 availability zones in us-east-1, where you previously only had 4 or 5.
Adding capacity is the reason that new availability zone is now available.
Because of the way the spot market works, there is no way to know what the highest winning bid was, at any given time... because the current market price is based on the ...
The article you are referencing mentions encrypted RAM but that was only a reference to possibilities of enhancing the technology more in the future, not about the current state of confidential computing.
Confidential computing means, that when data is in use these things are prevented:
Malicious insiders with administrative privilege or direct access to ...
you create a Docker Swarm stack file:
image: ubuntu # or your custom Docker image
Then, with docker stack you can run your 10000 Ubuntu's on one or - probably better in this case - a set of Swarm hosts. This could be bare metal or AWS. Enjoy!
For MAC address is very ...
A good way to find comparison information about things like this is googling for "X vs Y", e.g. "Chef vs AWS stacks", "Chef vs Puppet" or something like that. That does turn out subjective information, and while it is nothing like having hands-on experience, you still get a few nice nuggets here or there.
For example, Chef gives you the full Ruby language ...
Update 12/05/2020: Fargate prices have been greatly reduced over the last months and are now comparable with EC2 prices for the same CPU / RAM configurations. That makes Fargate a valid option for 24x7 workloads.
Originally written in 2018 - no longer correct!
Fargate is more expensive than EC2 for the same vCPU/RAM amount.
m5.large (2 vCPU,...
I used the following script on GitHub by Rodrigue Koffi (bonclay7) and it works pretty good.
From the documentation blog post it does some more things:
It actually does a bit more than that, at of today it allows:
Removing a list of images and associated snapshots
How about user-data ?
I believe adding the "#cloud-boothook" allow to force the user-data to run at every restart.
echo 'test' > /home/ec2-user/user-script-output.txt
If so, you could fix your sudo scripts hopefully... or install / add AWS run commands configuration https://aws.amazon.com/blogs/aws/new-ec2-run-command-remote-...
I like to use ELK, Elasticsearch, Logstash and Kibana with Beats.
Filebeat, that belongs to Beats series will forward the logs that read from a file to logstash.
In logstash you could tag, filter, parse and modify the log entries that are stored in Elasticsearch.
For visualizing you can use Kibana. You can create a great dashboards based on the logs with ...
You might not be able to get this done on the free tier. Puppet for example isn't going to want to start because of the RAM limitations. The AWS free tier uses thje t2.micro instance which only has 1GB of RAM. Your operating system alone probably needs 512 to run at idle. This leaves you a mere 512 MB or ram for all of the things that you listed.
While the ...
From Granting a User Permissions to Pass a Role to an AWS Service:
To pass a role (and its permissions) to an AWS service, a user must
have permissions to pass the role to the service. This helps
administrators ensure that only approved users can configure a service
with a role that grants permissions. To allow a user to pass a role to
an AWS ...
We faced exactly the same choices for our live service that deploys a laravel API and multiple react apps.
There is no generic correct answer to a question "what tool/provider is better?" as it depends on your circumstances. Cost is a particularly tricky question. Are we just talking about the cost of getting something live, or the cost of running an ...
Let me give you a tool-agnostic way of trying to do this.
every so often the price spikes and it'll be an hour or two before anyone notices that no new agents are coming up, and then we have to go in manually and check the spot price and adjust accordingly, or switch zones, etc
We faced the same problem in the infrastructure which we're building. So, we ...
Generally recovering from this sort of mistake involves editing the boot loader to force a root shell. Since you don't have physical access to the (non-physical) machine, things are a bit different.
If this is an EBS boot drive, you can try detaching it, fixing it on another machine, then reattaching.
This is a good opportunity to learn why it's useful to ...
Essentially yes. From what I've observed when there is limited capacity in that availability zone the spot price jumps to 10x the on demand price.
Be sure to use as many AZs as possible (6 currently in us-east-1) when using spot instances to provide better guarantees you can launch an instance.
If editing critical flat-files across multiple servers in one place is too scary..
One approach I have often recommended is to let system users reside in /etc/passwd files, and to add an additional Name Service Switch (NSS) source for humans that need to log in. For big projects (lots of users authenticating against the PAM user data), that means LDAP, and ...
EBS Snapshots aren't really disk image files. They're logical containers of pointers to chunks of compressed and deduplicated backup data that EBS stores in S3 when you create snapshots -- but these are stored in buckets that EBS owns and controls.
They are user-accessible via only one mechanism -- creating a new EBS volume from the snapshot. When you do ...