19

According to Amazon AWS Customers may use any AWS service in an account designated as a HIPAA account, but they should only process, store and transmit PHI in the HIPAA-eligible services defined in the BAA. There are ten HIPAA-eligible services today, including AWS Snowball, Amazon DynamoDB, Amazon EBS, Amazon EC2, Amazon Elastic MapReduce (EMR), ...


16

As with any outage communications, a non-technical reader will be primarily looking to understand: How long was it? How bad was it? Amazon CloudWatch metrics provides the following metrics for SQS queues that can help answer these questions: NumberOfMessagesSent: The number of messages added to a queue. NumberOfMessagesReceived: The number of messages ...


9

As far as I know, it is not possible. The the offcial AWS Lambda documentation says it support CloudWatch Events, but no mention of CloudWatch Alarms (and CW Events != CW Alarms) What you are currently doing, sending the alarms on a SNS and using listening on a topic seems the way to go. In fact, CloudWatch Alarms only outputs to SNS so far.


8

For the last overall goal, I would set the alarm on ApproximateNumberOfMessagesVisible from Cloudwatch documentation on SQS this is: The number of messages available for retrieval from the queue. Units: Count Valid Statistics: Average, Minimum, Maximum, Sum, Data Samples (displays as Sample Count in the Amazon SQS console) For the more ...


7

Recently AWS just announced CloudWatch Metric Math which should be suitable for your use case. References:- Amazon CloudWatch Adds Metric Math to Enable Custom Operations on Metrics Create a CloudWatch Alarm Based on a Metric Math Expression


6

If there are 100 messages in the queue and 5 consumers, the initial distribution will be no more than 10-10-10-10-10. A single response can never return more than 10 messages. This seems like a non-issue. Race conditions related to multiple consumers should be a non-issue as well. SQS is designed for multiple simultaneous consumers. Use long polls ...


4

Disclaimer: this is certainly not the best solution for backend workers, I'm just trying to be creative within the given constraints. First, you'll need to be familiar with SQS's visibility timeout (documentation) for this to work. Let's assume the following, then I'll walk through how it might work: We only have 1 request (henceforth "message") coming in ...


3

Long polling is always beneficial since it results in higher performance at reduced cost for the majority of use cases. Unfortunately, you cannot control the number of messages each worker receives from the queue due to the distributed nature of the queue. But there are some client-side workarounds that can help you in balancing the load for workers. So, ...


3

Good architecture in how you use SQS queues will solve your problems. If we assume there are, say, 3 minutes of processing per message, then you can almost guarantee equal distribution of the messages as this is very large compared to the time needed to poll the queue, if you delete the message from the queue only after it has been processed. Do be aware ...


3

An SQS queue can now be wired directly to a Lambda function as an event source, so you don't have to start the Lambda function on a schedule. https://aws.amazon.com/blogs/aws/aws-lambda-adds-amazon-simple-queue-service-to-supported-event-sources/


2

I would recommend using a scheduler to schedule job execution, rather than building logic to delay execution. Schedulers give you the added advantage of status updates. Consider using Simple Workflow Service from AWS. Instead of using beanstalk queue engine, could you use Lambda to process your marketing jobs? Simple Workflow Service works with Lambda. You ...


2

This sounds on the fence with opinion based but I think there's an important thing that narrow enough the problem to be answered. It is still an opinion but, as far as you're horizontally scaling, instances may appear and disappear at will, keeping the delay in memory is thus not an option, so at this point you'll have to workaround this and bake something ...


2

there was no sudden surge in the number of messages received or depression in number of messaged being deleted from the queue that corresponds with this surge Yes, it looks like there was. But first, be sure you have the correct definitions in mind for these metrics, because your intuitive interpretation might be reversed. "Sent" means sent to SQS and ...


2

While you can benchmark this yourself, benchmark testing has shown that there isn't much of a difference in time between long and short polling when the queue is full. Additionally, with long polling you always have data coming back unlike short polling. As for when to use short polling the AWS FAQ covers this: For example, if your application uses a ...


2

Call Lambda's UpdateEventSourceMapping API with the {"Enabled":true/false} request body. If, for some reason, you want your own API for this, you can create one in API Gateway as described below. Before you begin, you need the UUID of the event source mapping you wish to enable/disable. Use the AWS CLI to get this (it isn’t shown on the UI): aws ...


1

In the end I decided to go another way. I don't think it was actually a good idea to modify things in that way when the worker boots. It can cause problems because if you have to do some emergency modification of the configuration those changes will be reverted the next time your worker instances restart. Instead, on boot the worker checks if the queue ...


1

Use AWS Automation document on state change to modify autoscaling group to 0 and return to 1 after. It should terminate and recreate.


Only top voted, non community-wiki answers of a minimum length are eligible