Questions tagged [amazon-sqs]

Amazon Simple Queue Service (SQS) is part of Amazon Web Services, providing a fully-managed message queuing service for reliably communicating among distributed software components and microservices. Use this tag for questions that describe Amazon's Simple Queuing services.

1
vote
0answers
9 views

SQS Displaying only “data” of logs

I am new to SQS and as a POC, I am only pushing "var/log/auth.log" to SQS. I now want to listen to the SQS que. I am able to do it, however, its displaying the entire json file. I want to display only ...
0
votes
0answers
17 views

Developing a progress meter for uploading and processing a file

I saw some posts on the internet about AWS Lambda user progress meter but there were no specific answers. One answer was that "you can use SNS" but I think I solved it using SQS: Lambda 1 receives ...
0
votes
1answer
42 views

Processing message queues quickly without wasting money on AWS Lambda

What are some strategies and patterns for processing message queues quickly? I have a Lambda function that is invoked on a schedule every x minutes to process a number of messages in the queue. I ...
4
votes
2answers
477 views

Is it better to perform delayed jobs using queues or at app level?

I'm trying to scale up my push notification infrastructure. We currently use laravel app running on 2 ec2 instances horizontally scaled behind a load balancer, and each instance has a beanstalk queue ...
7
votes
3answers
911 views

How to ensure equitable distribution of SQS messages in a distributed systems setting?

I have multiple servers, each having a script polling an SQS queue [all polling the same queue]. So, is there any way I can ensure an equitable distribution of messages to all those clients [i.e. my ...
11
votes
1answer
365 views

How can I communicate queue-based processing delays to non-technical team members?

I am responsible for a set of SQS queue-processing jobs with a scaling policy on the ApproximateNumberOfMessagesVisible CloudWatch metric. These jobs can fail to keep up with the amount of messages ...
11
votes
1answer
2k views

Is it possible to trigger an AWS Lambda function directly from CloudWatch alarm?

Currently, for triggering a lambda function in response to a CloudWatch alarm, I need to route the alarm via SNS, like this: AWS CloudWatch Alarm --> Send to a topic in SNS --> SNS topic ...
18
votes
2answers
3k views

Is it possible to set CloudWatch alarm for a combination of metrics?

For example, I want to trigger a CloudWatch alarm for the following: Number of messages sent - Number of messages deleted >= 6 The above is for AWS SQS metrics. And I want that metric to trigger AWS ...
21
votes
1answer
383 views

How do I work around the fact that AWS SQS is not HIPAA compliant?

I have a use case where data from S3 is queued into AWS SQS, which is in turn connected to CloudWatch, whose metrics will be triggering AWS Lambda. However, I want the architecture to be HIPAA ...