Questions tagged [amazon-sqs]
Amazon Simple Queue Service (SQS) is part of Amazon Web Services, providing a fully-managed message queuing service for reliably communicating among distributed software components and microservices. Use this tag for questions that describe Amazon's Simple Queuing services.
14
questions
0
votes
1
answer
254
views
Filtering AWS SQS Tags using JQ
I'm having doubts when trying to run a query with JQ.
I have an SQS queue with Tags applied and when I run
aws sqs list-queue-tags --region sa-east-1 --queue-url <my-url-queue> --output json | ...
- 466
2
votes
1
answer
910
views
AWS SNS-SQS: Checking a subscription already exists
I've found it's a useful strategy to create relevant queues whenever a queue worker service boots. For instance, on service boot, it checks whether its queues exist already in AWS and creates them if ...
- 31
1
vote
1
answer
4k
views
How to programmatically enable and disable SQS triggering AWS Lambda?
I have a requirement where I want to stop my lambda from receiving messages from SQS. And I want to implement this like an API which can toggle between the disabling and enabling of the AWS Lambda ...
- 143
2
votes
2
answers
92
views
Terminate single processing server based on SQS queue condition, but keep/ reinstate autoscaling size at 1
I have a processing server that processes requests in an SQS queue, only about 10 per day, each taking around 2 minutes to process.
Sometimes the process gets "stuck" and it would be ideal to ...
- 650
0
votes
1
answer
2k
views
SQS: What could high number of visible messages mean if messages received & deleted are normal?
I'm new to SQS.
I recently noticed the ApproximateNumberOfMessagesVisible in my SQS queue started rising. What was interesting though is that there was no sudden surge in the number of messages ...
5
votes
1
answer
656
views
AWS SQS - use case for "short polling"
When using event driven processing of work with AWS SQS and worker clients. the use of long polling is often recommended. AWS documents this here
Essentially when there are no message in the queue, ...
- 650
1
vote
0
answers
19
views
SQS Displaying only "data" of logs [closed]
I am new to SQS and as a POC, I am only pushing "var/log/auth.log" to SQS. I now want to listen to the SQS que. I am able to do it, however, its displaying the entire json file. I want to display only ...
- 11
0
votes
1
answer
51
views
Processing message queues quickly without wasting money on AWS Lambda
What are some strategies and patterns for processing message queues quickly?
I have a Lambda function that is invoked on a schedule every x minutes to process a number of messages in the queue. I ...
- 103
4
votes
2
answers
906
views
Is it better to perform delayed jobs using queues or at app level?
I'm trying to scale up my push notification infrastructure. We currently use laravel app running on 2 ec2 instances horizontally scaled behind a load balancer, and each instance has a beanstalk queue ...
- 473
7
votes
3
answers
2k
views
How to ensure equitable distribution of SQS messages in a distributed systems setting?
I have multiple servers, each having a script polling an SQS queue [all polling the same queue].
So, is there any way I can ensure an equitable distribution of messages to all those clients [i.e. my ...
- 2,816
13
votes
1
answer
2k
views
How to communicate queue-based processing delays to non-technical team members?
I am responsible for a set of SQS queue-processing jobs with a scaling policy on the ApproximateNumberOfMessagesVisible CloudWatch metric. These jobs can fail to keep up with the amount of messages ...
- 1,484
13
votes
2
answers
5k
views
Is it possible to trigger an AWS Lambda function directly from CloudWatch alarm?
Currently, for triggering a lambda function in response to a CloudWatch alarm, I need to route the alarm via SNS, like this:
AWS CloudWatch Alarm --> Send to a topic in SNS --> SNS topic ...
- 2,816
23
votes
2
answers
9k
views
Is it possible to set CloudWatch alarm for a combination of metrics?
For example, I want to trigger a CloudWatch alarm for the following:
Number of messages sent - Number of messages deleted >= 6
The above is for AWS SQS metrics. And I want that metric to trigger AWS ...
- 2,816
23
votes
1
answer
570
views
How do I work around the fact that AWS SQS is not HIPAA compliant?
I have a use case where data from S3 is queued into AWS SQS, which is in turn connected to CloudWatch, whose metrics will be triggering AWS Lambda.
However, I want the architecture to be HIPAA ...
- 2,816