Questions tagged [amazon-sqs]
Amazon Simple Queue Service (Amazon SQS) offers a reliable, highly scalable, hosted queue for storing messages as they travel between computers.
3,643
questions
0
votes
0
answers
9
views
How sns can conditionally push message to different subscribers
I'm exploring ways to conditionally send message from sns to it's subscribers based on condition. The default sns publishes the message to all it's subscribers. I want to send it only based on certain ...
0
votes
1
answer
19
views
Is it possible to use cloudformation to update a queue created through the aws sdk
I have created an SQS queue using the aws sdk in java. I want to migrate the queue for dev into my yaml cloudformation file. Test uses cloudformation already.
After adding dev to the list of regions ...
1
vote
1
answer
22
views
Rate-Limiting / Throttling SQS Consumer in conjunction with Step-Functions
Given following architecture:
The issue with that is that we reach throttling due to the maximum number of concurrent lambda executions (1K per account).
How can this be address or circumvented?
We ...
-1
votes
0
answers
10
views
Persist data to DB from the Queue
I am new to AWS and I have a below requirement.
One of our service is going to push a message in SQS and that data needs to be saved in a database(AWS RDS). To persist the data I could simply create a ...
0
votes
0
answers
21
views
Intermittent failure to connect to external ftp server in lambda triggered by sqs
Working on a lambda that's triggered by SQS and just needs to connect to FTP and validate a file (no download). Every 5 minutes at least 20 messages are put into the queue that triggers the lambda. I ...
0
votes
1
answer
21
views
Cross account SNS SQS subscription set up (CDK)
I'm trying to set up a cross account subscription between a SQS queue (in account 11111111111) and a SNS topic (in account 22222222222). However, I've been having trouble having this done thru CDK. ...
-1
votes
1
answer
25
views
What does 'Messages available' in the AWS SQS console mean?
What is 'Messages available' in the AWS SQS console?
I'm familiar with message queueing concepts, and 'messages in flight' seems obvious - that's the amount of unprocesed messages on the queue. But ...
0
votes
0
answers
26
views
"Error sending event to AWS EventBridge: SerializationError: failed to unmarshal response" with MongoDB trigger connected to AWS EventBridge
I have created a MongoDB trigger on a collection with Create, Update, Replace and Delete enabled. Document Preimage is also enabled.
I have connected the trigger to AWS EventBridge. From the ...
0
votes
1
answer
17
views
Cannot delete SQS Message using Dotnet Lambda
I have spent nearly 6 hours and searched all over google and I can't seem to find what I am doing wrong. I have created a Lambda function using the dotnet framework and it's triggered by an SQS ...
0
votes
1
answer
31
views
Can I redrive messages from an SQS queue that is an onFailure target from a Lambda function back to that that function?
I have an SQS queue (Q) that receives messages via the onFailure "destinations" setting from a Lambda function (F). The Lambda function is triggered by and EventBridge event bus "rule&...
0
votes
1
answer
33
views
What are the benefits of using SNS to SQS to Lambda compared to having SNS to SQS to SQSConsumers?
We have a SQS queue subscribe to SNS Topic which publishes about 1-5 million events per month. I want to know which of these combinations - SNS->SQS->Lambda vs SNS->SQS->SQSConsumer would ...
0
votes
1
answer
23
views
AWS SQS and visibility timeout during an exception
Im curious to know, is the visibility timeout respected if an exception is thrown by the app after taking a message off the sqs qeue..
Im of the impression it is respected, so if an exception is ...
1
vote
0
answers
24
views
Good way to notify distributor that the worker can't process any more messages
I currently have a distributor service and a worker service that communicate via SQS. The distributor searches for expired values in a postgresql database and sends them to SQS.
The worker picks up ...
0
votes
0
answers
25
views
Cross Account SNS subscription by SQS and poll SQS from Lambda
I have a use case to -
create an SNS Topic in a AWS Account, let's say Account "A" and
SQS and Lambda on Account "B".
I have to implement it using CDK (Preferably MonoCDK).
The ...
0
votes
1
answer
30
views
System design : AWS chat application with API Gateway / SQS - how does server send message to recipient?
This is a system design question about creating a messaging application (like WhatsApp or FB messenger).
I saw a video that had the users connected via websocket to the API Gateway , then API GW ...
-3
votes
0
answers
25
views
Using AWS SQS Queue to store data in blocks into the database [closed]
I have a method that pushes the properties of a class into the queue and then from this queue i am deserializing the string in the push Queue method and then calling a method that stores the data from ...
0
votes
0
answers
21
views
Type parameter 'com.amazonaws.services.sqs.AmazonSQS' is not within its bound Error
I am working on update our java service, it is a common helper service which inherit several AWS features include DynamoDB, SQS...
We found out that AWSSQSClient is deprecated based on the aws doc ...
0
votes
0
answers
19
views
Configure SQS Dead letter queue for PHP Lumen application
In PHP Lumen, I use SQS queue to process jobs, the queue is defined in the config/queue.php file as mentioned in: https://lumen.laravel.com/docs/5.3/queues and https://laravel.com/docs/5.5/queues
the ...
0
votes
1
answer
21
views
Inject mocked dependency into mocked constructor
How can I mock the return value of _SqsService.getSqsQueueURL? This compiles, but the function returns undefined. Do I need to inject sqsMocked into sqsServiceMocked? If so, how?
Test:
jest.mock('aws-...
0
votes
0
answers
14
views
How to set milliseconds of delivery delay on an AWS SQS FIFO queue?
I would like to set 85 milliseconds of delivery delay for each message on an AWS SQS FIFO queue.
After I set delivery delay field as 0.085, when I try to save queue settings, AWS shows a blank page ...
0
votes
0
answers
16
views
Migrate from springframework cloud aws messaging to awspring cloud messaging version 2.4.1
After changing my imports to use the awspring library, I had @EnableSqs marked as deprecated, how to enable sqs if @EnableSqs is deprecated?
0
votes
1
answer
26
views
AWS Terraform - push message to SQS every X minutes using CloudWatch shedule
What resources do I need to set up to create a schedule which will push message to a SQS queue with body specified within terraform?
I found this https://registry.terraform.io/providers/hashicorp/aws/...
-2
votes
0
answers
21
views
Is it a common practice to have one dead letter queue for multiple SQS queues? [closed]
I have simple data pipeline with three queues (e.g. pre-proc, proc, post-proc). Structure of a message in each queue similar to id, type and additional_data. It seems overkill to have a dead letter ...
1
vote
0
answers
15
views
Scale replicas in AWS EKS based on multiple conditions of a custom Cloudwatch metric
I am fairly new to EKS. I want to autoscale on the basis of no of messages in my SQS queue. I have read about HPA in kubernetes (https://kubernetes.io/docs/tasks/run-application/horizontal-pod-...
0
votes
1
answer
13
views
How to insert bulk message manually in AWS SQS queue from AWS UI?
I want to do some testing for my application where it is trying to fetch messages from SQS and trying to process in batches. So for that, i want to insert bulk message manually through sqs from aws UI....
0
votes
0
answers
14
views
AWS SNS Queue Policy
I was trying to create a SNS queue programmatically via spring-cloud-starter-aws-messaging maven dependency. I could create the queue, send and receive messages
The next I would like to have is to set ...
2
votes
1
answer
127
views
How to create Context using traceId in Open Telemetry
I try to get all spans created in the following chain associated to the same trace context/traceId by context propagation:
service1 -> aws sqs queue -> service2
Auto. context propagation is not ...
0
votes
1
answer
26
views
How to develop Apps for AWS SQS using Kotlin
I'm looking for newest AWS SQS libraries written in Kotlin for Android. What do you use in your projects to find the right solution?
... or you could solve the problem using libraries written in Java?:...
1
vote
1
answer
35
views
AWS Boto3 SQS endpoint URL issue
We have a Lambda function that sends messages to SQS queue.
We are using boto3.
We have built a new environment and Lambda is running in a VPC on a private subnet.
The VPC end point is com.amazonaws....
1
vote
0
answers
19
views
How to have dedicated instances for celery containers when deploying django project using docker-compose on elastic beanstalk?
I have a web-app that uses celery for heavy calculations. It was deployed on AWS with elastic beanstalk, using docker-compose. The broker is Amazon SQS, while redis is used as result backend.
Here is ...
0
votes
0
answers
21
views
How to check if --authenticate-on-unsubscribe flag is set true during subscription creation by SNS Owner?
A SQS queue in account A is subscribed to a SNS Topic in account B.
I have access only to account A.
I want to check if the SNS owner has set --authenticate-on-unsubscribe flag as true while creating ...
0
votes
1
answer
32
views
Managing large AWS SQS payloads over S3
I am working in a project Java 11/Spring boot project that I need to send and consume a SQS message that is over 256KB, that is the common limit for SQS. I can't change the modeling of the system in a ...
0
votes
0
answers
38
views
SNS message is not sent to dead letter queue (DLQ) after retry delivery is exhausted
I have a subscription to a SNS topic that is configured to move a message to a DLQ if it can't be delivered successfully to a lambda function.
As described by this document, there are client-side and ...
0
votes
0
answers
12
views
why does the "celery" queue appear in sqs when deploying to beanstalk?
Every time I deploy to a beanstalk, I run celery workers, each of which is configured for a specific queue, and logs are also configured for each worker separately.
The question is where does this ...
0
votes
2
answers
56
views
Why does celery worker keep trying to connect to amqp even though the broker is sqs?
I tried to configure broker via settings and directly from the celery file .
Settings that apply to celery below.
AWS_SQS_SECRET = os.environ.get("AWS_SQS_SECRET")
broker_url = 'sqs://%s:%s@...
-1
votes
1
answer
39
views
How to consume messages from a queue
I have a scenario wherein I am supposed to consume messages from an AWS SQS.
I know @SqsListener can be used or something like that?
Thanks
0
votes
0
answers
16
views
Sleuth SQS integration not working with MessageConverter
We are using Spring Boot application using Spring Cloud and Sleuth, along with @SqsListener and the custom MessageConverter.
My Spring Boot 2.2.x and Sleuth v2.1.x, when a message is received, I don't ...
0
votes
1
answer
24
views
SQS SendMessageBatch Concurrency Limit
What is the maximum concurrency limit on SQS FIFO Queue? If I call 200 sendMessageBatch APIs with 10 messages(< 256KB) per API invocation. Will SQS be able to handle this?
Sample Code:
const ...
1
vote
1
answer
35
views
Does AWS SQS MessageGroupId ensure that messages of the same group are handled by the same lambda?
I have one sqs-fifo receiving payment notifications from another service that insert items into this sqs-fifo queue.
I also have one lambda that is configured to be triggered by the sqs-fifo (lambda ...
0
votes
1
answer
29
views
SNS redrive to Dead letter queue not working
I have a queue with a policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
&...
0
votes
1
answer
38
views
Dynamically create AWS SQS DLQ
I am trying to alter my AWS SQS module:
This is the before:
resource "aws_sqs_queue" "main" {
name = var.queue_name
redrive_policy = jsonencode({
...
1
vote
1
answer
39
views
How to send S3 object path or URI when an object created event is generated
I am new to AWS, I wanted to get S3 object URI/path when the object creation event is generated. Object can be created anywhere in the bucket, like there can be multiple sub folders that dynamically ...
0
votes
0
answers
33
views
How to prevent lambda triggered multiple times by SQS
I have a lambda function that is triggered by SQS, but it seems that SQS triggered it multiple times even when the operation is succeeded. Here are some of my code
// handle.ts to handle sqs
exports....
0
votes
1
answer
25
views
Dynamic permission policy to access sqs queue based from name
I want to create a dynamic permission policy with attributes substitution that allows access to respective customer's queue (indicated as part of the SQS queue name)
For example:
SQS queue name: ...
0
votes
0
answers
30
views
SQS CloudWatch Log group
I saw CloudWatch metrics logged for my SQS queue and operations.
However I want to view the details and events/ data of the sent messages. I have cloudwatch log group permissions for my SQS queue, but ...
0
votes
1
answer
37
views
Test sending SQS message from role in Cloud9 having send / received message permission to queue but no message received
I was using Cloud9 to assume a role to test on the dynamic permission to send message to a queue.
In cloud9, i assumed this role with the following permissions
{
"Version": "2012-10-17&...
0
votes
1
answer
36
views
AWS FIFO SQS: How does SQS maintain ordering in a group when a message is stuck in DLQ?
I am planning to use AWS FIFO SQS to keep record of current status of each item in my datastore. I will be using the unique identifier of each item as the messageGroupId to ensure strict ordering of ...
0
votes
0
answers
16
views
NO error but still messages are moved to DLQ
I am using SQS queue for task execution and DLQ enabled for this queue. In my case successfully processed task messages are also moved to DLQ..
We cross checked in our logs for particular task no nay ...
0
votes
0
answers
41
views
Polling multiple SQS messages using Airflow SQSSensor
I am using this SQSSensoe settings to poll messages
fetch_sqs_message = SQSSensor(
task_id="...",
sqs_queue="...",
aws_conn_id="aws_default",
...
0
votes
1
answer
41
views
How to remove deprecated AWS queues/topics created by MassTransit?
I'm using MassTransit AmazonSQS and I would want to know if there is any way of removing/deleting the queues and the topics that got created by MassTransit,for which there is no more need to exist, ...