How are we doing? Please help us improve AWS. Take our short survey

Questions tagged [amazon-sqs]

Amazon Simple Queue Service (Amazon SQS) offers a reliable, highly scalable, hosted queue for storing messages as they travel between computers.

Filter by
Sorted by
Tagged with
0 votes
1 answer
16 views

Configure AWS SQS to take runtime exceptions into Dead Letter Queue

I am trying to integrate an Async Design with API Gateway -> SQS -> Lambda. The gateway receives a request and via the SQS integration at gateway we are passing the messages to SQS. The SQS then ...
Lazy Lizzard's user avatar
0 votes
0 answers
23 views

Can't able to resolve dependencies of the SqsService when i tried to use it in Custom Provider

I'm aiming to make the code less dependent on specific parts. My idea is to create a provider, which would allow me to easily switch between using a queueService or bypassing the queue during local ...
aasis504's user avatar
1 vote
0 answers
33 views

Connecting to Amazon SQS key (SSE-SQS) Encrypted SQS Queue via Django Celery [sqs]

I am unable to get my Celery working running while connecting to an SQS Encrypted Queue This is my settings.py for my django project SQS_AWS_ACCESS_KEY_ID = 'xxxx' SQS_AWS_SECRET_ACCESS_KEY = 'xxxx' ...
stackoverflow1234's user avatar
-1 votes
1 answer
23 views

Is there a pricing for using multiple threads to send data to a AWS SQS queue at the exact same moment?

Let's say an HTTP request posts data to a AWS SQS queue. In case if there are like 300 of these requests at the same time or maybe 3000 requests at the same time. Is this chargeable like for more than ...
Tarun Sharma's user avatar
0 votes
0 answers
13 views

AWS API Gateway Mapping Template for form submissions dies with empty form values

I have an API Gateway, with the following mapping template: application/x-www-form-urlencoded #set($form=$input.path("$").split("&")) Action=SendMessage&MessageBody={#...
Luke Dyson's user avatar
0 votes
2 answers
56 views

AWS: Only invoke lambda during a specific timeframe each day

I currently have a service that publishes a message to an SNS topic which has a subscription with name filter to add it to an SQS queue which ultimately triggers a lambda function. I have a use-case ...
Reece Young's user avatar
0 votes
1 answer
49 views

How to map request headers from ApiGatewayV2 to SQS event

I'm using dotnet framework 6.0, CDK v2 and ApiGatewayV2.Alpha I'm trying to send messages from the API Gateway directly to an SQS queue. An integration for this does not yet exist in this version, so ...
Tom's user avatar
  • 15
1 vote
1 answer
39 views

AWS SNS Filtering based on the Message Body

Sample message looks like below: { "Type" : "Notification", "MessageId" : "23456f-9cca-8f1a-5746363", "Message" : "{\"Records\":[{...
Rohith's user avatar
  • 1,097
0 votes
0 answers
15 views

AWS Webhook custom template results in 403 when going to sqs

I am running into a strange problem whilst trying to leveraging a custom mapping template in my webhook which has an integration with an SQS service. As I understand it, the purpose of the mapping ...
SomeStudent's user avatar
  • 2,931
0 votes
0 answers
16 views

SQS Min Batch size when triggering Lambda [duplicate]

I have an SQS queue that gets messages into it not very frequently. And I want to run a lambda when there are enough items in the queue to process (let's say 10). Is there any way to trigger a lambda ...
Tolga Evcimen's user avatar
-1 votes
1 answer
43 views

How to efficiently consume messages from an Amazon Simple Queue Service (SQS) in Golang? [closed]

I want to create an consumer in Golang but theres a few articles and info about it. So i'm accepting any suggestion about how to do in a reliable way! I tried using channels and some work arounds but ...
M. David's user avatar
-1 votes
0 answers
17 views

What alternatives can provide the exact same features as AWS SQS?

I am currently using AWS SQS fifo queues with the message group id functionality to enforce ordering within subgroups. However, I have to find a replacement due to the limitation of Sqs where message ...
fxxixx's user avatar
  • 114
0 votes
0 answers
26 views

All the messages from SQS goes into DLQ even after successful execution of Lambda

Type: AWS::SQS::Queue Properties: QueueName: "notification-service-queue.fifo" FifoQueue: true ContentBasedDeduplication: true VisibilityTimeout: 180 FifoThroughputLimit: ...
Abu Tahir's user avatar
0 votes
0 answers
6 views

Optimizing Real-time IoT Data Processing: Harnessing the Power of SQS with Kinesis Data Streams

I'm in the midst of architecting a robust solution for real-time processing of IoT streaming data and would like to seek guidance on whether incorporating Amazon Simple Queue Service (SQS) in ...
anonymous_33008899's user avatar
0 votes
1 answer
36 views

retain message in sqs even after read

I've a Airflow Dag, which runs for multiple tables processing, but for one table at a time. Dag instances run in parallel if need to run more than one table processing. My problem is, I need to send ...
morgan's user avatar
  • 29
0 votes
1 answer
42 views

Why should the SQS visibility timeout for Lambda bet set to 6?

The AWS doc states To allow your function time to process each batch of records, set the source queue's visibility timeout to at least six times the timeout that you configure on your function. The ...
John Doe's user avatar
  • 444
0 votes
0 answers
38 views

Spring boot SQS listener makes message visible if error/exception in processing

As per documentation , The visibility timeout begins when Amazon SQS returns a message. During this time, the consumer processes and deletes the message. However, if the consumer fails before deleting ...
Viraj Nalawade's user avatar
0 votes
0 answers
19 views

Localstack throws The security token included in the request is invalid when it tries to send sqs message

Localstack throws The security token included in the request is invalid when it tries to send a message. It seems like it tries to connect to real AWS. Below is my configuration: companion ...
Janani Sampath Kumar's user avatar
0 votes
2 answers
40 views

Buffer of 30 mins between Lambda invocations

I want to invoke lambda multiple times but not concurrent with invocation having 30mins(some minutes) having event custom information & buffer in between 2 lambda triggers. So basically, 1 lambda ...
Nikhil Gupta's user avatar
-1 votes
1 answer
23 views

How to allow anonymous access for SendMessage to Amazon SQS?

The AWS SQS docs states that anonymous access is supported: https://aws.amazon.com/sqs/faqs/#:~:text=Q%3A%20Does%20Amazon%20SQS%20support%20anonymous%20access%3F It also provides an example policy ...
Michael Pu's user avatar
0 votes
0 answers
24 views

What is the correct syntax for Swagger 2.0 AWS requestParameters for MessageAttributes ? to map them from body to header

I'm not familiar enough with AWS to verify whether the syntax for mapping requestParameters is correct for AWS Swagger 2.0 config. The message will be send to SQS with a body like this { "...
John Smith's user avatar
0 votes
1 answer
178 views

Getting error while using @aws-sdk/client-sqs in node.js

While using the sendMessage method of SQS getting the following error error SyntaxError: Unexpected token T in JSON at position 0 Deserialization error: to see the raw response, inspect the hidden ...
n9dhal's user avatar
  • 18
1 vote
1 answer
52 views

SQS - Step function pipe failed

I am new to step functions and have a Pipe to poll SQS queue and initiate the step function execution per message received. The intention is to process one message at a time. However, the Step ...
user3000538's user avatar
-1 votes
1 answer
32 views

What is the purpose of an sqs in a stepfunction?

So I'm trying to wrap my head around what the purpose of having an sqs in a stepfunction is. Lets say there's a lambda that sends messages to an queue, and then another lambda that consumes the ...
leed38's user avatar
  • 314
0 votes
1 answer
38 views

How to configure celery with SQS as backend?

I'm trying to setup a SQS broker with a celery app, configured in a django project. Here's my setup: celery.py: import os from celery import Celery os.environ.setdefault("DJANGO_SETTINGS_MODULE&...
koleror's user avatar
  • 338
1 vote
2 answers
76 views

SQS Messages not deleted after DLQ Implementation

I recently implemented DLQ(Dead Letter Queue) for SQS. I have done the following three configurations. Default visibility timeout in SQS. In the Dead Letter Queue configuration, enable and Maximum ...
bgs's user avatar
  • 3,101
0 votes
1 answer
28 views

SQS subscribed to multiple SNS with different data models

Context I have 2 SNS both publishing messages with different data model. I want a lambda to get these messages and do some processing. Creating a SQS between the SNS and Lambda is mandatory. What i ...
Codemaster's user avatar
0 votes
1 answer
48 views

How can I trigger a job after 15 minutes in Laravel?

I need to create a job that is going to be trigger after 15 minutes a post is created, I can set 15 minutes of delay, but imagine that the job is going to take 1 minute to process, in total will be 16 ...
DeveloperX's user avatar
1 vote
0 answers
14 views

AWS Textract Asynchronous Queue Messaging

I'm getting the following error when calling DeleteMessageCommand for my pdf processing. error InvalidAddress: The address arn:aws:sns:us-west-1:**** is not valid for this endpoint. Here's where the ...
Sean's user avatar
  • 1,379
0 votes
0 answers
35 views

How to remove some jobs from running in Laravel

I have one worker currently running, utilizing SQS with Laravel Jobs. Within my jobs, I invoke third-party services to perform actions on my products. Each job corresponds to a specific product (e.g., ...
rabie ali's user avatar
0 votes
0 answers
34 views

AWS SQS message push taking 50 secs

For some of the requests, sqsClient send message taking 50+ seconds to push message in SQS queue. How to pass CLientConfiguration so that timeout happens quickly and accordingly retry happens. Tried ...
Ashish Gupta's user avatar
0 votes
0 answers
44 views

Customize Request Timeout while publish message to SQS using java sdk - Reducing SQS Message Processing Time: Minimizing Retries after 10 Seconds

We are using SQS in our Java Spring Boot Project. While publishing messages to SQS, most of the messages are processed in <=300ms. However, some requests took more than 50 seconds. Upon initial ...
Piyush Maheshwari's user avatar
0 votes
0 answers
26 views

How to use AWS SQSExtendedClient library to make it work asynchrously using JMSListener java

Am trying to achieve exactly what the below link is trying to do. Amazon SQS Async Listener for the Extended Client Library using Spring JmsListener I couldn't find AmazonS3Client and AmazonSQSAsync ...
Pradeep's user avatar
  • 1,232
1 vote
0 answers
29 views

Job completion tracking across kafka

I have a distributed pipeline that reads jobs from a Queue (SQS). Each job then produces millions of messages and sent over to Kafka topic TopicA to analyze. After the analysis is done, it’s then sent ...
cosmos's user avatar
  • 2,183
0 votes
1 answer
19 views

Will reducing retention period in AWS SQS boost the performance?

I have an application that has a Consumer and a Producer. The Producer will push the messages to the SQS. The Consumer is reading from the queue but reading the last message is taking too much time. I ...
Mukul Varshney's user avatar
0 votes
2 answers
60 views

Serialize consumers of SQS fifo queue in node app

I am trying to implement serialization in the consumers of a SQS queue, which runs in multiple nodeJs instances. i.e. what I am trying to do is ensure that stuff is done in the same order they came in,...
aggaton's user avatar
  • 3,134
0 votes
1 answer
51 views

How to send large number of mails (more than sending rate limit) at a time in AWS SES

I have an AWS SES service with a sending rate of 100 mails per second. Our application sends emails to all students within educational institutes. There's a need to distribute a substantial volume of ...
Voldemort's user avatar
0 votes
1 answer
189 views

AWS SQS FIFO in Python - deleted messages are available even after correct VisbilityTimeout

I created an FIFO queue. I send 10 msgs, receive them and delete them. But they are still available in queue and received again. I receive 10 msgs initially, then 5 , then 2 and then 0. "...
JTX's user avatar
  • 3
1 vote
0 answers
25 views

Overlook AWS credentials from env and instead use IAM role attached to node_group of EKS to push msgs to SQS

Problem: I have some downstream services which needs to push msgs to sqs using my python package but boto3 uses the credentials defined in their respect env instead of using IAM role of node_group. I ...
Mansi Chauhan's user avatar
0 votes
1 answer
93 views

Backend stack with microservice architecture

Currently I am migrating my CRM to microservice architecture. Is this arch suitable to be called solid and robust? (High loads are expected in future) Main point of this architecture is one service - ...
vitaliyJR's user avatar
0 votes
0 answers
21 views

Difference between 0 delay schedule and white true loop in SQS long polling Java

I'm writing a custom SQS long polling service. Basically Option-1 while(True) { //long poll SQS for 20 seconds // process messages } Option-2 using ScheduledExecutorService with scheduling of 0 delay....
Katuri Vijaya's user avatar
0 votes
0 answers
25 views

How to get message payload on @SqsListener exception handling

I have a scenario where I want send notification in case of validation failure public void onMessage(@Valid @Payload String jsonPayload, @Headers Map<String, String> ...
riteshmaurya's user avatar
0 votes
0 answers
19 views

How apache camel ensures idempotent messaging with SQS

Standard SQS queues guarantee that a message is delivered at least once and duplicates can be introduced into the queue. So we need to handle the scenario where in we receive same message multiple ...
Viraj Nalawade's user avatar
1 vote
1 answer
26 views

SQSEvent object needed for SQS triggered lambda function in postman

I am trying to test my SQS triggered lambda function in my local machine using postman. I am able to call the function using postman but it requires SQSEvent type object as input. How do I provide ...
Jalaj Goyal's user avatar
0 votes
0 answers
23 views

What is the use of endpoint in sns.subscribe in case of sns fan-out events

I'm working on a service that uses an SNS-SQS fanout model in Python. I understand that in order to send a notification to a specific SQS, I need to provide a filter policy. However, I'm confused ...
J3ck's user avatar
  • 1
1 vote
1 answer
47 views

Auto-scaling group based on SQS depth

I am writing Terraform code that does the following: sets up S3 bucket and S3 event notifications, also Lambda that is being triggered by S3 notifications. Then it sends the messages to SQS. And then ...
dnd1993's user avatar
  • 141
0 votes
0 answers
38 views

Java SQS client throughput doesn't scale up in parallel

I want to poll and process SQS messages with high throughput. I have a number of threads, each with a SQS poller in a Spring @Async. At the moment the code is simply printing the message. My hope is ...
Joe's user avatar
  • 46.8k
0 votes
1 answer
37 views

Some events are not available when the EventBridge rule triggers the push to SQS

I have an EventBridge rule that gets triggered on the insert and update of a specific item type. The data that satisfies the rule is then pushed to an SQS queue, which in turn triggers a Lambda ...
Sparsh Rawat's user avatar
0 votes
0 answers
45 views

Celery worker is not starting for my django project hosted on elastic beanstalk

error in the eb consoleGood day by co-developers. I have a project working on aws elasticbeanstalk and i want to integerate celery to it but the celery worker is not starting and everything is ...
Shittu Abdulrasheed Ayomide 's user avatar
0 votes
1 answer
6 views

qbo3 Import Running Multiple Times

I have queued a qbo3 API call importfile/import to bulk import a file with 80M records. The job completes successfully, but continues to run. The logging shows: Queued @ 2:39:42 AM Processing @ 2:39:...
Eric Patrick's user avatar
  • 2,127

1
2 3 4 5
84