Amazon SQS – Message Visibility Timeout
Visibility Timeout is a crucial feature in Amazon SQS
that controls how long a message stays invisible to other consumers after being
retrieved from the queue.
The concept:
When a consumer retrieves(polls) a message
from an SQS queue using ReceiveMessage, the message does not get deleted—it's just marked
as invisible for a certain period, called the visibility timeout.
This gives the consumer time to process and delete the message.
By default, the “message visibility timeout” is 30 seconds. That means the message has 30 seconds to be processed. When the message visibility timeout is over, the message becomes “visible” in the SQS(simple queue service).
If the message isn't deleted before
the timeout expires, it becomes visible again and can be retried or
redelivered to the same or another consumer.
If a message is not processed within the visibility timeout, it will be processed twice. A consumer may make API call for ChangeMessageVisibility to get more time.
If the visibility timeout is high (in hours), and consumer
crashes, re-processing would take time.
If visibility timeout is very low (say in seconds), duplicate messages will appear.
Importance of Amazon SQS Message Visibility Timeout:
- Prevents duplicate processing while a message is
being worked on.
- Ensures reliability: if a consumer fails or
crashes before processing is complete, the message will reappear for
someone else to handle.
- Helps in fault-tolerant and at-least-once
message delivery.
Key Points.
Feature |
Description |
Default |
30 seconds |
Range |
0 seconds to 12 hours |
Change per message |
Yes, using ChangeMessageVisibility |
Queue-level setting |
Yes, set during queue creation or
updated later |
Message reprocessing |
If a message isn't deleted within
the timeout, it will reappear for reprocessing |
Example Workflow.
- A message is sent to an SQS queue.
- A consumer receives the message → SQS sets it as
invisible (default: 30 seconds).
- The consumer processes the message.
- If successful → the consumer calls DeleteMessage
→ message is removed.
- If not → after timeout expires → the message becomes
visible again for reprocessing.
Changing the Visibility Timeout.
- Per-queue
(default for all messages):
Use AWS Console, CLI, or SDK to set VisibilityTimeout when creating or updating a queue. - Per-message
(useful for long-running jobs):
Use the ChangeMessageVisibility API to increase the timeout dynamically.
Common Pitfalls(limitations).
- Short visibility timeout: If your message processing takes longer than the
timeout, it may get redelivered and processed multiple times.
- Not deleting messages: Forgetting to delete processed messages leads to them being reprocessed after timeout.
twtech
Best Practices
- Set a visibility timeout slightly longer than the average
processing time.
- Use dead-letter queues (DLQs) for messages that
exceed the max receive count.
- Adjust visibility dynamically for long or variable processing times.
Project: Hands-on
How twtech configures visibility
timeout on its SQS(simple queue
service).
Go to SQS(simple queue service) UI: to
send a message.
Send and receive a message: Hello
twtech family
Poll for messages:
From:
To:
How twtech may change (edit) the default visibility
timeout for SQS(simple queue service)
From:
To: Remembeer to save
changes.
Insights:
The beauty of this configuration can be viewed by opening two UI for sending and receive messages.
No comments:
Post a Comment