Amazon Simple Queue Service (SQS) - Overview & Hands-On.
Scope:
- Intro,
- Key Concepts of Amazon SQS,
- Benefits of SQS,
- Common Use Cases,
- Project: Hands-on.
Intro:
- Amazon Simple Queue Service (SQS) is a fully managed message queuing service that enables twtech to decouple and scale microservices, distributed systems, and serverless applications.
- It allows components of twtech application to communicate and coordinate by exchanging messages through a queue in a reliable, scalable, and secure way.
Key
Concepts of Amazon SQS
1. Queue
- A buffer that stores messages until they are processed.
- SQS provides two types of queues:
- Nearly unlimited throughput
- At-least-once delivery (duplicate messages possible)
- Best-effort ordering (not guaranteed)
- Exactly-once processing
- Messages are processed in the order sent
- Limited throughput (up to 300 messages/sec with batching)
2. Message
- A message is the data sent between components (up to 256 KB in size).
- Can include metadata (message attributes) for filtering/routing.
3. Visibility Timeout
- The time during which a message is "invisible" after being received by a consumer.
- Prevents multiple consumers from processing the same message.
4. Dead-Letter Queue (DLQ)
- Used to isolate messages that can’t be processed successfully.
- Helps in troubleshooting issues by capturing failed messages.
5. Long Polling
- Reduces cost and latency by waiting up to 20 seconds for a message to arrive before returning a response.
- Helps avoid empty responses.
6. Delay Queues
- Introduces a delay (0–15 mins) for all new messages added to the queue.
Benefits of SQS
- Fully managed (no servers to manage)
- Scales automatically
- Secure with AWS IAM and KMS
- Integrates easily with other AWS services (Lambda, EC2, SNS)
- High availability and durability
Common Use Cases
- Decoupling microservices
- Task offloading (e.g., video encoding, image processing)
- Event buffering (e.g., logging systems)
- Load leveling (smoothing spikes in traffic)
- Retry mechanisms for failed processes.
Project: Hands-on
- How twtech creates and use SQS to decouple its microservices applications.
- Search the aws services: SQS (simple queue service).
- Create queue:
- Assign a name for the Queue: twtech-Queue
Configuration
- Set the maximum message size, visibility to other consumers, and message retention.
Encryption
- Amazon SQS provides in-transit encryption by default.
- To add at-rest encryption to twtech queue, enable server-side encryption.
Access
policy
- Define who can access twtech queue.
- Create Queue: twtech-Queue
- twtech eventually send and receive messages from Queue
created: twtech-Queue
- From the UI of SQS (simple
queue service) , Click on:
send and receive messages.
From:
To:
- twtech accesses the messages sent on on SQS (simple
queue service) UI: Poll for messages.
- Messages are being: Polled
From:
To:
- Click on the message ID to see the message: sent and polled.
- More information about the message can be got from: Datails
tab
- twtech –insight:
- If the message is not processed within the configured polling duration (set for 30 seconds), the number of counts would be increasing.
- Once the message has been processed, twtech goes ahead
and deleted from the SQS (simple queue
service) : Confirm and delete message.
- This is how twtech demonstrates the power
of SQS (simple queue
service) messaging between the producer and the
consumers.
- twtech may reconfigure the attributes of the SQS(simple queue service).
- twtech may delete all the messages in the SQS (simple queue service): Purge in dev env
- Confirm and permanently purge (delete all messages in the Queue): Purge (never purge SQS in prod)
NB:
- Purge action is only acceptable in the dev or test
environment.
- Messages should never be purged in the Prod (production) environment.
- Messages in the SQS can be monitored to know the number
of messages in the Queue.
No comments:
Post a Comment