Amazon Simple Queue Service (SQS).

 

Amazon Simple Queue Service (SQS) is a fully managed message queuing service that enables twtech to decouple and scale microservices, distributed systems, and serverless applications.

 It allows components of twtech application to communicate and coordinate by exchanging messages through a queue in a reliable, scalable, and secure way.

 Key Concepts of Amazon SQS

1. Queue

·        A buffer that stores messages until they are processed.

·        SQS provides two types of queues:

o   Standard Queue (default):

§  Nearly unlimited throughput

§  At-least-once delivery (duplicate messages possible)

§  Best-effort ordering (not guaranteed)

o   FIFO Queue (First-In-First-Out):

§  Exactly-once processing

§  Messages are processed in the order sent

§  Limited throughput (up to 300 messages/sec with batching)

2. Message

·        A message is the data sent between components (up to 256 KB in size).

·        Can include metadata (message attributes) for filtering/routing.

3. Visibility Timeout

·        The time during which a message is "invisible" after being received by a consumer.

·        Prevents multiple consumers from processing the same message.

4. Dead-Letter Queue (DLQ)

·        Used to isolate messages that can’t be processed successfully.

·        Helps in troubleshooting issues by capturing failed messages.

5. Long Polling

·        Reduces cost and latency by waiting up to 20 seconds for a message to arrive before returning a response.

·        Helps avoid empty responses.

6. Delay Queues

·        Introduces a delay (0–15 mins) for all new messages added to the queue.

 Benefits of SQS

·        Fully managed (no servers to manage)

·        Scales automatically

·        Secure with AWS IAM and KMS

·        Integrates easily with other AWS services (Lambda, EC2, SNS)

·        High availability and durability

 Common Use Cases

·        Decoupling microservices

·        Task offloading (e.g., video encoding, image processing)

·        Event buffering (e.g., logging systems)

·        Load leveling (smoothing spikes in traffic)

·        Retry mechanisms for failed processes.

Project:  Hands-on

How twtech creates and use SQS to decouple its microservices applications.

Search the aws services: SQS(simple queue service).

Create queue:

Assign a name for the Queue: twtech-Queue

Configuration

Set the maximum message size, visibility to other consumers, and message retention.

Encryption

Amazon SQS provides in-transit encryption by default. To add at-rest encryption to your queue, enable server-side encryption.

Access policy

Define who can access your queue.

Create Queue: twtech-Queue

How twtech eventually send and receive messages from Queue created: twtech-Queue

From the  UI of SQS(simple queue service)  , Click on: send and receive messages.

From:

To:

How twtech accesses the messages sent on on SQS(simple queue service) UI: Poll for messages.

Messages are being: Polled

From:

To:

Click on the message ID to see the message:  sent and polled.

More information about the message can be got from: Datails tab

twtech –insight:

If the message is not processed within the configured polling duration (set for 30 seconds), the number of counts would be increasing.

Once the message has been processed, twtech goes ahead and deleted from the SQS(simple queue service) : Confirm and delete message.

This is how twtech demonstrates the power of SQS(simple queue service)  messaging between the producer and the consumers.

How twtech may reconfigure the attributes of the SQS(simple queue service).

How twtech may delete all the messages in the SQS (simple queue service): Purge in dev env

Confirm and permanently purge(delete all messages in the Queue): Purge (never purge SQS in prod)

NB:

Purge action is only acceptable in the dev or test environment.

Messages should never be purged in the Prod(production) environment.

Messages in the SQS can be monitored to know the number of messages in the Queue.