AWS Service of Messaging (SNS) & Integration with SQS, & Kinesis | Overview

AWS Service of  Messaging (SNS) & Integration with SQS, & Kinesis - Overview.
Scope:

• Intro,
• AWS Messaging service & Integration with SQS, & Kinesis,
• Amazon SNS (Simple Notification Service),
• Amazon SQS (Simple Queue Service),
• Amazon Kinesis (Data Streams),
• Summary Table,
• Recommendation of When to Use Which,
• Comparison table for AWS SNS, SQS, & Kinesis. 

Intro:

Here’s twtech Overview of AWS Integration & Messaging service (SNS) Plus integration with SQS, and Kinesis.

AWS Messaging service & Integration with SQS, & Kinesis

1. Amazon SNS (Simple Notification Service)

Concept

•         Fully managed pub/sub messaging service for broadcasting messages to multiple subscribers.
•         Pushes notifications to endpoints (email, SMS, HTTP/S, Lambda, SQS).

Key Features

• ·       Supports multiple protocols (HTTP/S, Email, SMS, Lambda, SQS, Mobile Push).
•         Message filtering policies to deliver messages selectively.
•         Fan-out architecture by publishing to multiple subscribers simultaneously.
•         Server-side encryption (SSE).
•         Message durability with replication across multiple AZs.

Benefits

•         Enables real-time, push-based messaging.
•         Simplifies fan-out to multiple systems/endpoints.
•         Supports both direct notifications and triggering downstream AWS services.
•         Scales automatically without user management.

Limitations

•         No message persistence (no queueing or retry beyond delivery attempts).
•         No message ordering guarantees.
•         Limited message size (up to 256 KB).
•         Delivery retries follow a fixed policy with limited configurability.

Use Cases

•         Mobile app push notifications.
•         System alerts and monitoring notifications.
•         Fan-out messaging to trigger workflows (e.g., send SMS, invoke Lambda, queue jobs).
•         Broadcasting events to multiple distributed systems.

2. Amazon SQS (Simple Queue Service)

Concept

•         Message Queue service designed to decouple distributed systems and microservices.
•         Stores messages until processed and deleted by consumers.
•         Supports asynchronous, reliable communication between components.

Key Features

•         Two queue types:
•    Standard queues: At-least-once delivery, best-effort ordering, unlimited throughput.
•    FIFO queues: Exactly-once processing, guaranteed order, limited throughput.
•         Visibility timeout: Temporarily hides messages during processing.
•         Dead-letter queues: For handling message failures.
•         Server-side encryption with AWS KMS.
•         Long polling to reduce empty responses and costs.

Benefits

•         Fully managed, scalable, and highly available.
•         Decouples producers and consumers, improving fault tolerance.
•         Supports batching of messages to optimize throughput.
•         Integrates with AWS Lambda for serverless processing.

Limitations

•         FIFO queues have throughput limits (up to 300 TPS without batching).
•         No native support for message filtering or pub/sub model.
•         Messages can be delivered multiple times (Standard queues).

Use Cases

•         Asynchronous task processing (e.g., order processing, image resizing).
•         Buffering requests between components to handle load spikes.
•         Decoupling microservices in event-driven architectures.

3. Amazon Kinesis (Data Streams)

Concept

•         Managed service for real-time streaming data ingestion and processing.
•         Allows building data pipelines and streaming applications.

Key Features

•         Sharded data streams provide ordering and parallel processing.
•         Data retention (up to 7 days by default, extendable).
•         Integrates with AWS Lambda, Firehose, and analytics tools.
•         Exactly-once processing with enhanced fan-out.
•         Real-time data ingestion with millisecond latency.

Benefits

•         Enables near real-time analytics and data processing.
•         Scalable and durable streaming infrastructure.
•         Supports complex event processing and aggregation.
•         Integrates well with big data tools and AWS ecosystem.

Limitations

•         Requires manual shard scaling or use of auto-scaling features.
•         More complex to set up and manage compared to SQS/SNS.
•         Higher cost for large scale, continuous streaming.

Use Cases

•         Real-time log and event data ingestion.
•         IoT telemetry and sensor data streaming.
•         Financial transactions and fraud detection pipelines.
•         Machine learning model training with streaming data.

Summary Table

Aspect	Amazon SQS	Amazon SNS	Amazon Kinesis
Messaging Model	Message queue (point-to-point)	Pub/Sub (broadcast)	Streaming data ingestion
Delivery Model	Pull-based	Push-based	Pull-based (sharded streams)
Ordering	FIFO queues support ordering	No ordering guarantees	Ordering within shards
Delivery Guarantees	At least once (standard), Exactly once (FIFO)	Best effort delivery	Exactly-once with enhanced fan-out
Protocols	AWS API	HTTP/S, Email, SMS, Lambda, SQS	AWS API
Retention	4 days max	No retention	Up to 7 days (configurable)
Scalability	Automatic	Automatic	Shard-based scaling
Use Cases	Decoupling, buffering	Notifications, fan-out	Real-time analytics and streaming

twtech Comparison for AWS SNS, SQS, and Kinesis. 

Feature	Amazon SQS	Amazon SNS	Amazon Kinesis
Purpose	Message queuing (decoupling components, buffering)	Pub/Sub messaging (broadcasting)	Real-time streaming data processing
Message Model	Point-to-point (queue)	Publish-subscribe (topic)	Data streams (ordered records)
Use Case Examples	Decoupling microservices, buffering tasks, asynchronous processing	Notifications, fan-out to multiple subscribers	Analytics pipelines, real-time metrics, log processing
Message Retention	Up to 14 days	No retention (delivers messages immediately)	Data retention up to 7 days (Kinesis Data Streams)
Delivery Guarantees	At least once delivery, with visibility timeout	Best effort delivery to multiple subscribers	Exactly-once processing (with enhanced fan-out)
Protocol Support	N/A (used via API or SDK)	HTTP/S, Email, SMS, Lambda, SQS, Mobile Push	API, SDK, Kinesis Agents
Scaling	Automatic scaling	Automatic scaling	Scales by shards (manual or automatic via enhanced features)
Ordering	FIFO queues available	No ordering guarantees	Strict ordering within shards
Integration	Can trigger Lambda, used with SNS for fan-out	Pushes to multiple endpoints including SQS queues, Lambda	Integrates with AWS analytics and processing tools like Lambda, Firehose, EMR

twtech Recommendation of When to Use Which

Scenario	Service
Decouple components with reliable, asynchronous message queuing	SQS
Send notifications or messages to multiple subscribers	SNS
Ingest, process large streams of data & Fan-out in real-time	Kinesis