Here's twtech comparison of Amazon SQS, SNS, and Kinesis,
highlighting their key differences and best use cases:
Overview
|
Feature |
SQS
(Simple Queue Service) |
SNS
(Simple Notification Service) |
Kinesis |
|
Type |
Message Queue (Pull) |
Pub/Sub Notification (Push) |
Data Streaming Platform |
|
Pattern |
Point-to-Point |
Publish-Subscribe |
Real-time data streaming |
|
Message Flow |
Producer → Queue → Consumer. |
Publisher → Topic → Subscribers. |
Producer → Stream → Consumer. |
Use Case Scenarios
|
Use
Case |
Recommended
Service |
|
Decoupling microservices or
systems |
SQS |
|
Broadcasting messages to multiple
systems (e.g., SMS, Email, Lambda) |
SNS |
|
Real-time analytics or processing
high-throughput data streams (e.g., logs, IoT) |
Kinesis |
Detailed Comparison
|
Feature |
SQS |
SNS |
Kinesis |
|
Delivery Type |
Pull-based |
Push-based |
Pull or Enhanced Fan-out |
|
Delivery Guarantee |
At-least-once (FIFO supports
exactly-once) |
At-least-once |
At-least-once (manual deduping
needed) |
|
Ordering |
FIFO queues preserve order |
No ordering guarantee |
Order per shard |
|
Durability |
4-day retention by default (14-day
max) |
Short-lived (no storage) |
24 hrs to 7 days retention |
|
Latency |
Low |
Very low |
Low (milliseconds) |
|
Throughput |
High (scales with batch, FIFO has
limits) |
Very high |
Extremely high (designed for
large-scale ingestion) |
|
Fan-out |
No (1-to-1 delivery) |
Yes (1-to-many) |
Yes (with enhanced fan-out) |
|
Retry Mechanism |
Yes (DLQs supported) |
Yes (retry policies) |
Yes (checkpointing via consumer
apps) |
|
Cost Model |
Per request & payload size |
Per request & delivery attempt |
Per shard-hour + PUT payload units |
Best Fit Summary
|
Use
Case |
Best
Option |
|
Decoupled task queues (e.g.,
background jobs, retries) |
SQS |
|
Notifications to multiple systems
(e.g., SMS + Email + Lambda) |
SNS |
|
Real-time log processing,
analytics, or ETL pipelines |
Kinesis |
Combine Them
These services are often used
together:
- SNS → SQS:
Broadcast messages to multiple SQS queues (fan-out pattern).
- Kinesis + Lambda + SQS: Real-time processing with fallback queues for failed
events.
·
A decision tree and architecture
diagram for choosing between Amazon SQS, SNS, and Kinesis
twtech Insights:
A decision tree and architecture
diagram for choosing between Amazon SQS, SNS, and Kinesis.
Here's a decision tree and an architecture diagram that helps twtech to choose
between Amazon SQS, SNS, and Kinesis based on its use case.
Decision
Tree: Choosing Between SQS, SNS, and Kinesis
# Plaintext ┌─────────────────────────────┐ │ Does twtech need real-time data │ │ streaming with analytics? │ └──────────────┬──────────────┘ │Yes ▼ ┌──────────────┐ │ Use Kinesis │ └──────────────┘ ▲ │No ┌──────────────┴──────────────┐ │ Does twtech want to deliver the │ │ same message to multiple │ │ consumers (fan-out)? │ └──────────────┬──────────────┘ │Yes ▼ ┌───────────────┐ │ Use SNS │ │ (fan-out) │ └───────────────┘ ▲ │No ┌──────────────┴──────────────┐ │ Does twtech need message queues │ │ with decoupling, retries, │ │ and delayed processing? │ └──────────────┬──────────────┘ │Yes ▼ ┌───────────────┐ │ Use SQS │ └───────────────┘ ▲ │No ┌─────────┴─────────┐ │ Consider SNS or │ │ another pattern │ └───────────────────┘ Architecture Diagram: Common Usage Scenarios
1. SNS
Fan-Out to SQS Queues
# Markdown ┌─────────────┐ │ Publisher │ └─────┬───────┘ │ ┌─────▼───────┐ │ SNS │ │ Topic │ └────┬─┬──────┘ │ │ ┌─────────▼ ▼─────────┐ │ │┌─────▼─────┐ ┌─────▼─────┐│ SQS Q1 │ │ SQS Q2 │└───────────┘ └───────────┘ Use
case: One message sent to SNS is delivered
to multiple consumers (via SQS or Lambda, etc.)
2. Kinesis Stream for Real-Time
Processing
# Arduino ┌──────────────┐ │ Data Source │ │ (App/Device) │ └──────┬───────┘ ▼ ┌──────────────┐ │ Kinesis │ │ Stream │ └────┬────┬────┘ │ │ ┌───────▼┐ ┌▼────────┐ │Lambda/ │ │ Custom │ │Consumer│ │Processor│ └────────┘ └─────────┘Use case: High-throughput data ingestion with real-time transformation or analytics.
3. SQS
for Decoupling Microservices
# markdown ┌─────────────┐ │ Producer │ └─────┬───────┘ ▼ ┌─────────────┐ │ SQS │ │ Queue │ └─────┬───────┘ ▼ ┌─────────────┐ │ Consumer │ │ (Worker) │ └─────────────┘Use case: Decouple microservices, retry failed jobs, buffer traffic.
No comments:
Post a Comment