AWS S3 Event Notifications with Amazon EventBridge | Deep Dive.

AWS S3 Event Notifications with Amazon EventBridge - Deep Dive.

Sceope:

•  Intro,      
• The reason we Use EventBridge Instead of Native S3 Notifications,
•  How S3 Sends Events to EventBridge,
•  Event Structure,
•  Event Filtering with EventBridge Rules,
•  Common Architectures,
•  Reliability & Delivery Guarantees,
•  Best Practices (Production),
•  Comparison Table: S3 (Native) vs EventBridge,
•  Architecture Diagram.

Intro:

• Amazon EventBridge is the newer, more reliable, more flexible mechanism for consuming S3 notifications.
• Amazon EventBridge  solves several long-standing limitations of native S3 → SNS/SQS/Lambda event notifications.

1. The reason we Use EventBridge Instead of Native S3 Notifications

Benefits

Feature	Native S3 Notifications	S3 → EventBridge
Delivery Guarantee	Best effort (can lose events)	Guaranteed at-least-once
Filtering	Prefix/suffix only	Advanced JSON-based filtering
Multiple Consumers	Only 1 destination per event	Unlimited consumers
Event Replay	No	EventBridge Archive + Replay
Auditing	No	CloudTrail integration
Schema Registry	No	EventBridge Schema Registry

NB:

• EventBridge is the modern, recommended approach for high-integrity event-driven architectures.

 2. How S3 Sends Events to EventBridge

NB:

• For most regions, S3 event delivery to EventBridge is automatically enabled (some older regions require enabling manually).

Flow looks like:

NB:

• There is no need to configure S3 notification rules in the bucket (they only apply to SNS/SQS/Lambda).

 3. Event Structure (S3 → EventBridge)

A typical S3 event:

# json

{

  "version": "0",

  "id": "abcd-1234-efgh-5678",

  "detail-type": "Object Created",

  "source": "aws.s3",

  "account": "accountId",

  "time": "2025-02-01T12:34:56Z",

  "region": "us-east-2",

  "resources": [

    "arn:aws:s3:::twtech-s3bucket"

  ],

  "detail": {

    "bucket": {

      "name": "twtech-s3bucket"

    },

    "object": {

      "key": "uploads/file1.csv",

      "size": 2048,

      "etag": "abcd1234...",

      "version-id": "xyz987"

    },

    "request-id": "123ABC456",

    "requester": "arn:aws:iam::accountID:role/twtech-s3-role",

    "source-ip-address": "192.0.2.0",

    "reason": "PutObject"

  }

}

Key benefits:

•  Rich metadata
•  Consistent schema
•  Works with EventBridge filtering rules

 4. Event Filtering with EventBridge Rules

• EventBridge supports advanced JSON/logic filtering:

Sample: Only process .csv uploads in uploads/ prefix

# json

{

  "source": ["aws.s3"],

  "detail-type": ["Object Created"],

  "detail": {

    "bucket": { "name": ["twtech-s3bucket"] },

    "object": {

      "key": [{ "prefix": "uploads/" }, { "suffix": ".csv" }]

    }

  }

}

Sample: Only route large files

# json

{

  "detail": {

    "object": {

      "size": [{ "numeric": [">", 100000000] }]

    }

  }

}

NB:

• EventBridge filtering removes the need for deduplication and noise suppression in Lambda.

 5. Common Architectures

Architecture A: S3 → EventBridge → Lambda

[ S3 ] → [ EventBridge Rule ] → [ Lambda ]

Best for:

• Real-time processing
• Simple flows
•  Fan-out

Architecture B: S3 → EventBridge → SQS → Lambda Worker Pool

S3 → EventBridge → SQS → Lambda (Batch Processor)

Best for:

• High-volume ingestion
• Need backpressure
• Guaranteed processing + DLQ

Architecture C: S3 → EventBridge → Step Functions (ETL/ML pipelines)

S3 → EventBridge → Step Functions → Glue / EMR / Batch

Best for:

• Multi-step workflows
• ML model pipelines

Architecture D: S3 → EventBridge Pipes → Kinesis/Firehose/SQS/HTTP

S3 → EventBridge → EventBridge Pipes → (Target)

Best for:

• Direct integration without Lambda
• Streaming or system-to-system pipelines

 6. Reliability & Delivery Guarantees

Unlike native S3 notifications:

•        EventBridge guarantees delivery
•        EventBridge supports retries
•        EventBridge supports DLQs when combined with targets like SQS

NB:

• This solves long-standing S3 event-loss issues.

 7. Best Practices (Production)

A. Use EventBridge, not native S3 notifications

• Unless a legacy integration forces SNS/SQS/Lambda directly.

B. Use SQS between EventBridge and Lambda for resilience

• Avoid Lambda throttling or invocation failures during spikes.

C. Use EventBridge Archive + Replay

• Keeps all events for replaying failed processing streams.

D. Use EventBridge schema registry

• Auto-discovers S3 schemas, eliminates event format guesswork.

E. Build idempotent consumers

• Even with EB’s guarantees, at-least-once delivery requires deduplication:
• Use object.version-id
• Track processed objects in DynamoDB

F. Route Different S3 Event Types to Different Services

• Use EB rules to create a microservices fan-out architecture:

ObjectCreated      → Lambda A  

ObjectRemoved      → Lambda B  

RestoreCompleted   → SNS alert  

Large size         → SQS priority queue  

 8. Comparison Table: S3 (Native) vs EventBridge

Feature	Native S3 Notifications	S3 → EventBridge
Delivery	Best effort	Guaranteed
Fan-out	Limited	Unlimited
Filtering	Simple	Complex JSON logic
DLQ (Dead-Letter-Gueue)	No	Yes (via target)
Replay	No	Yes
Schema Registry	No	Yes
Auditing	Limited	Full CloudTrail integration

NB:

• EventBridge is better for pro, mission-critical, and high-scale architectures.

 9. Architecture Diagram