S3 Events Sent to Multiple SQS Queues | Overview.

S3 Events Sent to Multiple SQS Queues  - Overview.

Scope:

• Intro,
• Architecture,
• Step-by-Step Guide to create S3 Events Sent to Multiple SQS Queues,
• Permissions Summary,
• Sample Use Case,
• Sample Use Case When a new file is uploaded to an S3 bucket.

Intro:

• To send Amazon S3 events to multiple SQS queues, twtech can combine S3 → SNS → SQS fan-out. 
• Direct S3-to-multiple-SQS isn’t supported, but SNS acts as a scalable intermediary.

Architecture: S3 → SNS → Multiple SQS Queues

 Step-by-Step Guide to create S3 Events Sent to Multiple SQS Queues 

1. Create an SNS Topic

# bash

aws sns create-topic --name twtechs3-event-topic

2. Create SQS Queues

# bash

aws sqs create-queue --queue-name twtech-queue-a

aws sqs create-queue --queue-name twtech-queue-b

3. Subscribe SQS Queues to SNS Topic

Get the ARNs:

# bash

aws sns subscribe --topic-arn <twtech-sns-topic-arn> \

  --protocol sqs --notification-endpoint <sqs-twtech-queue-a-arn>

aws sns subscribe --topic-arn <twtech-sns-topic-arn> \

  --protocol sqs --notification-endpoint <sqs-twtech-queue-b-arn>

 # Make sure each SQS queue's access policy allows the SNS topic to send messages.

4. Attach S3 Event Notification to SNS Topic

Go to S3 → Properties → Event Notifications and set:

• Event types (e.g., s3:ObjectCreated:*)
• Destination: SNS Topic

Or use AWS CLI:

# bash

aws s3api put-bucket-notification-configuration \

  --bucket twtech-s3bucket \

  --notification-configuration '{

    "TopicConfigurations": [{

      "TopicArn": "arn:aws:sns:us-east-2:accountID:twtech-s3-event-topic",

      "Events": ["twtech-s3:ObjectCreated:*"]

    }]

  }'

 Permissions Summary

• S3 bucket needs permission to publish to the SNS topic.
• SNS topic must have access to publish to SQS queues.
• Update SQS queue policy like:

# json

{

  "Version": "2012-10-17",

  "Statement": [{

    "Effect": "Allow",

    "Principal": {"Service": "sns.amazonaws.com"},

    "Action": "sqs:SendMessage",

    "Resource": "arn:aws:sqs:us-east-2:accountID:twtech-queue-name",

    "Condition": {

      "ArnEquals": {"aws:SourceArn": "arn:aws:sns:us-east-2:accountID:twtech-s3-event-topic"}

    }

  }]

}

 Sample Use Case When a new file is uploaded to an S3 bucket:

• Queue A → Triggers ...media processing.
• Queue B → Triggers ...metadata extraction.
• Queue C → Triggers ....auditing/logging.