Amazon S3 – Replication (CRR & SRR)
Amazon S3 Replication allows twtech to
automatically and asynchronously copy objects across S3 buckets, either within
the same AWS region or across different regions. This is useful for compliance,
lower latency, disaster recovery, and data sovereignty.
Types
of Replication
1. Cross-Region Replication (CRR)
·
Definition:
Automatically replicates S3 objects from one AWS region to a bucket in a
different AWS region.
·
Use
Cases:
o Compliance
with geographic data residency requirements.
o Disaster
recovery.
o Lower
latency access in different regions.
2. Same-Region Replication (SRR)
·
Definition:
Replicates objects between buckets in the same AWS region.
·
Use
Cases:
o Logging
or auditing.
o Live
data processing in multiple buckets.
o Maintaining
multiple copies within the same region for resilience.
Key
Features
·
Selective
replication using prefix or tag filters
·
Replication
of delete markers (optional)
·
Preserves
object metadata and ACLs (optional)
·
Supports
replicating new objects and object metadata changes
·
Can
replicate encrypted objects (SSE-S3, SSE-KMS, or customer-managed keys)
·
Supports
object ownership override with bucket owner enforced setting
·
Can
replicate existing objects using S3 Batch Replication
How to Set Up Replication
1.
Source
and destination buckets:
o Must
have versioning enabled.
o Need
appropriate IAM permissions.
2.
Create
IAM Role: Grant S3 permission to replicate objects on twtech
behalf.
3.
Configure
Replication Rule:
o Choose
all or filtered objects (by prefix or tag).
o Enable/disable
delete marker replication.
o Optionally
override object ownership.
4.
Save
and activate rule.
Benefits
·
Automatic
backup
·
Compliance
and data sovereignty
·
Improved
availability and durability
·
Multi-region
access for performance
Limitations
·
Versioning
must be enabled on both buckets.
·
Replication
is not retroactive (unless using S3 Batch Replication).
·
Metadata
changes (e.g., ACL updates) are not replicated unless you re-upload the object.
·
There’s
a replication delay (asynchronous process).
· Additional costs apply (storage, requests, inter-region data transfer).
Use Case Examples
|
Use Case |
Solution |
Type |
|
Disaster Recovery in another region |
Replicate objects to another region |
CRR |
|
Real-time analytics in a secondary bucket |
Replicate in-region for processing |
SRR |
|
Compliance with data residency laws |
Keep data within region |
SRR |
|
Global low-latency data access |
Distribute copies across regions |
CRR |
Project: Hands-on
How twtech creates replications on
s3 buckets: SRR/CRR
Go to s3 console and create an s3 bucket:
twtech-s3bucket
Object Ownership
Bucket Versioning: enabled
NB: Replication only works if s3 versioning is enabled.
Create another bucket (target bucket) in: us-west-1(N.California)
Switch region to : us-west-1(N.California).
Assign a name: twtech-s3bucket-Replica.
Cross region replication (CRR) from:
us-east-2 (Ohio) to us-west-1 (N. California) : CRR
Also enable bucket versioning
Assign a name to the target bucket (replica): twtech-s3bucket-crr-replica
Create the bucket target(replica): twtech-s3bucket-replication
The prmary bucket is: twtech-s3bucket in us-east-2 (Ohio)
The secondary bucket is: twtech-s3bucket-crr-replca
in us-west-1 (N. California)
Uploads in the primary bucket:
twtech-s3bucket in us-east-2 (Ohio)
Switch to us-east-region to upload flile to the primary bucket.
Upload a jpg file:
Upload file:
How twtech set up replication for its files:
Select the primary bucket to replicate and click open: On
the origine bucket
Navigate to management tab:
twtech-s3bucket-crr-replca in us-west-1 (N.
California)
Create a replication rule: twtech-s3bucket-crr-replication-rule.
Destination: Choose a bucket
in this account
Replication rule name: twtech-s3bucket-crr-replca
in us-west-1 (N. California).
Source bucket: twtech-s3bucket
Choose a rule scope: Apply to all objects in the bucket.
Destination: twtech-s3bucket-crr-replca in us-west-1 (N.
California)
from:
Select destination bucket for replication: twtech-s3bucket-crr-replca
in us-west-1 (N.
California)
To:
No, don’t replicate any existing objects:
Or:
Yes, replicate any existing objects.
Switch to the primary bucket in us-east-2 (ohio) and
upload files: twtech-s3bucket
Upload file: Papa.jpg
How twtech verifies that files from the bucket in
use-east-1 (Ohio) are been replicated in the bucket
in us-west-1 ( N. California)
Switch to region with the replication bucket click open the
Destination bucket: twtech-s3bucket-crr-replica
Successfully twtech has the the files are replicated
from twtech-s3bucket in us-east-2 (Ohio) to the
twtech-s3bucket-crr-replica in us-west-1(N. California)
From origin bucket: twtech-s3bucket
To Destination bucket: twtech-s3bucket-crr-replica
How twtech configures the delele marker replication.
Switch to the original bucket on which the replication
rule was created:twtech-s3bucket
Select management tab to edit the replication rule:
Select the rule to edit: twtech-s3bucket-crr-replication-rule
Scroll down to edit : Delete marker replication
From
To: by default, delete marker replication are not
enabled from creation: twtech is expected to enable the option after
creation.
Save changes: update the replication rule
twtech can now switch to the bucket (twtech-s3bucket-crr-replcica) in us-west-1 (N. California) to verify that a
version of the object deleted, was created by: deleted
marker replication.
Turn on the show version to see the object created by: Deleted
marked replication
From:
How twtech may also choose to permanently remove the delete markers version of the object replicated.
Verify permanent deletion of : Delete
marker version
From: 2 delete marker version
To: one left
The concept of the:
Delete Marker Replication in Amazon S3
In versioned S3 buckets, when
twtech deletes an object, Amazon S3 doesn’t immediately remove the object.
Instead, it adds a delete marker — a special kind of object that becomes
the current version and hides previous versions.
Delete Marker Replication
Delete Marker Replication is a feature of S3 Replication (SRR or CRR) that determines whether delete markers should be replicated from the source bucket to the destination bucket.
When
It's Enabled
- A delete marker added to the source bucket is also
added to the destination bucket.
- Ensures both source and destination reflect the same
current state (i.e., object appears deleted in both).
- Useful for compliance, synchronization, and
consistent versioning across replicated environments.
When It's Disabled
- The delete marker is not replicated.
- The object remains accessible in the destination
bucket, even though it’s "deleted" in the source.
- This allows for asymmetric retention or recovery
scenarios.
Example Scenario
Let’s say you're replicating s3://twtech-s3bucket (source) to s3://twtech-s3bucket-crr-replica (destination):
- twtech deletes file.txt in twtech-s3bucket
- A delete marker is created in twtech-s3bucket
- If delete marker replication is enabled, twtech-s3bucket-crr-replica also gets a delete marker → file.txt
disappears from both buckets.
- If disabled, file.txt is still visible in twtech-s3bucket-crr-replica
Important Notes
- Works only in versioned buckets.
- Doesn’t replicate actual object deletions (e.g.,
deleting a specific version).
- Not enabled by default — you must opt in when configuring replication rules.
Use Cases
|
Use Case |
Enable Delete Marker Replication |
|
Disaster recovery &
consistency |
✅ Yes |
|
Asymmetric retention policies |
❌ No |
|
Multi-environment dev/test setups |
❌ No |
|
Legal compliance (e.g., GDPR) |
✅ Yes |
twtech –insights:
With version enabled, the same objects uploaded several times will appear as different versions of that object upload.
This gives twtech the possibility to roll back (deleted versions) to
original version if updated versions are not okay.
The Delete marker version of deleted object can be restored if they were accidentally removed by twtech junior engineers.
No comments:
Post a Comment