Amazon S3 Moving (Transitioning) objects Between Storage Classes - Overview & Hands-On.
Scope:
- Intro,
- Lifecycle Configuration Rules (Automatic),
- Architecture diagram
- Sample Lifecycle Transitions,
- How to Set Up (CLI),
- Sample lifecycle.json,
- Manual Movement (One-time or Ad Hoc),
- Upload with Storage Class Specified,
- Available Storage Classes for Transitions,
- Best Practices
- Hands-On.
Intro:
- Moving data between Amazon S3 storage classes is a common practice to optimize costs as data ages or access patterns change.
- There are several ways to move objects between S3 storage classes:
Architecture diagram:
1. Lifecycle Configuration Rules (Automatic)
twtech can define lifecycle policies
to automatically transition objects between storage classes based on object
age or creation date.
Sample Lifecycle Transitions:
# yaml
Transition:
- Day 30
→ S3 Standard-IA
- Day 90
→ S3 Glacier Instant Retrieval
- Day 180
→ S3 Glacier Deep Archive
How to Set Up:
- AWS Console: Go to S3 bucket > Management > Lifecycle rules.
- AWS: CLI
# bash
aws
s3api put-bucket-lifecycle-configuration --bucket twtech-s3bucket
\
--lifecycle-configuration file://lifecycle.json
# Sample lifecycle.json:
# json
{
"Rules": [
{
"ID":
"TransitionToGlacier",
"Status": "Enabled",
"Prefix": "",
"Transitions": [
{
"Days": 30,
"StorageClass": "STANDARD_IA"
},
{
"Days": 90,
"StorageClass": "GLACIER"
}
],
"NoncurrentVersionTransitions":
[ ],
"Expiration": {
"Days": 365
}
}
]
}
2. Manual Movement (One-time or Ad Hoc)
- twtech Uses this when it needs to manually move a specific object or batch of objects to a different class.
AWS CLI:
# bash
aws s3 cp
s3://twtech-s3bucket/twtech-object.txt s3://twtech-s3bucket-crr-replica/twtech-object.txt \
--storage-class
GLACIER --metadata-directive REPLACE
NB:
- S3 does not allow in-place class changes. twtech must copy and overwrite the object to change its storage class.
3. Upload with Storage Class Specified
- twtech can specify the desired storage class at upload time.
AWS: CLI
# bash
aws s3 cp twtech-file.txt s3://twtech-s3bucket/ --storage-class STANDARD_IA
Available Storage Classes for Transitions
|
Storage Class |
Notes |
|
STANDARD |
Default |
|
INTELLIGENT_TIERING |
Auto-tiering based on access |
|
STANDARD_IA |
Infrequent Access |
|
ONEZONE_IA |
Lower-cost IA in one AZ |
|
GLACIER_INSTANT_RETRIEVAL |
Archive, fast access |
|
GLACIER |
Archive, flexible retrieval |
|
DEEP_ARCHIVE |
Cheapest, slowest access |
twtech Best Practices.
- Use Intelligent-Tiering for unpredictable
access.
- Use Lifecycle rules for predictable aging data.
- Use tag-based policies to transition only selected objects.
Project: Hands-on
How twtech creates its s3-buckets then edit storage class life cycle rules to: Move or transition objects betweenstorage classes.
- twtech Creates a bucket: twtech-s3-class-transitioning
- Assign name: twtech-s3-class-transition.
- Created in any region of choice:
us-east-2(Ohio)
- Create bucket in any region of choice:
us-east-2 (Ohio)
- Select object to Upload to the bucket:
twtech-logo4
- Configure storage class for object to upload: twt-logo4.jpg
- Navigate to properties: To configure the choice of
storage class desired for the object.
- Select the preferred storage class:
standard storage class.
- Go to the bucket to access object: twtech-s3-class-transition
- The object indicates a storage class: Standard.
- twtech changes the storage class of its objects in the bucket by
editing properties of the object: from standart to
- Select the object to edit the properties of the storage class and click open: twt-logo4.jpg
- Navigate to the section:
storage class to edit.
- From standard to: One Zone-IA (meaning this object can
only be stored in one zone only)
From:
To:
- Save changes:
- Return to the bucket and Verify that the storage class has
actually been changed: from standard to One Zone-IA
Yes.
- twtech creates lifecycle rule to move (transition) objects between search classes: by creating lifecyle rules.
- Go to: bucket
management tab.
- Create lifecycle rule:
twtech-lifecycle-rule
- Create lifecycle rule:
twtech-lifecycle-rule
No comments:
Post a Comment