Here’s a comprehensive breakdown of Amazon S3 (Simple Storage Service),.
Including the: concept,
key features, setup process, benefits, limitations,
and use cases.
The Concept of Amazon S3
Amazon S3 is an object storage service that provides scalable, durable, and secure storage for any type of data (files, images, backups, logs, etc.) over the internet. It's designed for high availability and allows you to store and retrieve any amount of data from anywhere.
Key Features
Feature |
Description |
Object Storage |
Stores data as objects within
buckets. |
Durability |
99.999999999% (11 9’s) durability. |
Scalability |
Virtually unlimited storage. |
Security & Access Control |
IAM, bucket policies, ACLs,
encryption (SSE, KMS). |
Versioning |
Keeps multiple versions of an
object. |
Lifecycle Management |
Automate transition or deletion of
objects. |
Storage Classes |
S3 Standard, Intelligent-Tiering,
Glacier, etc. |
Event Notifications |
Triggers for Lambda, SNS, SQS,
etc. |
Static Website Hosting |
Host websites directly from a
bucket. |
Relication |
Cross-Region or Same-Region
replication (CRR/SRR). |
Query in Place |
Use S3 Select or Athena to query
data without moving it. |
Setup Overview
- Sign in to AWS Console
- Go to Amazon S3
- Create a Bucket
- Choose bucket name (globally unique...eg twtech-s3)
- Select region
- Configure options like versioning, encryption, etc.
- Upload Objects
- Via console, CLI, SDK, or API
- Manage Permissions
- Use IAM policies, bucket policies, ACLs
- Enable Features
- Enable logging, events, lifecycle rules, etc.
Benefits
- High Durability and Availability
- Elastic Scalability
- Secure and Compliant
- Cost-Effective
with different storage classes
- Easy Integration
with AWS services (EC2, Lambda, Athena, etc.)
- Global Access
over HTTPS
- Pay-as-you-go Pricing
Limitations
Limitation |
Description |
Object Size Limit |
Max object size is 5 TB (single
PUT up to 5 GB). |
Eventual Consistency |
Some operations may be eventually
consistent (mostly mitigated). |
No File System Semantics |
Not suitable for applications
requiring traditional file system behavior. |
Latency |
Not ideal for real-time
performance-critical workloads. |
Cost Management |
Can become costly without proper
lifecycle and access management. |
Common Use Cases
Use
Case |
Description |
Backup & Restore |
Store backups, snapshots, and
disaster recovery data. |
Big Data Analytics |
Store logs and datasets for
processing using EMR, Athena, etc. |
Media Hosting |
Store images, videos, and static
files. |
Static Website Hosting |
Host static HTML/CSS/JS websites
directly. |
Software Delivery |
Distribute files, updates,
binaries. |
Data Archiving |
Store cold data in Glacier or
Glacier Deep Archive. |
IoT Data Storage |
Store telemetry data from devices. |
Machine Learning |
Store training data for ML models
(integrates with SageMaker, etc.). |
Project: Hands-on
How twtech creates and use s3
buckets for resources.
Go to aws service and search for: s3
Create a bucket: twtechs3
Object Ownership:
ACLs disabled for: security purpose
Control
ownership of objects written to this bucket from other AWS accounts and the use
of access control lists (ACLs). Object ownership determines who can specify
access to objects.
Block Public Access settings for this bucket: security
purpose
Bucket Versioning: enabled
Default encryption
Select Encryption type: Server-side encryption with
Amazon S3 managed keys (SSE-S3)
Advanced
settings
Store objects using a write-once-read-many (WORM) model to help you prevent objects from being deleted or overwritten for a fixed amount of time or
indefinitely. Object Lock works only in versioned buckets:
Enable object locking: To prevent bucket from accidental deletion by junior engineers
Create the bucket: twtechs3
How twtech adds (uploads) files to its s3 bucket:
Select the bucket to upload files
to and click open.
How twtech adds Files to its s3 buckets:
Confirm to upload the files in the folder: Upload
Uploading the files in the folder, this may take a
couple of minutes depending on the size of the folder.
From: in progresss of upload.
To:
Upload successful.
To view or access the objects internally by the owner: Select the object, then click on open from
the task bar.
This routes twtech traffic to the url of the object: Don’t share this link, it would leak the object to the public
https://twtechs3.s3.us-east-2.amazonaws.com/Music/Bob%20Marley.mp3?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=ASIA6IY357LYJI5FWVQD%2F20250616%2Fus-east-2%2Fs3%2Faws4_request&X-Amz-Date=20250616T150440Z&X-Amz-Expires=300&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEHcaCXVzLWVhc3QtMiJGMEQCIFtUgKV3ziZ6STpUmi4ZxCFryIw4ft3e8APz3xuMiuvgAiBHaHTTNXMCatBCk%2F4KITv1aUB9f9nyRgwPXiT9xgiJxSrWAghgEAAaDDk4MDkyMTc0NDExMiIM0H9H%2FxsEylGV1HQHKrMCJuvH9aMKY82NCKQz4vuskw8FWmd8x683C9RWwUtrczRWfjxhYSB1ca%2F2KZ0FH%2F4xHg3NYYsM%2F%2FXGP6GTdyZ%2BNdnW187ROuSOQDy%2FohPlNRFfNFty03gYnoG4wHEBFKBwP5bPzSWATDQDqLz0LGcoFZ5td2aATDjQPCxdphxe9G3zou35HMnW05PDmSn56Z1dM48fC1ypWD%2FUH8AfbTQ%2Fob1f%2BFTCkZ3fb4vHIjbwm9JnrVxdsaVBoKPRCr%2FJDsaToQzBPQmzV93ZReIRZ4zEkrsAUlJ8LrxLvgXcYVIn7EeHBbiNpTlB2dhjHe95PpR%2FIiw0vjiXMyQkQzdsIMCQRAEjQgqHmbwNhxTycUYj0uPgAOvtFIVc3PwXRzqlHHz4ySgwGlv%2B0JAfBsUdEG5eVi8YqTDCm8DCBjquAmMfOuJHw0bzJGVklV2qi30CAvbB7ECJUKPD0kFxeCpCGuVDwEcd0KDUAlTlC3mabaxxuX2MFH6%2FA4iyPLLDw0UBrsggsRPj2JJH34Aaii2KqC6z%2BQWdBsipJptj7BnYO9vVolz7wtYay%2B%2Bg5QA0OTf0SFPtOz1DXd9DTy00pma2bhGCP4k7MBmCrki%2FXs%2F%2Bxxxxxxxxxxxxxq2p1tsytCWRBnsUFjmGnFXDW3sPDgXREpC4rvPCkcPerKhdSQ8WUzIxI%2FEzQ4WGYq2KldVnD8f%2BWtNsunky4ZDcqBdDVgvPaCrszmCX1X1VKr1ZRUg1xcuqi3uIN%2FKqynq7FCEgmtpGfP8iiXqB5QMdLp00ms0ozjAZiHqxaSr8H35pzdGt7%2B%2BMTpXLhAYzxhGQoF11aL&X-Amz-Signature=51dda3b7266a31266d6ee4c7a5ddb54ff28a8288285e3e6a1ad3502ba959ebe6&X-Amz-SignedHeaders=host&response-content-disposition=inline
If twtech try to access the url from public: there will be access denied message.
https://twtechs3.s3.us-east-2.amazonaws.com/Music/Bob+Marley.mp3
How twtech eventually enables public access specific
objects that would not jeopardize security(or leak
information).
Select the file to enable public access: Bob
marley-mp3
Go to permission and edit: to allow public access.
Uncheck: Block public access (bucket settings)
From:
To uncheck the: block public access
Save changes and type confirm to make changes: confirm
Block of all public access has been turned: off
Create a bucket policy to allow access: To
specific objects only.
Use aws policy generator: edit bucket policy.
Edit bucket policy with: Policy generator
AWS Policy Generator
Select the bucket policy type: s3 Bucket Policy.
Step 2: Add
statement(s)
Allow anyone(wildcat):
*
We need the Amazon
Resource Name (ARN): arn:aws:s3:::twtechs3/*
/* : allows everything in the s3
bucket(twtechs3) to be access
If twtech wants to allow only a particular object ( isolate
0bjects) it will use /object-name: arn:aws:s3:::twtechs3/Bob
Marley.mp3
Add Statement:
Add Statement and generate the Policy:
Copy the policy generated and paste on the: Bucket policy
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Statement1",
"Effect": "Allow",
"Principal": "*",
"Action": [
"s3:GetObject"
],
"Resource": "arn:aws:s3:::twtechs3/*"
}
]
}
How twtech eventually access the objects in the publice with the url: https://twtechs3.s3.us-east-2.amazonaws.com/Music/Bob+Marley.mp3
To download the object in public: click the three vertical dots , then Download
No comments:
Post a Comment