Monday, June 16, 2025

Amazon S3 : Features, Setup, Benefits, Limitations, Use Cases

 

Here’s a comprehensive breakdown of Amazon S3 (Simple Storage Service),.

Including the: concept, key features, setup process, benefits, limitations, and use cases.

 The Concept of Amazon S3

Amazon S3 is an object storage service that provides scalable, durable, and secure storage for any type of data (files, images, backups, logs, etc.) over the internet. It's designed for high availability and allows you to store and retrieve any amount of data from anywhere.

 Key Features

Feature

Description

Object Storage

Stores data as objects within buckets.

Durability

99.999999999% (11 9’s) durability.

Scalability

Virtually unlimited storage.

Security & Access Control

IAM, bucket policies, ACLs, encryption (SSE, KMS).

Versioning

Keeps multiple versions of an object.

Lifecycle Management

Automate transition or deletion of objects.

Storage Classes

S3 Standard, Intelligent-Tiering, Glacier, etc.

Event Notifications

Triggers for Lambda, SNS, SQS, etc.

Static Website Hosting

Host websites directly from a bucket.

Relication

Cross-Region or Same-Region replication (CRR/SRR).

Query in Place

Use S3 Select or Athena to query data without moving it.

 Setup Overview

  1. Sign in to AWS Console
  2. Go to Amazon S3
  3. Create a Bucket
    • Choose bucket name (globally unique...eg twtech-s3)
    • Select region
    • Configure options like versioning, encryption, etc.
  4. Upload Objects
    • Via console, CLI, SDK, or API
  5. Manage Permissions
    • Use IAM policies, bucket policies, ACLs
  6. Enable Features
    • Enable logging, events, lifecycle rules, etc.

Benefits

  • High Durability and Availability
  • Elastic Scalability
  • Secure and Compliant
  • Cost-Effective with different storage classes
  • Easy Integration with AWS services (EC2, Lambda, Athena, etc.)
  • Global Access over HTTPS
  • Pay-as-you-go Pricing

 Limitations

Limitation

Description

Object Size Limit

Max object size is 5 TB (single PUT up to 5 GB).

Eventual Consistency

Some operations may be eventually consistent (mostly mitigated).

No File System Semantics

Not suitable for applications requiring traditional file system behavior.

Latency

Not ideal for real-time performance-critical workloads.

Cost Management

Can become costly without proper lifecycle and access management.

 Common Use Cases

Use Case

Description

Backup & Restore

Store backups, snapshots, and disaster recovery data.

Big Data Analytics

Store logs and datasets for processing using EMR, Athena, etc.

Media Hosting

Store images, videos, and static files.

Static Website Hosting

Host static HTML/CSS/JS websites directly.

Software Delivery

Distribute files, updates, binaries.

Data Archiving

Store cold data in Glacier or Glacier Deep Archive.

IoT Data Storage

Store telemetry data from devices.

Machine Learning

Store training data for ML models (integrates with SageMaker, etc.).

Project: Hands-on

How twtech creates and use s3 buckets for resources.

Go to aws service and search for: s3

Create a bucket: twtechs3

Object Ownership:

ACLs disabled for: security purpose

Control ownership of objects written to this bucket from other AWS accounts and the use of access control lists (ACLs). Object ownership determines who can specify access to objects.

Block Public Access settings for this bucket: security purpose

Bucket Versioning: enabled


Default encryption

Select Encryption type: Server-side encryption with Amazon S3 managed keys (SSE-S3)

Advanced settings

Store objects using a write-once-read-many (WORM) model to help you prevent objects from being deleted or overwritten for a fixed amount of time or indefinitely. Object Lock works only in versioned buckets:

Enable object locking: To prevent bucket from accidental deletion by junior engineers

Create the bucket: twtechs3

How twtech adds (uploads) files to its s3 bucket:

Select the bucket to upload files to and click open.

Upload files: Music, video, images, documents (txt, pdf,words)

How twtech adds Files to its s3 buckets:


How twtech adds Entire folders to its s3 buckets: Music folder


Confirm to upload the files in the folder: Upload


Uploading the files in the folder, this may take a couple of minutes depending on the size of the folder.

From: in progresss of upload.

To:

Upload successful.

To view or access the objects internally by the owner: Select the object, then click on open from the task bar.


This routes twtech traffic to the url of the object: Don’t share this link,  it would leak the object to the public 

 https://twtechs3.s3.us-east-2.amazonaws.com/Music/Bob%20Marley.mp3?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=ASIA6IY357LYJI5FWVQD%2F20250616%2Fus-east-2%2Fs3%2Faws4_request&X-Amz-Date=20250616T150440Z&X-Amz-Expires=300&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEHcaCXVzLWVhc3QtMiJGMEQCIFtUgKV3ziZ6STpUmi4ZxCFryIw4ft3e8APz3xuMiuvgAiBHaHTTNXMCatBCk%2F4KITv1aUB9f9nyRgwPXiT9xgiJxSrWAghgEAAaDDk4MDkyMTc0NDExMiIM0H9H%2FxsEylGV1HQHKrMCJuvH9aMKY82NCKQz4vuskw8FWmd8x683C9RWwUtrczRWfjxhYSB1ca%2F2KZ0FH%2F4xHg3NYYsM%2F%2FXGP6GTdyZ%2BNdnW187ROuSOQDy%2FohPlNRFfNFty03gYnoG4wHEBFKBwP5bPzSWATDQDqLz0LGcoFZ5td2aATDjQPCxdphxe9G3zou35HMnW05PDmSn56Z1dM48fC1ypWD%2FUH8AfbTQ%2Fob1f%2BFTCkZ3fb4vHIjbwm9JnrVxdsaVBoKPRCr%2FJDsaToQzBPQmzV93ZReIRZ4zEkrsAUlJ8LrxLvgXcYVIn7EeHBbiNpTlB2dhjHe95PpR%2FIiw0vjiXMyQkQzdsIMCQRAEjQgqHmbwNhxTycUYj0uPgAOvtFIVc3PwXRzqlHHz4ySgwGlv%2B0JAfBsUdEG5eVi8YqTDCm8DCBjquAmMfOuJHw0bzJGVklV2qi30CAvbB7ECJUKPD0kFxeCpCGuVDwEcd0KDUAlTlC3mabaxxuX2MFH6%2FA4iyPLLDw0UBrsggsRPj2JJH34Aaii2KqC6z%2BQWdBsipJptj7BnYO9vVolz7wtYay%2B%2Bg5QA0OTf0SFPtOz1DXd9DTy00pma2bhGCP4k7MBmCrki%2FXs%2F%2Bxxxxxxxxxxxxxq2p1tsytCWRBnsUFjmGnFXDW3sPDgXREpC4rvPCkcPerKhdSQ8WUzIxI%2FEzQ4WGYq2KldVnD8f%2BWtNsunky4ZDcqBdDVgvPaCrszmCX1X1VKr1ZRUg1xcuqi3uIN%2FKqynq7FCEgmtpGfP8iiXqB5QMdLp00ms0ozjAZiHqxaSr8H35pzdGt7%2B%2BMTpXLhAYzxhGQoF11aL&X-Amz-Signature=51dda3b7266a31266d6ee4c7a5ddb54ff28a8288285e3e6a1ad3502ba959ebe6&X-Amz-SignedHeaders=host&response-content-disposition=inline


How twtech enables public access to the its files: Very important security strategy. Not every (object goes public.)

If twtech try to access the url from public: there will be access denied message.

https://twtechs3.s3.us-east-2.amazonaws.com/Music/Bob+Marley.mp3


Access denied: Permissions need to be edited to allow public access.

How twtech eventually enables public access specific objects that would not jeopardize security(or leak information).

Select the file to enable public access:  Bob marley-mp3


Go to permission and edit: to allow public access.

Uncheck: Block public access (bucket settings)

From:

To uncheck the:  block public access 

Save changes and type confirm to make changes: confirm

Block of all public access has been turned:  off

Create a bucket policy to allow access:  To specific objects only.

Use aws policy generator: edit bucket policy.

Edit bucket policy with: Policy generator

AWS Policy Generator

Select the bucket policy type: s3 Bucket Policy.

Step 2: Add statement(s)

Allow anyone(wildcat): *


We need the Amazon Resource Name (ARN): arn:aws:s3:::twtechs3/*

/* : allows everything in the s3 bucket(twtechs3) to be access

If twtech wants to allow only a particular object ( isolate 0bjects) it will use /object-name: arn:aws:s3:::twtechs3/Bob Marley.mp3

Add Statement: 

Add Statement and generate the Policy:

Copy the policy generated and paste on the: Bucket policy

{

  "Version": "2012-10-17",

  "Statement": [

    {

      "Sid": "Statement1",

      "Effect": "Allow",

      "Principal": "*",

      "Action": [

        "s3:GetObject"

      ],

      "Resource": "arn:aws:s3:::twtechs3/*"

    }

  ]

}




 How twtech eventually access the objects in the publice with the url: https://twtechs3.s3.us-east-2.amazonaws.com/Music/Bob+Marley.mp3


To download the object in public: click the three vertical dots , then Download



No comments:

Post a Comment

Kubernetes Clusters | Upstream Vs Downstream.

  The terms "upstream" and "downstream" in the context of Kubernetes clusters often refer to the direction of code fl...