Amazon S3 (Object Storage Service) | Overview & Hands-On.

Amazon S3 (Simple Object Storage Service) - Overview & Hands-On.

Scope:

•  The Concept of Amazon S3,
•  Key Features,
•  Setup Overview,
•  Benefits,
•  Limitations,
•  Common Use Cases,
•  S3 access Policies,
•  Hands-On.

 The Concept of Amazon S3

• Amazon S3 is an object storage service that provides scalable, durable, and secure storage for any type of data (files, images, backups, logs, etc.) over the internet. 
• S3 designed for high availability and allows twtech to store and retrieve any amount of data from anywhere.

 Key Features

Feature	Description
Object Storage	Stores data as objects within buckets.
Durability	99.999999999% (11 9’s) durability.
Scalability	Virtually unlimited storage.
Security & Access Control	IAM, bucket policies, ACLs, encryption (SSE, KMS).
Versioning	Keeps multiple versions of an object.
Lifecycle Management	Automate transition or deletion of objects.
Storage Classes	S3 Standard, Intelligent-Tiering, Glacier, etc.
Event Notifications	Triggers for Lambda, SNS, SQS, etc.
Static Website Hosting	Host websites directly from a bucket.
Relication	Cross-Region or Same-Region replication (CRR/SRR).
Query in Place	Use S3 Select or Athena to query data without moving it.

 Setup Overview

1. Sign in to AWS Console
2. Go to Amazon S3
3. Create a Bucket
• Choose bucket name (globally unique...eg twtech-s3)
• Select region
• Configure options like versioning, encryption, etc.
4. Upload Objects
• Via console, CLI, SDK, or API
5. Manage Permissions
• Use IAM policies, bucket policies, ACLs
6. Enable Features
• Enable logging, events, lifecycle rules, etc.

Benefits

• High Durability and Availability
• Elastic Scalability
• Secure and Compliant
• Cost-Effective with different storage classes
• Easy Integration with AWS services (EC2, Lambda, Athena, etc.)
• Global Access over HTTPS
• Pay-as-you-go Pricing

 Limitations

Limitation	Description
Object Size Limit	Max object size is 5 TB (single PUT up to 5 GB).
Eventual Consistency	Some operations may be eventually consistent (mostly mitigated).
No File System Semantics	Not suitable for applications requiring traditional file system behavior.
Latency	Not ideal for real-time performance-critical workloads.
Cost Management	Can become costly without proper lifecycle and access management.

 Common Use Cases

Use Case	Description
Backup & Restore	Store backups, snapshots, and disaster recovery data.
Big Data Analytics	Store logs and datasets for processing using EMR, Athena, etc.
Media Hosting	Store images, videos, and static files.
Static Website Hosting	Host static HTML/CSS/JS websites directly.
Software Delivery	Distribute files, updates, binaries.
Data Archiving	Store cold data in Glacier or Glacier Deep Archive.
IoT Data Storage	Store telemetry data from devices.
Machine Learning	Store training data for ML models (integrates with SageMaker, etc.).

Project: Hands-on

How twtech creates and use s3 buckets for resources in it evironment.

Step-1:

• twtech searches for AWS service: s3

• Create a bucket: twtechs3

Step-2:

wtech configures Object Ownership:

• twtech disables ACLs for: security purpose
• twtech Controls ownership of objects written to this bucket from other AWS accounts and the use of access control lists (ACLs). 
• Object ownership determines who can specify access to twtech objects.

Step-3:

• twtech Blocks Public Access settings for this bucket: To harden the security posture.

Step-4:

• twtech enables the Bucket Versioning: enabled (optional but recommended).
• To subsequently see its deleted object cache
• To later apply Object Lock ( that prevents objects from being accidentally deleted by junior engineers)  

Step-5:

• twtech sets Default encryption
• Select Encryption type: Server-side encryption with Amazon S3 managed keys (SSE-S3)

Step-6:

• twtech sets Advanced settings: Object Lock 
• Add layer of Security Posture by preventing objects from being deleted.
• Store objects using a write-once-read-many (WORM) model to help twtech prevent objects from being deleted or overwritten for a fixed amount of time or indefinitely. Object Lock works only in versioned buckets:
• Enable object locking: To prevent bucket from accidental deletion by junior engineers

Step-7:

• Create the bucket: twtechs3

Step-8:

• twtech adds (uploads) files to its s3 bucket:
• Select the bucket to upload files to and click open.

• Upload files: Music, video, images, documents (txt, pdf,words)

• twtech adds Files to its s3 buckets:

Step-9:

• twtech adds Entire folders to its s3 buckets: Music folder

• twtech Confirms to upload the files in the folder: Upload

NB:

• Uploading the files in the folder, this may take a couple of minutes depending on the size of the folder.
• From: in progresss of upload.

To:

Upload successful.

Step-10:

• To view or access the objects internally by the object owner: 
• Select the object, then click on open from the task bar.

• This routes twtech traffic to the url of the object: Don’t share this link,  it would leak the object to the public 

 https://twtechs3.s3.us-east-2.amazonaws.com/Music/Bob%20Marley.mp3?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PA4iyPLLDw0UBrsggsRPj2JJH34Aaii2KqC6z%2BQWdBsipJptj7BnYO9vVolz7wtYay%2B%2Bg5QA0OTf0SFPtOz1DXd9DTy00pma2bhGCP4k7MBmCrki%2FXs%2F%2Bxxxxxxxxxxxxxq2p1tsytCWRBnsUFjm8H35pzdGt7%2B%2BMTpXLhAYzxhGQoF11aL&X-Amz-Signature=51dda3b7266a31266d6ee4c7a5ddb54ff28a8288285e3e6a1ad3502ba959ebe6&X-Amz-SignedHeaders=host&response-content-disposition=inline

Step-11:

• If twtech tries to access the object url from public, there will be access denied message,
• twtech needs to enable public access to the its files, 

NB

• Blocking public access to S3 object addes a layer to the security posture.

https://twtechs3.s3.us-east-2.amazonaws.com/Music/Bob+Marley.mp3

• Access denied: Permissions need to be edited to allow public access.

Step-11:

•  twtech eventually enables public access specific objects that would not jeopardize security posture of the Organization (or leak information),
• Very important Security step to always consider.,
• Select the file to enable public access:  Bob marley-mp3

• twtech Goes to permission and edit to:  unblock or allow public access.

• Uncheck: Block public access (bucket settings)

From:

To 

uncheck the:  block public access 

• Save changes and type confirm to make changes: confirm

• Block of all public access has been turned:  off

Step-12:

• Create a bucket policy to allow access:  To specific objects only.
• Use aws policy generator: edit bucket policy.

• Edit bucket policy with: Policy generator

• AWS Policy Generator
• Select the bucket policy type: s3 Bucket Policy.

Step-13:

 Add statement(s)

Allow anyone(wildcat): *

• twtech needs the Amazon Resource Name (ARN):arn:aws:s3:::twtechs3/* 
• The /* allows public access to everything  in the s3 bucket (twtechs3).

Step-14:

• If twtech wants to allow only a particular object (isolate 0bjects) it will use /<object-name> in the Policy configuration.
• Example: arn:aws:s3:::twtechs3/Bob Marley.mp3

• Add Statement: 

• Add Statement and generate the Policy:

Step-15:

# twtech Copies the policy generated and paste on the: Bucket policy

{

  "Version": "2012-10-17",

  "Statement": [

    {

      "Sid": "Statement1",

      "Effect": "Allow",

      "Principal": "*",

      "Action": [

        "s3:GetObject"

      ],

      "Resource": "arn:aws:s3:::twtechs3/*"

    }

  ]

}

Step-16:

• twtech must save configure chages to the s3 bucket.

Step-17:

twtech eventually access the objects in the public (internet) with the url: https://twtechs3.s3.us-east-2.amazonaws.com/Music/Bob+Marley.mp3

Step-18:

• twtech downloads the object in public to local environmetn: click the three vertical dots , then Download