Amazon S3 : Baseline Performance.

 

Amazon S3 – Baseline Performance

Amazon S3 (Simple Storage Service) is designed to offer high durability, availability, and scalable performance. When we talk about baseline performance, we refer to the expected performance that S3 provides without requiring special tuning or configurations.

 Baseline Performance Characteristics

1. Request Rate Performance

·        No performance limits per prefix:

o   As of recent improvements, **S3 supports unlimited request rates per prefix (used to be limited to 3,500 PUT/POST/DELETE and 5,500 GET requests/sec per prefix).

·        S3 scales automatically to handle high request rates, so twtech doesn’t need to shard prefixes manually anymore.

2. Object Size Limits

·        Single PUT upload: Up to 5 GB per object.

·        Multipart Upload: Allows objects up to 5 TB.

·        Multipart uploads are recommended for files >100 MB, and required for files >5 GB.

3. Latency

·        Typical latency for GET and PUT operations is in the tens to low hundreds of milliseconds.

·        For latency-sensitive workloads, using S3 Transfer Acceleration or Amazon CloudFront can improve performance.

4. Throughput

·        Depends on network bandwidth and object size.

·        For large files or high throughput needs:

o   Use multipart uploads (parallelizes the upload).

o   For downloads, you can use Range GETs to retrieve different parts of an object in parallel.

 Best Practices for Baseline Performance

·        Use sequential or random key names—prefix-based performance limitations are no longer an issue.

·        Use multipart upload for large files.

·        Use parallelism (multithreading or multiprocessing) in twtech application for uploads/downloads.

·        Enable Transfer Acceleration for long-distance transfers (e.g., global users).

·        Consider S3 Intelligent-Tiering for balancing performance and cost over time.

 twtech Example Throughput

File Size	Single Upload Speed	With Multipart Upload
< 100 MB	Good performance	Not needed
100 MB – 5 GB	Slower if single PUT	Use multipart for faster transfer
> 5 GB	Not allowed (PUT)	Required (multipart)