Monday, June 23, 2025

Amazon S3 | Storage Lens & Default Dashboard.

 

Amazon S3 | Storage Lens Overview.

Amazon S3 Storage Lens is a cloud storage analytics tool built into S3 that gives twtech organization-wide visibility into storage usage and activity trends.

It's designed to help twtech optimize costs, improve performance, and enhance data protection strategies.

The concept:  S3 Storage Lens

S3 Storage Lens provides metrics and insights across all twtech buckets and accounts (in an AWS Organization), helping twtech analyze and act on:

Category

Insights Provided

Usage

Object count, storage size by class, versions, etc.

Activity

Requests, bytes uploaded/downloaded, etc.

Cost Optimization

Unused storage, non-current versions, infrequent access candidates.

Data Protection

Encryption usage, replication status, etc.

 Key Features

Feature

Description

Organization-wide metrics

Aggregated insights across all accounts and buckets.

Pre-configured dashboard

AWS Console provides an easy-to-use, visual dashboard.

Metrics export

Export data to S3 in CSV or Parquet format for custom analysis.

Free & Advanced tiers

Free tier includes ~30 metrics; Advanced tier includes 35+ more, including activity metrics.

Daily metrics

Data updated daily (no real-time metrics).

 Metrics Categories (Examples)

Category

Free Tier

Advanced Tier (Paid)

Storage

Total bytes, object counts

Per storage class, per prefix

Activity

❌ (not available)

PUT, GET, DELETE counts, 4xx/5xx

Cost Optimization

Limited

Stale noncurrent versions, deletable bytes

Data Protection

Default encryption usage

Replication and MFA delete status

 Use Cases

  1. Cost Optimization
    • Identify buckets storing infrequently accessed or incomplete multipart uploads.
  2. Lifecycle Policy Tuning
    • Find buckets with too many old noncurrent versions.
  3. Security Posture Review
    • Spot buckets not using default encryption or without replication enabled.
  4. Operational Analysis
    • Discover high-activity buckets and optimize for performance/load.

 How twtech Enables Storage Lens | Via AWS Console:

  1. Go to S3 → Storage Lens.
  2. Click Create dashboard.
  3. Choose scope (account or organization).
  4. Select metrics tier (free or advanced).
  5. Optionally enable metrics export to an S3 bucket.
  6. Review and create.

 Metrics Export Example (S3)

Exported to S3 in CSV or Parquet format with data such as:

# csv

account_id,bucket,region,storage_class,total_bytes,object_count,...

1234567xxxx,twtech-s3bucket,us-east-2,STANDARD,104857600,500,...

twtech can analyze this data using Amazon Athena, Redshift, or QuickSight.

Storage Lens – Default Dashboard 

The default dashboard in Amazon S3 Storage Lens provides preconfigured visibility into your S3 storage usage and activity trends across all regions, buckets, and accounts (if using AWS Organizations).

It is automatically created when twtech enable Storage Lens.

 Dashboard Overview

Element

Description

Scope

Choose between AWS account or organization-wide view.

Time Range

Filter metrics by last 1, 7, 30, or 90 days.

Filters

Drill down by region, bucket, storage class, prefix, or account.

Widgets

Visual cards for storage, object count, encryption, versioning, request activity, and more.

 Key Metrics in the Default Dashboard

Section

Metrics Shown

🔸 Storage Summary

- Total storage by class
- Number of objects
- Average object size

🔸 Cost Optimization

- Noncurrent version bytes
- Incomplete multipart uploads
- Lifecycle policy coverage

🔸 Data Protection

- Encryption usage
- Replication status
- MFA delete coverage

🔸 Activity Metrics

(Advanced tier only)
- PUT/GET/DELETE requests
- 4xx/5xx errors
- Bytes uploaded/downloaded

🔸 Top-N Reports

- Top buckets by size, object count, versioned bytes, etc.

 Example Widgets (Visual Cards)

  1. Total Storage by Storage Class
    • Bar chart showing STANDARD, INTELLIGENT_TIERING, GLACIER, etc.
  2. Object Count Over Time
    • Line graph of object growth trends.
  3. Unencrypted Storage
    • Pie chart showing % of objects without default encryption.
  4. Buckets With No Lifecycle Rules
    • Highlighted risk area for cost optimization.

 Metrics Export (Optional)

You can configure the dashboard to export metrics to an S3 bucket in CSV or Parquet format for deeper analysis using:

  • Amazon Athena
  • Amazon QuickSight
  • Amazon Redshift

 How twtech may Access the Default Dashboard

  1. Open the Amazon S3 console.
  2. In the left menu, click Storage Lens.
  3. Select the dashboard.
  4. View metrics by date range, scope, filters, or download/export options.

 twtech-Best Practices 

  • Use the “Top N Buckets” views to detect hot spots or unused data.
  • Monitor for high 4xx/5xx error rates to fix performance issues (advanced tier).
  • Identify buckets with no encryption, replication, or lifecycle for policy improvement.
  • Regularly compare trends (30 vs. 90 days) to forecast growth or unusual spikes.
  • Use Athena or QuickSight for deeper analytics on exported data.
  • Set up alerts in CloudWatch based on thresholds (e.g., growth spikes).
  • Enable advanced metrics if twtech needs detailed request-level insights.
  • Combine with S3 Lifecycle policies for automation based on insights.
Addendum:
https://aws.amazon.com/blogs/aws/s3-storage-lens/

No comments:

Post a Comment

Kubernetes Clusters | Upstream Vs Downstream.

  The terms "upstream" and "downstream" in the context of Kubernetes clusters often refer to the direction of code fl...