AWS Snowball is a physical device designed for secure,
large-scale data transfers between twtech on-premises infrastructure and AWS
cloud.
It helps move large datasets efficiently without saturating twtech network. Here’s a concise overview of data migrations using AWS Snowball:
1. Use Cases for Snowball Data Migration
- Large data transfers (typically over 10 TB)
- Limited or slow internet connectivity
- One-time migrations or recurring batch transfers
- Migrations to Amazon S3, EBS Snapshots, or into an Amazon Glacier vault
2. Types of Snowball Devices
Device |
Capacity
(usable) |
Compute
Capabilities |
Notes |
Snowball Edge Storage Optimized |
~80 TB |
Optional EC2, Lambda |
For large-scale data transfer and
local processing |
Snowball Edge Compute Optimized |
~42 TB |
Enhanced EC2, GPU, Lambda |
For edge computing with local
analytics/ML |
Snowcone |
8 TB |
Lightweight compute |
Small scale, portable, can run
offline |
3. Migration Workflow
- Create a Snowball job
in AWS Snow Family Management Console.
- AWS ships the device
to twtech location (encrypted and tamper-resistant).
- Connect the device
to twtech local network via Ethernet.
- Use Snowball client tools (Snowball CLI, S3 SDK
Adapter) to transfer data.
- Ship the device back
to AWS using the pre-attached label.
- AWS uploads the data
to twtech specified S3 bucket.
- Data is automatically deleted from the device after successful transfer.
4. Data Security
- 256-bit encryption
with AWS Key Management Service (KMS)
- Tamper-evident enclosures
- Trusted Platform Module (TPM) for hardware-based key protection
- Chain of custody tracking
5. Best Practices
- Use Snowball Edge for compute-intensive tasks
(e.g., preprocessing before upload)
- Verify checksums before and after data transfer
- Parallelize uploads using multiple workers
- Organize data to optimize transfer (e.g., avoid too
many small files)
6. Alternatives & Comparisons
Option |
Best
For |
Speed |
Limits |
Snowball |
10s of TBs+ |
Days (shipping + transfer) |
Physical shipping time |
Direct Connect |
Continuous large data pipelines |
Fastest (Gbps-level) |
Setup cost & time |
AWS DataSync |
Online migrations (up to 10s of
TBs) |
Slower than Snowball |
Dependent on network |
S3 Transfer Acceleration |
Internet-based S3 uploads |
Faster than standard S3 |
Extra cost, depends on latency |
No comments:
Post a Comment