A deep dive into Amazon DynamoDB … what it does, how it works and how twtech makes the most of it.
1. The Concept: DynamoDB
Amazon DynamoDB is AWS’s fully
managed NoSQL key-value and document database designed for:
- Low-latency
(single-digit millisecond reads/writes)
- Massive scalability
(handles millions of requests/sec)
- Serverless operation
(no manual provisioning of servers)
It’s built on principles from
Amazon’s internal Dynamo paper (2007), but with added features like on-demand
scaling, global replication, and integrated security.
# Simplified visual architecture
+ ---------------------- +
| Client Apps / API |
| (Lambda, API GW) |
+----------+-----------+
|
v
+----------------------+
| DAX Cluster (opt) |
| In-memory microsecond |
| latency cache |
+----------+-----------+
|
v
+---------+----------+
| DynamoDB Table |
+---------+----------+
|
v
| Partition Key Hash |
|
v
+---------------+---------------+
| | |
v v v
+------+-----+
+------+-----+ +------+-----+
|Partition 1 |
|Partition 2 | |Partition N |
|
SSD | |
SSD | |
SSD |
+------+-----+
+------+-----+ +------+-----+
|
v
| --> Streams --> Lambda / Kinesis |
|
v
| Global Tables Replication |
|
v
+ ----------------+ ----------------+
|
|
v v v
Region A Region B ..
Region N
2. Core Concepts
Data
Model
- Table
– top-level container for data.
- Items
– equivalent to a row in RDBMS.
- Attributes
– key-value pairs inside an item.
- Primary Key
– uniquely identifies each item:
- Partition key only (simple primary key)
Example: UserID - Partition key + Sort key (composite primary key)
Example: UserID + OrderDate
Indexes
- Primary Index
– automatically created using the table’s primary key.
- Local Secondary Index (LSI) – same partition key as the table, but different sort
key.
- Global Secondary Index (GSI) – different partition and sort key, can be queried
independently.
3. Storage & Partitioning
- Data is automatically sharded into partitions.
- Partition key value → hashed to determine which physical partition stores it.
- Partitions scale horizontally as throughput or storage
needs grow.
4. Capacity Modes
Two modes determine cost and
throughput:
- Provisioned Capacity
- twtech sets Read Capacity Units (RCUs) & Write
Capacity Units (WCUs).
- Optional Auto Scaling adjusts based on traffic.
- On-Demand Capacity
- Pay-per-request pricing.
- Ideal for unpredictable workloads.
5. Read & Write Internals
Read
Types
- Eventually Consistent Reads – lower latency, cheaper (1 RCU = 2 reads/sec for 4KB
items).
- Strongly Consistent Reads – always return latest data (1 RCU = 1 read/sec for
4KB items).
- Transactional Reads
– part of ACID transaction.
Write
Types
- PutItem
– insert or replace.
- UpdateItem
– modify existing.
- DeleteItem
– remove.
- BatchWriteItem
– write multiple items.
6. Advanced Features
- Streams
– real-time change logs for Lambda triggers or replication.
- Global Tables
– multi-region active-active replication.
- TTL (Time-to-Live)
– auto-delete expired items.
- Transactions
– ACID compliance for multiple items/tables.
- Point-in-Time Recovery (PITR) – restore to any second in last 35 days.
- DynamoDB Accelerator (DAX) – in-memory caching for microsecond reads.
7. Security
- IAM Policies
– control API access at table/item/attribute level.
- KMS Encryption
– at rest (default enabled).
- VPC Endpoints
– private network access.
8. Cost Drivers
- RCUs/WCUs
(provisioned mode) or request count (on-demand mode).
- Storage
(GB-month).
- Data transfer
across regions.
- Optional features
(DAX, Streams, PITR, Global Tables).
9. Common Use Cases
- User profile stores
- IoT telemetry ingestion
- Gaming leaderboards
- Session stores
- Serverless backends
- Real-time analytics with Streams + Lambda + Kinesis
10. High-Level Architecture Flow
# maths
Client App
(AWS SDK / API Gateway / Lambda)
↓
DynamoDB Table
↓
Partition Key → Hash → Physical Partition
↓
Data Storage (SSD)
11. Gotchas(things to watch out) & Best Practices
- Hot Partitions
– avoid skewed partition key distribution.
- Item Size Limit
– 400 KB max.
- Query vs Scan
– use Query whenever possible (faster, cheaper).
- Sparse Indexes
– GSI with non-existent attributes only stores relevant items.
- Write Sharding
– append random suffix to hot keys to spread load.
Project: Hands-On
How twtech creates and use its DynamoDB tables for items.
Search in the aws services for: DynamoDB
Create tables with name: twtech-dynamodb-table
Create table: Table details
Capacity calculator: Average item size (KB)
Read/write capacity settings: Capacity
mode
or
Warm throughput
Increasing the warm throughput value pre-warms the table to
handle planned peak events without throttling or scaling delays. By default,
warm throughput values are visible for all tables and global secondary indexes.
These values automatically adjust as twtech increases its provisioned
throughput or on-demand consumption without extra charges, but if twtech
chooses to change them manually, additional charges will apply.
Secondary indexes
Estimated read/write capacity cost
Here is the estimated total cost of provisioned read and write capacity for twtech table and indexes, based on its current settings.
Encryption at rest
All user data stored in Amazon DynamoDB is fully encrypted at
rest. By default, Amazon DynamoDB manages the encryption key, and twtech is not
charged any fee for using it.
Deletion protection: turn on
Deletion
protection is turned off by default. Deletion protection protects the table
from being deleted unintentionally. twtech can turn on deletion protection now,
and it can also turn it on after the table has been created.
Resource-based
policy:
auto-created by aws
The resource-based
policy, written in JSON, helps manage access to this DynamoDB table.
Example of policy link: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/rbac-examples.html
Create item: From
To:
How twtech Accesses table created
for its details:
Select table and click open: twtech-dynamodb-table
Exploring table items: explore table items
NB:
New Items can be inserted into dynmodb table: twtech-dynmodb-table
How twtech inserts items into its dynamodb table: twtech-dynamodb-table
Specify value to attribute(twtech-key1: twtech-pat
How twtech adds attributes into the table: Add new attribute
NB:
More items can be added to the tables, following the same steps as above.
No comments:
Post a Comment