Here is twtech deep-dive guide for RDS & Aurora MySQL migrations.
Scope:
- migration patterns,
- DMS internals,
- MySQL-specific replication considerations,
- Cutover approaches,
- Best-practice architectures.
Breakdown:
- Understanding RDS MySQL vs Aurora MySQL,
- Migration Approaches,
- How AWS DMS Works with MySQL,
- Preparing Source MySQL for
Migration,
- Schema Migration Process,
- DMS Task Best Practices (MySQL),
- Aurora MySQL-Specific
Considerations,
- Cutover Strategies,
- Validation & QA,
- Architecture Diagrams.
- Migrating workloads to Amazon RDS for MySQL or Amazon Aurora MySQL-Compatible Edition has its own unique behaviors, especially around binary logs (binlogs), replication formats, triggers, stored procedures, and schema compatibility.
Amazon RDS for MySQL
- Managed MySQL on EC2-based storage.
- Storage is EBS-backed.
- Instance performance is bounded by EBS throughput.
- Supports native MySQL functionality + many common plugins.
Amazon Aurora MySQL
- MySQL-compatible cluster engine using highly distributed storage.
- 5x throughput of MySQL due to optimized log-structured storage.
- Auto-scaling storage (up to 128 TiB).
- Sub-10 second failovers.
- Some MySQL features unsupported (e.g., SUPER user, certain plugins).
Migration Tip:
Aurora
requires binlog_row_image=FULL for DMS CDC. Validate before migration.
2. Migration Approaches
A. Homogeneous Migration
Source: MySQL → Target: RDS MySQL or Aurora MySQL
Use when:
- Same engine family
- Few schema changes
- Moderate downtime permitted
Tools:
- mysqldump / mysqlpump
- Percona XtraBackup
- AWS DMS for minimal downtime
B. Zero/Minimal-Downtime Migration
Use:
- AWS DMS (Full Load + CDC)
- Native MySQL replication
- Percona hot-copy options
- Aurora fast cloning (for test iterations)
This is the most common migration path.
3. How AWS DMS Works with MySQL
A. Full Load
- Bulk data copy using multiple threads.
B. Change Data Capture (CDC)
Reads MySQL binlogs from:
- On-prem MySQL
- EC2 MySQL
- RDS MySQL binlog
- Aurora MySQL binlog
DMS requires:
binlog_format = ROWbinlog_row_image = FULLexpire_logs_days = <large enough>C. Apply
Phase
DMS replays INSERT/UPDATE/DELETE on the target.
4. Preparing Source MySQL for Migration
A. Enable Binary Logging
# In my.cnf:
server_id = 100log_bin = mysql-binbinlog_format = ROWbinlog_row_image = FULLbinlog_checksum = NONEexpire_logs_days = 7B. Create a DMS User
# pgsqlCREATE USER 'dms_user'@'%' IDENTIFIED BY 'password';GRANT REPLICATION SLAVE, REPLICATION CLIENT, SELECT, SHOW VIEW ON *.* TO 'dms_user'@'%';5. Schema Migration Process
Unlike PostgreSQL MySQL does NOT require AWS SCT for
compatibility, but it's useful for:
- Stored procedures
- Trigger compatibility
- Function differences
- Collation mismatches (utf8 vs utf8mb4)
- Partitioning differences
Recommended
Use:
mysqldump --no-data --routines --triggers > schema.sqlThen
import into RDS or Aurora.
6. DMS Task Best Practices (MySQL)
Full Load
- Enable parallel table loading
- Enable
BatchApplyfor faster inserts
Change Data Capture (CDC)
- Ensure binlog retention covers peak load time
- Avoid high-transaction tables without primary keys
- Turn off foreign keys on target during load:
SET FOREIGN_KEY_CHECKS = 0;LOB Columns
Choose appropriately:
- Limited LOB mode (better performance)
- Full LOB mode (slower but safer)
7. Aurora MySQL-Specific Considerations
A. Cluster Endpoints
- Cluster endpoint: read/write
- Reader endpoint: read scaling
- Custom endpoints: traffic distribution
B. Storage Benefits
- Auto-expand
- High write throughput
- Six-way replication across three AZs
C. Failover
- Aurora has near-instant recovery due to log-structured storage.
D. Migrating from RDS MySQL → Aurora MySQL
- Aurora MySQL supports drop-in migration
- Use built-in snapshot migration
- OR DMS if continuous replication needed
8. Cutover Strategies
Option A: Simple Migration
- Full load
- Scheduled downtime
- Stop app writes
- CDC catch-up
- Cutover
- Validate
Downtime: minutes
Option B: Near-Zero
Downtime Migration (Most Common)
- Full load
- Continuous CDC
- Perform live read-only cutover test
- Stop writes briefly
- Final catch-up
- Flip app writers
- Re-enable triggers & FKs
Downtime: <30 seconds
9. Validation and QA
Recommended Tools
- DMS validation
- Table-level checksum (pt-table-checksum)
- Row count comparison
- Application-level validation tests
No comments:
Post a Comment