How AWS Transit Gateway (TGW) Shares Direct Connect (DX) Across Multiple AWS Accounts - Overview.
Scope:
- Intro.
- How the sharing process Works (High-Level),
- Why DX Must Go Through DXGW (Not TGW Directly),
- Architecture,
- Routing Flow,
- How Multiple AWS Accounts Use the Same DX,
- Security & Isolation Controls,
- Throughput (DX + TGW),
- Advanced Multi-Account Design Patterns,
- Common Pitfalls (and Fixes),
- Summary.
Intro:
AWS Direct
Connect (DX) can be shared across many AWS accounts only
when combined with:
- Direct Connect Gateway (DXGW)
- Transit Gateway (TGW)
- Resource Access Manager (RAM)
NB:
- This model allows many AWS accounts to consume on-prem connectivity through a single DX circuit, while maintaining strict account boundaries.
How the sharing process Works (High-Level)
1. DX circuit → DX Gateway (DXGW)
2. DXGW is associated with TGW (one or more TGWs)
3. TGW is shared via RAM with other AWS accounts
4. Workload accounts create VPC attachments to that shared TGW
5. Traffic
flows:
NB:
- This enables full multi-account, scalable, hub-and-spoke connectivity.
Why DX Must Go Through DXGW (Not TGW Directly)
- Direct Connect never attaches directly to a TGW.
Instead:
- TGW attachments are regional
- DXGW is a global construct
- DXGW supports:
- Multiple VGWs
- Multiple TGWs
- Multi-account routing boundaries
- Route filtering + CIDR control
- DXGW = the global routing hub for DX.
Architecture
1. The Central Networking Account
- This is where centralized networking teams host:
- Direct Connect circuits (1G, 10G, 100G)
- LAGs (Link Aggregation Groups)
- Transit VIFs
- DXGW
- Central TGW
- This account becomes the network hub.
2. Linking DXGW to TGW
- twtech creates a Transit Gateway Association:
DXGW↔ TGW
NB:
- One DXGW can associate with up to 3 TGWs (Useful for multi-region hub architectures)
3. Publishing TGW to other Accounts via RAM
- The TGW is shared using AWS Resource Access Manager:
Networking Account → Shared TGW → Application Accounts
- Each workload account creates its own VPC attachment to the shared TGW.
This allows:
- Full isolation
- Separate teams/accounts
- Independent VPC CIDR space
- Shared connectivity through DX
Routing Flow
On-Prem → AWS Routing
1. On-prem router advertises routes into DX using BGP.
2. DXGW receives those routes.
3. DXGW passes routes to TGW only if they are allowed by the route filter.
4. TGW propagates routes to all VPC attachments.
AWS → On-Prem Routing
1. Each VPC advertises its CIDR(s) to TGW.
2. TGW can aggregate or propagate them upward.
3. TGW advertises routes to DXGW (again, filtered).
4. DXGW advertises filtered routes to on-prem through BGP.
How Multiple AWS Accounts Use the Same DX
Sample setup:
- Networking account
- DX circuit
- DXGW
- TGW
- Application Account A
- VPC (10.10.0.0/16)
- TGW VPC attachment
- Application Account B
- VPC (10.20.0.0/16)
- TGW VPC attachment
Flows:
On-Prem ↔ DX ↔ DXGW ↔ TGW ↔ VPC A ↕ VPC B
NB:
- Both accounts use the same physical DX.
- but only through their VPC attachments to the shared TGW.
Security & Isolation Controls
- AWS ensures boundaries at three layers:
1. DXGW Route Filtering (CIDR-based)
- twtech configures which VPC CIDRs are allowed to be advertised to on-prem.
Sample:
Allow:10.10.0.0/1610.20.0.0/16
Deny:172.31.0.0/16
2. TGW Route Tables
- Each VPC attachment can be associated with a different route table.
This supports:
- Isolated account partitions
- Shared services partitions
- Regional segmentation
3. IAM / RAM Controls
Only selected AWS accounts can:
- Attach to TGW
- View routes
- Modify routing tables
Throughput (DX +
TGW)
Baseline numbers:
- Single Transit VIF: up to the speed of twtech DX line (1G/10G/100G)
- TGW supports ECMP across multiple Transit VIFs
Sample:
|
|
|
|
|
|
|
|
|
|
Important:
- TGW does flow-based hashing, so single TCP flows typically max at:
- 5–10 Gbps per flow (depending on NIC/ENA Express)
- But aggregate flows = use all links
Advanced Multi-Account Design Patterns
Pattern 1 — Single Region, Many Accounts (Most Common)
DX ↔ DXGW ↔ TGW (central region) ↔ VPCs in many accounts
- Best for enterprise shared services.
Pattern 2 — Multi-Region with Shared DX
Region ADX ↔ DXGW ↔ TGW-A ↔ VPCs (many accounts) ↓ (peering) TGW-B ↔ VPCs (many accounts) Region B
DXGW can associate with up to 3 TGWs.
Pattern 3 — Segmented Multi-Account Design
- “Prod TGW”
- “Non-prod TGW”
- “Shared services TGW”
- All sharing the same DX circuit via DXGW.
Common Pitfalls (and
Fixes)
|
Problem |
Explanation |
Fix |
|
No traffic from account B through DX |
DXGW route filters not updated |
Add the new CIDR |
|
Overlapping VPC CIDRs |
TGW rejects overlapping prefixes |
Re-IP or use NAT segmentation |
|
Traffic asymmetric routing |
Mixed VPN + DX inbound |
Force route priorities (AS-PATH prepending) |
|
Single-flow bottlenecks |
TGW is flow-hashed |
Add parallel streams or ENA Express |
|
Application account can’t attach to TGW |
IAM + RAM permissions missing |
Update RAM share |
Design Summary
Using:
- DX Gateway
- TGW
- RAM
Recommendation:
- Share 1 physical DX circuit with hundreds of AWS accounts
- Maintain strong security boundaries
- Scale DX bandwidth using ECMP
- Centralize twtech networking without slowing down application teams
No comments:
Post a Comment