An Overview of how to share AWS
Direct Connect (DX) across
multiple AWS accounts using AWS Transit
Gateway (TGW)
Scope:
- Networking flows,
- BGP models,
- DXGW attachment behavior,
- Multi-account architecture patterns,
- Quotas,
- Routing design,
- Common pitfalls.
Breakdown:
- How the sharing process Works (High-Level),
- Why DX Must Go Through DXGW (Not TGW Directly),
- Detailed Architecture,
- Routing Flow,
- How Multiple AWS Accounts Use the Same DX,
- Security & Isolation Controls,
- Throughput (DX + TGW),
- Advanced Multi-Account Design Patterns,
- Common Pitfalls (and Fixes),
- Summary.
Intro:
AWS Direct
Connect (DX) can be shared across many AWS accounts only
when combined with:
- Direct Connect Gateway (DXGW)
- Transit Gateway (TGW)
- Resource Access Manager (RAM)
NB:
- This model allows many AWS accounts to consume on-prem connectivity through a single DX circuit, while maintaining strict account boundaries.
How the sharing process Works (High-Level)
1. DX
circuit → DX Gateway (DXGW)
2. DXGW
is associated with TGW (one or more TGWs)
3. TGW
is shared via RAM with other AWS accounts
4. Workload
accounts create VPC attachments to that shared TGW
5. Traffic
flows:
NB:
This enables full multi-account, scalable, hub-and-spoke
connectivity.
Why DX Must Go Through DXGW (Not TGW Directly)
Direct Connect never attaches directly to a TGW.
Instead:
- TGW attachments are regional
- DXGW is a global construct
- DXGW supports:
- Multiple VGWs
- Multiple TGWs
- Multi-account routing boundaries
- Route filtering + CIDR control
DXGW = the global routing hub for DX.
Detailed Architecture
1. The Central Networking Account
This is where centralized networking teams host:
- Direct Connect circuits (1G, 10G, 100G)
- LAGs (Link Aggregation Groups)
- Transit VIFs
- DXGW
- Central TGW
This account becomes the network hub.
2. Linking DXGW to TGW
twtech creates a Transit Gateway Association:
DXGW ↔ TGW
One DXGW can associate with up to 3 TGWs (Useful for multi-region hub architectures)
3. Publishing TGW to other Accounts via RAM
The TGW is shared using AWS Resource Access Manager:
Networking Account → Shared TGW → Application Accounts
Each workload account creates its own VPC attachment to the
shared TGW.
This allows:
- Full isolation
- Separate teams/accounts
- Independent VPC CIDR space
- Shared connectivity through DX
Routing Flow
On-Prem → AWS Routing
1. On-prem
router advertises routes into DX using BGP.
2. DXGW
receives those routes.
3. DXGW
passes routes to TGW only if they are allowed by the route filter.
4. TGW
propagates routes to all VPC attachments.
AWS → On-Prem Routing
1. Each
VPC advertises its CIDR(s) to TGW.
2. TGW
can aggregate or propagate them upward.
3. TGW
advertises routes to DXGW (again, filtered).
4. DXGW
advertises filtered routes to on-prem through BGP.
How Multiple AWS Accounts Use the Same DX
Sample setup:
·
Networking account
- DX circuit
- DXGW
- TGW
·
Application Account A
- VPC (10.10.0.0/16)
- TGW VPC attachment
·
Application Account B
- VPC (10.20.0.0/16)
- TGW VPC attachment
Flows:
On-Prem ↔ DX ↔ DXGW ↔ TGW ↔ VPC A ↕ VPC B
NB:
Both accounts use the same physical DX, but only through their VPC attachments to the shared TGW.
Security & Isolation Controls
AWS ensures boundaries at three layers:
1. DXGW Route Filtering (CIDR-based)
twtech configures which VPC CIDRs are allowed to be advertised
to on-prem.
Example:
Allow: 10.10.0.0/16 10.20.0.0/16Deny: 172.31.0.0/16
2. TGW Route Tables
Each VPC attachment can be associated with a different route table.
This supports:
- Isolated account partitions
- Shared services partitions
- Regional segmentation
3. IAM / RAM Controls
Only selected AWS accounts can:
- Attach to TGW
- View routes
- Modify routing tables
Throughput (DX +
TGW)
Baseline numbers:
- Single Transit VIF: up to the speed of twtech DX line (1G/10G/100G)
- TGW supports ECMP across multiple Transit VIFs
Sample:
|
Number of
Transit VIFs |
Aggregate
Throughput |
|
1 × 10 Gbps |
10 Gbps |
|
2 × 10 Gbps (ECMP) |
20 Gbps |
|
4 × 10 Gbps |
40 Gbps |
|
2 × 100 Gbps |
200 Gbps |
Important:
TGW does flow-based hashing, so single TCP flows typically
max at:
- 5–10 Gbps per flow (depending on NIC/ENA Express)
- But aggregate flows = use all links
Advanced Multi-Account Design Patterns
Pattern 1 — Single Region, Many Accounts (Most Common)
DX ↔ DXGW ↔ TGW (central region) ↔ VPCs in many accounts
Best for enterprise shared services.
Pattern 2 — Multi-Region with Shared DX
Region ADX ↔ DXGW ↔ TGW-A ↔ VPCs (many accounts) ↓ (peering) TGW-B ↔ VPCs (many accounts) Region B
DXGW can associate with up to 3 TGWs.
Pattern 3 — Segmented Multi-Account Design
- “Prod TGW”
- “Non-prod TGW”
- “Shared services TGW”
All sharing the same DX circuit via DXGW.
Common Pitfalls (and
Fixes)
|
Problem |
Explanation |
Fix |
|
No traffic from account B through DX |
DXGW route filters not updated |
Add the new CIDR |
|
Overlapping VPC CIDRs |
TGW rejects overlapping prefixes |
Re-IP or use NAT segmentation |
|
Traffic asymmetric routing |
Mixed VPN + DX inbound |
Force route priorities (AS-PATH prepending) |
|
Single-flow bottlenecks |
TGW is flow-hashed |
Add parallel streams or ENA Express |
|
Application account can’t attach to TGW |
IAM + RAM permissions missing |
Update RAM share |
Design Summary
Using:
- DX Gateway
- TGW
- RAM
twtech can:
- Share 1 physical DX circuit with hundreds of AWS accounts
- Maintain strong security boundaries
- Scale DX bandwidth using ECMP
- Centralize twtech networking without slowing down application teams
No comments:
Post a Comment