A media company is designing a global streaming platform using AWS. They have a primary workload in us-east-1 (VPC A, 10.0.0.0/16) and a secondary workload in eu-west-1 (VPC B, 10.1.0.0/16). They need to replicate data between these VPCs with low latency and high throughput, and also allow their on-premises data center (10.2.0.0/16) in us-east-2 to communicate with both VPCs. The on-premises network is connected to AWS via two Direct Connect connections terminating in us-east-1 and eu-west-1. The company uses a Transit Gateway in each region, with inter-region peering between the Transit Gateways. The on-premises network has BGP advertisements for 10.2.0.0/16. The routing is set up such that the on-premises network can reach both VPCs via the Direct Connect connections. However, the VPCs cannot reach each other's CIDRs. The network engineer checks the Transit Gateway route tables and sees that the inter-region peering attachment is associated with the appropriate route tables, and static routes for the remote VPC CIDRs are present. What is the MOST likely cause of the problem?
This would cause traffic to be sent to on-premises, which cannot route between regions, causing blackhole.
Why this answer
In a Transit Gateway inter-region peering, you need to add static routes in both directions to point to the peering attachment. The engineer confirmed static routes are present, so that might not be the issue. However, a common misconfiguration is that the on-premises network's route propagation via Direct Connect may be causing the Transit Gateway to prefer the on-premises path (which may not have connectivity between regions) over the inter-region peering.
But the VPCs cannot reach each other's CIDRs, so the issue is likely that the route tables for the VPC attachments do not have the routes for the remote VPC CIDRs pointing to the inter-region peering attachment. Alternatively, the inter-region peering attachment might not be in the correct route table. The most likely issue is that the Transit Gateway route tables for the VPCs do not include routes for the remote VPC CIDRs via the peering attachment.
Since the engineer says they are present, the issue might be that the propagation from the Direct Connect attachment is overriding the static route with a less preferred route or causing a conflict. Actually, the most common problem is that the static routes for the inter-region CIDRs are pointing to the wrong attachment (e.g., the Direct Connect attachment instead of the peering attachment).