Cloud Network Interconnection Options for AWS, Azure, and GCP

As organizations adopt multi-cloud strategies and hybrid architectures, understanding network interconnection options becomes critical. This guide explores the various methods to connect networks within and across AWS, Azure, and GCP, helping you choose the right solution for your use case.

Why Network Interconnection Matters

Modern cloud architectures often require:

Hybrid Cloud Connectivity: Connecting on-premises data centers to cloud resources
Multi-Cloud Networking: Linking resources across different cloud providers
Regional Expansion: Connecting VPCs/VNets across regions
Disaster Recovery: Establishing redundant network paths
Compliance: Meeting data residency and security requirements

AWS Network Interconnection Options

1. VPC Peering

VPC Peering allows you to connect two VPCs using private IP addresses, as if they were on the same network.

Use Cases:

Connecting VPCs within the same region or across regions
Sharing resources between development and production environments
Simple hub-and-spoke architectures

Configuration Example:

# Terraform example for VPC Peering
resource "aws_vpc_peering_connection" "main_to_secondary" {
  peer_vpc_id   = aws_vpc.secondary.id
  vpc_id        = aws_vpc.main.id
  auto_accept   = true

  tags = {
    Name = "Main to Secondary VPC Peering"
  }
}

# Route table update
resource "aws_route" "main_to_secondary" {
  route_table_id            = aws_route_table.main.id
  destination_cidr_block    = aws_vpc.secondary.cidr_block
  vpc_peering_connection_id = aws_vpc_peering_connection.main_to_secondary.id
}

resource "aws_route" "secondary_to_main" {
  route_table_id            = aws_route_table.secondary.id
  destination_cidr_block    = aws_vpc.main.cidr_block
  vpc_peering_connection_id = aws_vpc_peering_connection.main_to_secondary.id
}
hcl

Limitations:

No transitive peering (A → B → C doesn't allow A → C)
Maximum 125 peering connections per VPC
CIDR blocks must not overlap
Regional peering requires separate connections

2. AWS Transit Gateway

Transit Gateway acts as a central hub for connecting VPCs, VPNs, and Direct Connect connections, solving the transitive routing problem.

Use Cases:

Hub-and-spoke architectures with many VPCs
Centralized network management
Multi-region connectivity
Hybrid cloud connectivity

Architecture Example:

# Transit Gateway
resource "aws_ec2_transit_gateway" "main" {
  description                     = "Main Transit Gateway"
  default_route_table_association = "enable"
  default_route_table_propagation = "enable"
  
  tags = {
    Name = "Main TGW"
  }
}

# VPC Attachment
resource "aws_ec2_transit_gateway_vpc_attachment" "vpc1" {
  subnet_ids         = aws_subnet.vpc1[*].id
  transit_gateway_id = aws_ec2_transit_gateway.main.id
  vpc_id             = aws_vpc.vpc1.id
}

resource "aws_ec2_transit_gateway_vpc_attachment" "vpc2" {
  subnet_ids         = aws_subnet.vpc2[*].id
  transit_gateway_id = aws_ec2_transit_gateway.main.id
  vpc_id             = aws_vpc.vpc2.id
}

# Route propagation
resource "aws_ec2_transit_gateway_route_table_propagation" "vpc1" {
  transit_gateway_attachment_id  = aws_ec2_transit_gateway_vpc_attachment.vpc1.id
  transit_gateway_route_table_id = aws_ec2_transit_gateway.main.association_default_route_table_id
}
hcl

Benefits:

Transitive routing between all attached networks
Centralized management and monitoring
Support for route tables and route filtering
Multi-region peering support
Integration with Direct Connect and VPN

3. AWS Direct Connect

Direct Connect provides dedicated network connections from on-premises to AWS, bypassing the public internet.

Use Cases:

High-bandwidth requirements (1 Gbps to 100 Gbps)
Consistent network performance
Compliance requirements for private connectivity
Cost reduction for high data transfer volumes

Architecture:

On-Premises Data Center
  │
  ├─ Direct Connect Location (Colocation Facility)
  │   ├─ Dedicated Connection (1/10/100 Gbps)
  │   └─ Hosted Connection (50 Mbps - 10 Gbps)
  │
  └─ AWS Direct Connect Gateway
      ├─ VPC 1 (us-east-1)
      ├─ VPC 2 (us-west-2)
      └─ VPC 3 (eu-west-1)

Configuration:

# Direct Connect Gateway
resource "aws_dx_gateway" "main" {
  name            = "main-dx-gateway"
  amazon_side_asn = "64512"
}

# Private Virtual Interface
resource "aws_dx_private_virtual_interface" "main" {
  connection_id   = aws_dx_connection.main.id
  dx_gateway_id   = aws_dx_gateway.main.id
  name            = "main-private-vif"
  vlan            = 100
  address_family  = "ipv4"
  bgp_asn         = 65000
}

# VPC Association
resource "aws_dx_gateway_association" "main" {
  dx_gateway_id         = aws_dx_gateway.main.id
  associated_gateway_id = aws_vpn_gateway.main.id
}
hcl

4. AWS Site-to-Site VPN

Site-to-Site VPN creates encrypted tunnels over the internet to connect on-premises networks to AWS VPCs.

Use Cases:

Quick setup for hybrid connectivity
Backup connectivity for Direct Connect
Low to medium bandwidth requirements
Cost-effective solution for smaller deployments

Configuration:

# Customer Gateway (on-premises device)
resource "aws_customer_gateway" "on_prem" {
  bgp_asn    = 65000
  ip_address = "203.0.113.12"  # On-premises public IP
  type       = "ipsec.1"

  tags = {
    Name = "On-Premises Gateway"
  }
}

# Virtual Private Gateway
resource "aws_vpn_gateway" "main" {
  vpc_id = aws_vpc.main.id

  tags = {
    Name = "Main VPN Gateway"
  }
}

# VPN Connection
resource "aws_vpn_connection" "main" {
  vpn_gateway_id      = aws_vpn_gateway.main.id
  customer_gateway_id = aws_customer_gateway.on_prem.id
  type                = "ipsec.1"
  static_routes_only  = false

  tags = {
    Name = "Main VPN Connection"
  }
}
hcl

Azure Network Interconnection Options

1. VNet Peering

VNet Peering connects two Azure Virtual Networks, enabling resources to communicate using private IP addresses.

Use Cases:

Connecting VNets within the same region (regional peering)
Connecting VNets across regions (global peering)
Hub-and-spoke architectures
Cross-subscription connectivity

Configuration Example:

# Regional VNet Peering
resource "azurerm_virtual_network_peering" "hub_to_spoke" {
  name                      = "hub-to-spoke"
  resource_group_name       = azurerm_resource_group.main.name
  virtual_network_name      = azurerm_virtual_network.hub.name
  remote_virtual_network_id = azurerm_virtual_network.spoke.id
  allow_forwarded_traffic   = true
  allow_gateway_transit     = true
  use_remote_gateways       = false
}

resource "azurerm_virtual_network_peering" "spoke_to_hub" {
  name                      = "spoke-to-hub"
  resource_group_name       = azurerm_resource_group.main.name
  virtual_network_name      = azurerm_virtual_network.spoke.name
  remote_virtual_network_id = azurerm_virtual_network.hub.id
  allow_forwarded_traffic   = true
  allow_gateway_transit     = false
  use_remote_gateways       = true
}
hcl

Key Features:

Global peering across Azure regions
Gateway transit support
Non-transitive by default (can enable with User-Defined Routes)
No bandwidth limitations (limited by VM SKU)

2. Azure Virtual WAN

Virtual WAN is a networking service that provides optimized and automated branch-to-branch connectivity through Azure.

Use Cases:

Large-scale branch connectivity
Global connectivity with multiple regions
Integration with Azure services (Firewall, ExpressRoute, VPN)
Simplified network management

Architecture:

# Virtual WAN
resource "azurerm_virtual_wan" "main" {
  name                = "main-vwan"
  resource_group_name = azurerm_resource_group.main.name
  location            = azurerm_resource_group.main.location
}

# Virtual Hub
resource "azurerm_virtual_hub" "main" {
  name                = "main-hub"
  resource_group_name = azurerm_resource_group.main.name
  location            = azurerm_resource_group.main.location
  virtual_wan_id      = azurerm_virtual_wan.main.id
  address_prefix      = "10.0.1.0/24"
}

# Hub Virtual Network Connection
resource "azurerm_virtual_hub_connection" "vnet1" {
  name                      = "vnet1-connection"
  virtual_hub_id            = azurerm_virtual_hub.main.id
  remote_virtual_network_id = azurerm_virtual_network.vnet1.id
}
hcl

Benefits:

Automated branch connectivity
Built-in security with Azure Firewall
Global reach with Azure backbone
Integration with ExpressRoute and VPN
Centralized routing and management

3. Azure ExpressRoute

ExpressRoute provides private connectivity to Azure services through a connectivity provider.

Use Cases:

High-bandwidth requirements (50 Mbps to 100 Gbps)
Private connectivity for compliance
Consistent network performance
Hybrid cloud scenarios

Connection Types:

CloudExchange Co-location: Connect at a colocation facility
Point-to-Point Ethernet: Direct fiber connection
Any-to-Any (IPVPN): Connect through an MPLS VPN provider

Configuration:

# ExpressRoute Circuit
resource "azurerm_express_route_circuit" "main" {
  name                  = "main-expressroute"
  resource_group_name    = azurerm_resource_group.main.name
  location              = azurerm_resource_group.main.location
  service_provider_name = "Equinix"
  peering_location      = "Washington DC"
  bandwidth_in_mbps     = 1000

  sku {
    tier   = "Standard"
    family = "MeteredData"
  }
}

# ExpressRoute Gateway
resource "azurerm_express_route_gateway" "main" {
  name                = "main-ergw"
  resource_group_name = azurerm_resource_group.main.name
  location            = azurerm_resource_group.main.location
  virtual_hub_id      = azurerm_virtual_hub.main.id
  scale_units         = 1
}

# ExpressRoute Connection
resource "azurerm_express_route_connection" "main" {
  name                      = "main-connection"
  express_route_gateway_id  = azurerm_express_route_gateway.main.id
  express_route_circuit_id  = azurerm_express_route_circuit.main.id
  express_route_circuit_peering_id = azurerm_express_route_circuit_peering.main.id
}
hcl

4. Azure VPN Gateway

VPN Gateway provides secure, cross-premises connectivity between Azure VNets and on-premises locations.

Use Cases:

Site-to-Site VPN connections
Point-to-Site VPN for remote users
VNet-to-VNet connections
Multi-site connectivity

Configuration:

# VPN Gateway
resource "azurerm_virtual_network_gateway" "main" {
  name                = "main-vpngw"
  location            = azurerm_resource_group.main.location
  resource_group_name = azurerm_resource_group.main.name

  type     = "Vpn"
  vpn_type = "RouteBased"

  active_active = false
  enable_bgp    = true
  sku           = "VpnGw1"

  ip_configuration {
    public_ip_address_id          = azurerm_public_ip.gateway.id
    private_ip_address_allocation = "Dynamic"
    subnet_id                     = azurerm_subnet.gateway.id
  }
}

# Local Network Gateway (on-premises)
resource "azurerm_local_network_gateway" "on_prem" {
  name                = "on-prem-gateway"
  location            = azurerm_resource_group.main.location
  resource_group_name = azurerm_resource_group.main.name
  gateway_address     = "203.0.113.12"
  address_space       = ["10.1.0.0/16"]
}

# Connection
resource "azurerm_virtual_network_gateway_connection" "on_prem" {
  name                = "on-prem-connection"
  location            = azurerm_resource_group.main.location
  resource_group_name = azurerm_resource_group.main.name

  type                       = "IPsec"
  virtual_network_gateway_id = azurerm_virtual_network_gateway.main.id
  local_network_gateway_id   = azurerm_local_network_gateway.on_prem.id

  shared_key = var.vpn_shared_key
}
hcl

GCP Network Interconnection Options

1. VPC Network Peering

VPC Network Peering connects two VPC networks so resources can communicate using internal IP addresses.

Use Cases:

Connecting VPCs within the same project
Cross-project connectivity
Regional and global peering
Shared VPC architectures

Configuration Example:

# VPC Network Peering
resource "google_compute_network_peering" "peering1" {
  name         = "peering-1"
  network      = google_compute_network.vpc1.id
  peer_network = google_compute_network.vpc2.id

  import_custom_routes = true
  export_custom_routes = true
}

resource "google_compute_network_peering" "peering2" {
  name         = "peering-2"
  network      = google_compute_network.vpc2.id
  peer_network = google_compute_network.vpc1.id

  import_custom_routes = true
  export_custom_routes = true
}
hcl

Key Features:

Global peering across regions
Support for custom routes
No bandwidth limitations
Transitive peering not supported (requires Network Connectivity Center)

2. Cloud VPN

Cloud VPN provides secure IPsec VPN connections between on-premises networks and GCP VPCs, or between GCP VPCs.

Use Cases:

Site-to-Site VPN
VPC-to-VPC connectivity
Hybrid cloud connectivity
Backup connectivity for Cloud Interconnect

Configuration:

# VPN Gateway
resource "google_compute_vpn_gateway" "main" {
  name    = "main-vpn-gateway"
  network = google_compute_network.main.id
  region  = "us-central1"
}

# External IP for VPN Gateway
resource "google_compute_address" "vpn_gateway" {
  name   = "vpn-gateway-ip"
  region = "us-central1"
}

# Forwarding Rule
resource "google_compute_forwarding_rule" "vpn_esp" {
  name        = "vpn-esp"
  ip_protocol = "ESP"
  ip_address  = google_compute_address.vpn_gateway.address
  target      = google_compute_vpn_gateway.main.id
}

resource "google_compute_forwarding_rule" "vpn_udp500" {
  name        = "vpn-udp500"
  ip_protocol = "UDP"
  port_range  = "500"
  ip_address  = google_compute_address.vpn_gateway.address
  target      = google_compute_vpn_gateway.main.id
}

resource "google_compute_forwarding_rule" "vpn_udp4500" {
  name        = "vpn-udp4500"
  ip_protocol = "UDP"
  port_range  = "4500"
  ip_address  = google_compute_address.vpn_gateway.address
  target      = google_compute_vpn_gateway.main.id
}

# VPN Tunnel
resource "google_compute_vpn_tunnel" "tunnel1" {
  name          = "tunnel1"
  peer_ip       = "203.0.113.12"  # On-premises gateway IP
  shared_secret = var.vpn_shared_secret

  target_vpn_gateway = google_compute_vpn_gateway.main.id

  local_traffic_selector  = ["0.0.0.0/0"]
  remote_traffic_selector = ["0.0.0.0/0"]
}

# Route
resource "google_compute_route" "vpn_route" {
  name       = "vpn-route"
  network    = google_compute_network.main.name
  dest_range = "10.1.0.0/16"  # On-premises network
  priority   = 1000

  next_hop_vpn_tunnel = google_compute_vpn_tunnel.tunnel1.id
}
hcl

3. Cloud Interconnect

Cloud Interconnect provides dedicated, high-bandwidth connections between on-premises networks and Google's network.

Types:

Dedicated Interconnect: Direct physical connection (10 Gbps or 100 Gbps)
Partner Interconnect: Connection through a service provider (50 Mbps to 50 Gbps)

Use Cases:

High-bandwidth requirements
Consistent network performance
Compliance requirements
Large-scale hybrid deployments

Configuration:

# Interconnect Attachment
resource "google_compute_interconnect_attachment" "main" {
  name                     = "main-interconnect"
  router                   = google_compute_router.main.id
  region                   = "us-central1"
  type                     = "PARTNER"
  edge_availability_domain = "AVAILABILITY_DOMAIN_1"
  admin_enabled            = true
}

# Router
resource "google_compute_router" "main" {
  name    = "main-router"
  region  = "us-central1"
  network = google_compute_network.main.id
  bgp {
    asn = 64514
  }
}

# Router Interface
resource "google_compute_router_interface" "main" {
  name       = "main-interface"
  router     = google_compute_router.main.name
  region     = "us-central1"
  ip_range   = "169.254.0.1/30"
  interconnect_attachment = google_compute_interconnect_attachment.main.name
}

# BGP Peer
resource "google_compute_router_peer" "main" {
  name                      = "main-peer"
  router                    = google_compute_router.main.name
  region                    = "us-central1"
  peer_ip_address           = "169.254.0.2"
  peer_asn                  = 65000
  advertised_route_priority = 100
  interface                 = google_compute_router_interface.main.name
}
hcl

4. Network Connectivity Center

Network Connectivity Center (NCC) provides a hub-and-spoke model for connecting VPC networks with transitive routing.

Use Cases:

Large-scale multi-VPC connectivity
Transitive routing requirements
Centralized network management
Hybrid and multi-cloud connectivity

Architecture:

# Hub
resource "google_networkconnectivity_hub" "main" {
  name        = "main-hub"
  description = "Main Network Connectivity Center Hub"
}

# Spoke (VPC Attachment)
resource "google_networkconnectivity_spoke" "vpc1" {
  name     = "vpc1-spoke"
  location = "global"
  hub      = google_networkconnectivity_hub.main.id

  linked_vpc_network {
    uri = google_compute_network.vpc1.id
  }
}

resource "google_networkconnectivity_spoke" "vpc2" {
  name     = "vpc2-spoke"
  location = "global"
  hub      = google_networkconnectivity_hub.main.id

  linked_vpc_network {
    uri = google_compute_network.vpc2.id
  }
}
hcl

Cross-Cloud Interconnection

Third-Party Solutions

For connecting across different cloud providers, consider:

Megaport: Software-defined networking platform connecting AWS, Azure, GCP, and on-premises
Equinix Cloud Exchange: Direct connections to multiple cloud providers
Aviatrix: Multi-cloud networking platform with centralized management
Alkira: Cloud-native network infrastructure for multi-cloud

VPN-Based Cross-Cloud

You can establish VPN connections between cloud providers:

AWS to Azure:

Use AWS VPN Gateway and Azure VPN Gateway
Connect via public internet with IPsec tunnels
Suitable for low to medium bandwidth

AWS to GCP:

Use AWS VPN Gateway and GCP Cloud VPN
Establish IPsec tunnels between gateways
Requires public IP addresses on both sides

Azure to GCP:

Use Azure VPN Gateway and GCP Cloud VPN
Similar IPsec tunnel configuration
Consider latency and bandwidth requirements

Comparison Matrix

Feature	AWS	Azure	GCP
VPC/VNet Peering	Regional & Cross-Region	Regional & Global	Global
Transitive Routing	Transit Gateway	Virtual WAN / UDR	Network Connectivity Center
Dedicated Connection	Direct Connect	ExpressRoute	Cloud Interconnect
VPN Gateway	Site-to-Site VPN	VPN Gateway	Cloud VPN
Bandwidth Limits	Varies by service	Varies by SKU	Varies by service
Global Reach	Multi-region	Global	Global
Cost Model	Per connection/hour	Per gateway/hour	Per tunnel/hour

Best Practices

1. Design for High Availability

Multiple Connections: Use redundant connections for critical workloads
Multi-Region: Distribute connections across regions
Active-Active: Configure active-active VPN tunnels when supported
Monitoring: Implement comprehensive monitoring and alerting

2. Security Considerations

Encryption: Always use encryption in transit (IPsec, TLS)
Network Segmentation: Use separate connections for different security zones
Firewall Rules: Implement network security groups/ACLs
Private Connectivity: Prefer dedicated connections for sensitive data

3. Cost Optimization

Right-Size Connections: Choose appropriate bandwidth for your needs
Data Transfer: Minimize cross-region data transfer
Reserved Capacity: Consider committed use discounts where available
Monitoring: Track data transfer costs and optimize routes

4. Performance Optimization

Latency: Choose regions close to your users/workloads
Bandwidth: Provision adequate bandwidth for peak loads
Routing: Optimize routing tables and BGP configurations
Caching: Use CDN and caching to reduce origin traffic

5. Operational Excellence

Automation: Use Infrastructure as Code (Terraform, CloudFormation, ARM)
Documentation: Maintain network topology diagrams
Change Management: Implement proper change control processes
Disaster Recovery: Test failover scenarios regularly

Use Case Recommendations

Small to Medium Hybrid Cloud

AWS: Site-to-Site VPN
Azure: VPN Gateway
GCP: Cloud VPN

Large-Scale Hybrid Cloud

AWS: Direct Connect + Transit Gateway
Azure: ExpressRoute + Virtual WAN
GCP: Cloud Interconnect + Network Connectivity Center

Multi-Cloud Architecture

Option 1: Third-party solution (Megaport, Aviatrix)
Option 2: VPN connections between cloud providers
Option 3: On-premises hub connecting to all clouds

Regional Expansion

AWS: Transit Gateway with multi-region peering
Azure: Virtual WAN with global reach
GCP: Global VPC peering or Network Connectivity Center

Conclusion

Choosing the right network interconnection option depends on your specific requirements:

Bandwidth: High bandwidth needs favor dedicated connections (Direct Connect, ExpressRoute, Cloud Interconnect)
Complexity: Simple architectures can use peering, complex ones benefit from hub services (Transit Gateway, Virtual WAN, NCC)
Cost: VPN solutions are cost-effective for low bandwidth, dedicated connections for high bandwidth
Compliance: Dedicated connections provide better security and compliance posture
Multi-Cloud: Consider third-party solutions or VPN-based connections

Key takeaways:

AWS Transit Gateway excels at hub-and-spoke architectures with many VPCs
Azure Virtual WAN provides comprehensive branch and cloud connectivity
GCP Network Connectivity Center offers transitive routing for complex topologies
Dedicated connections (Direct Connect, ExpressRoute, Cloud Interconnect) provide the best performance and security
VPN solutions are cost-effective for smaller deployments or backup connectivity
Cross-cloud connectivity often requires third-party solutions or VPN connections

Start with your specific requirements—bandwidth, latency, security, and cost—then choose the appropriate interconnection method. For production workloads, always design for redundancy and high availability.