Active/Active Azure VPN Gateways- IKEv2 VPN to CSR

Lab Objectives

This lab guide shows how to configure active/active Azure VPN gateways with IKEv2 VPNs to an "on prem" CSR. The Azure VPN GW utilizes BGP over IKEv2 tunnels to a CSR located in a VNET that simulates an on prem environment. The test VM subnet on the onprem side will have UDRs pointed to the CSR1 inside interface. BGP max paths allows ECMP load sharing as well as HA.

The main goal of this lab is to quickly stand up a sandbox environment for functionality testing of active/active tunnels to the Azure VPN GW. The test VMs will be able to ping each other and relevant interfaces. BGP prefix filters could be used to lock down route advertisement if required. The routing configuration is only an example and could be solved many ways. The entire environment is built on Azure and does not require any hardware. 

Requirements:

• A valid Azure subscription account. If you don’t have one, you can create your free azure account (https://azure.microsoft.com/en-us/free/) today.
• If you are using Windows 10, you can install Bash shell on Ubuntu on Windows (http://www.windowscentral.com/how-install-bash-shell-command-line-windows-10).
• Azure CLI 2.0, follow these instructions to install: https://docs.microsoft.com/en-us/cli/azure/install-azure-cli
• Basic knowledge of Azure networking.

Notes:

This is for functionality testing purposes only and should not be considered production configurations. There are a number of configuration options (security policies/NSG/timers/CLI etc) and designs you can use, this is just an example to use as a baseline. Azure CLI is used to show the building blocks and order of operations to make the environment work. All CLI is provided so you can fit to your environment. Azure Cloud Shell is an option if you cannot install Azure CLI on your machine. A loopback address is added to the CSR for troubleshooting and validation purposes only. The lab uses CSR IOS-XE 16.10, syntax could very based on code levels. You may need to accept the legal agreement for the CSR BYOL demo image. Below is a Powershell example that you can run in Cloud Shell (in portal) to accept the agreement:

Get-AzureRmMarketplaceTerms -Publisher "Cisco" -Product "cisco-csr-1000v" -Name "16_10-byol"

Get-AzureRmMarketplaceTerms -Publisher "Cisco" -Product "cisco-csr-1000v" -Name "16_10-byol" | Set-AzureRmMarketplaceTerms -Accept

BGP Layout:

Step 1: Login via Azure CLI. Open a command prompt and enter “az login”. It will prompt you for Azure credentials. All commands moving forward are done through Azure CLI on Windows 10 and Cisco CLI via SSH.

Step 2: Create resource group, VNET + address space and subnets for Hub VNET in East US. Note the GW Subnet:

az group create --name Hub --location "EastUS"
az network vnet create --name Hub --resource-group Hub --address-prefix 10.0.0.0/16
az network vnet subnet create --address-prefix 10.0.0.0/24 --name GatewaySubnet --resource-group Hub --vnet-name Hub
az network vnet subnet create --address-prefix 10.0.10.0/24 --name testVMSubnet --resource-group Hub --vnet-name Hub

Step 3: Create 2 Public IPs for the A/A Azure VPN GWs

az network public-ip create --name Azure-VNGpubip --resource-group Hub --allocation-method Dynamic
az network public-ip create --name Azure-VNGpubip2 --resource-group Hub --allocation-method Dynamic
az network vnet-gateway create --name Azure-VNG --public-ip-address Azure-VNGpubip Azure-VNGpubip2 --resource-group Hub --vnet Hub --gateway-type Vpn --vpn-type RouteBased --sku VpnGw1 --no-wait --asn 65001

Step 4: Create resource group, VNET + address space and subnets for onprem VNET in East US2:

az group create --name onprem --location "East US2"
az network vnet create --name onprem --resource-group onprem --address-prefix 10.1.0.0/16
az network vnet subnet create --address-prefix 10.1.1.0/24 --name InsideSubnet --resource-group onprem --vnet-name onprem
az network vnet subnet create --address-prefix 10.1.0.0/24 --name OutsideSubnet --resource-group onprem --vnet-name onprem
az network vnet subnet create --address-prefix 10.1.2.0/24 --name OutsideSubnet2 --resource-group onprem --vnet-name onprem
az network vnet subnet create --address-prefix 10.1.10.0/24 --name testVMSubnet --resource-group onprem --vnet-name onprem

Step 5: Create NSG and rules for the CSR1 interfaces. It allows SSH, UDP 500/4500, 10/8 and all outbound. You can fine tune the NSG to your liking:

az network nsg create --resource-group onprem --name onprem-CSR-NSG --location "East US2"

az network nsg rule create --resource-group onprem --nsg-name onprem-CSR-NSG --name CSR-IPSEC1 --access Allow --protocol Udp --direction Inbound --priority 100 --source-address-prefix "*" --source-port-range "*" --destination-address-prefix "*" --destination-port-range 500

az network nsg rule create --resource-group onprem --nsg-name onprem-CSR-NSG --name CSR-IPSEC2 --access Allow --protocol Udp --direction Inbound --priority 110 --source-address-prefix "*" --source-port-range "*" --destination-address-prefix "*" --destination-port-range 4500

az network nsg rule create --resource-group onprem --nsg-name onprem-CSR-NSG --name Allow-SSH-All --access Allow --protocol Tcp --direction Inbound --priority 120 --source-address-prefix Internet --source-port-range "*" --destination-address-prefix "*" --destination-port-range 22

az network nsg rule create --resource-group onprem --nsg-name onprem-CSR-NSG --name Allow-Tens --access Allow --protocol "*" --direction Inbound --priority 130 --source-address-prefix 10.0.0.0/8 --source-port-range "*" --destination-address-prefix "*" --destination-port-range "*"

az network nsg rule create --resource-group onprem --nsg-name onprem-CSR-NSG --name Allow-Out --access Allow --protocol "*" --direction Outbound --priority 140 --source-address-prefix "*" --source-port-range "*" --destination-address-prefix "*" --destination-port-range "*"

Step 6: Create 2x Public IP, 3x NICs (outside/outside2/inside), assign static private IPs, apply NSG.

az network public-ip create --name CSR1PublicIP --resource-group onprem --idle-timeout 30 --allocation-method Static
az network public-ip create --name CSR1PublicIP2 --resource-group onprem --idle-timeout 30 --allocation-method Static
az network nic create --name CSR1OutsideInterface -g onprem --subnet OutsideSubnet --vnet onprem --public-ip-address CSR1PublicIP --ip-forwarding true --network-security-group onprem-CSR-NSG --private-ip-address 10.1.0.4
az network nic create --name CSR1OutsideInterface2 -g onprem --subnet OutsideSubnet2 --vnet onprem --public-ip-address CSR1PublicIP2 --ip-forwarding true --network-security-group onprem-CSR-NSG --private-ip-address 10.1.2.4
az network nic create --name CSR1InsideInterface -g onprem --subnet InsideSubnet --vnet onprem --ip-forwarding true --network-security-group onprem-CSR-NSG --private-ip-address 10.1.1.4 

Step 7: Create CSR1 VM and specify CSR image 16.10. Tie in the previously created NICs, and SSH credentials:

az vm create --resource-group onprem --location eastus2 --name CSR1 --size Standard_DS3_v2 --nics CSR1OutsideInterface CSR1OutsideInterface2 CSR1InsideInterface  --image cisco:cisco-csr-1000v:16_10-byol:16.10.120190108 --admin-username azureuser --admin-password Msft123Msft123

Step 8: It’s highly recommended that you run the following commands to gather the public IP addresses. If the VNGs come back as "Null", provisioning is incomplete. Wait until you get a valid public IP address. Copy the output into notepad or editor to reference later:

az network public-ip show -g Hub -n Azure-VNGpubip --query "{address: ipAddress}"
az network public-ip show -g Hub -n Azure-VNGpubip2 --query "{address: ipAddress}"
az network public-ip show -g onprem -n CSR1PublicIP --query "{address: ipAddress}"
az network public-ip show -g onprem -n CSR1PublicIP2 --query "{address: ipAddress}"

Step 9: Create Local Network Gateway and VPN connection from the Azure VPN GW to CSR1PublicIP. Note: Azure side ASN is 65001, on prem ASN is 65002, CSR1 tunnel1 interface 192.168.1.1/32:

az network local-gateway create --gateway-ip-address "insert CSR1PublicIP" --name to-onprem --resource-group Hub --local-address-prefixes 192.168.1.1/32 --asn 65002 --bgp-peering-address 192.168.1.1
az network vpn-connection create --name to-onprem --resource-group Hub --vnet-gateway1 Azure-VNG -l eastus --shared-key Msft123Msft123 --local-gateway2 to-onprem --enable-bgp

Step 10: Create Local Network Gateway and VPN connection from the Azure VPN GW to CSR1PublicIP2. Note: Azure side is ASN 65001, on prem ASN is 65002, CSR1 tunnel2 interface 192.168.1.2/32:

az network local-gateway create --gateway-ip-address "insert CSR1PublicIP2" --name to-onprem2 --resource-group Hub --local-address-prefixes 192.168.1.2/32 --asn 65002 --bgp-peering-address 192.168.1.2
az network vpn-connection create --name to-onprem2 --resource-group Hub --vnet-gateway1 Azure-VNG -l eastus --shared-key Msft123Msft123 --local-gateway2 to-onprem2 --enable-bgp

Step 11: Validate BGP information on Azure VPN GW. This will display local BGP ASN and local BGP peer addresses to use over the tunnels:

az network vnet-gateway list --query [].[name,bgpSettings.asn,bgpSettings.bgpPeeringAddress] -o table --resource-group Hub

Step 12: SSH to CSR1PublicIP. Username=azureuser pw=Msft123Msft123

Paste in the following commands AFTER replacing all references to “Azure-VNGpubip” and “Azure-VNGpubip2” with the public IP addressed of VNGs documented earlier:

int gi1
no ip nat outside
int gi3
ip address 10.1.1.4 255.255.255.0
no shut
int lo0
ip address 1.1.1.1 255.255.255.255
!route test subnet out inside interface to fabric
ip route 10.1.10.0 255.255.255.0 10.1.1.1
!null route summary address for BGP advertisement
ip route 10.1.0.0 255.255.0.0 null0
!route tunnel 2 DIP out second public interface
ip route "Azure-VNGpubip2" 255.255.255.255 10.1.2.1

crypto ikev2 proposal to-onprem-proposal
  encryption aes-cbc-256
  integrity  sha1
  group      2
  exit

crypto ikev2 proposal to-onprem-proposal2
  encryption aes-cbc-256
  integrity  sha1
  group      2
  exit

crypto ikev2 policy to-onprem-policy
  proposal to-onprem-proposal
  match address local 10.1.0.4
  exit

crypto ikev2 policy to-onprem-policy2
  proposal to-onprem-proposal2
  match address local 10.1.2.4
  exit

crypto ikev2 keyring to-onprem-keyring
  peer "Azure-VNGpubip"
    address "Azure-VNGpubip"
    pre-shared-key Msft123Msft123
    exit
  exit

crypto ikev2 keyring to-onprem-keyring2
  peer "Azure-VNGpubip2"
    address "Azure-VNGpubip2"
    pre-shared-key Msft123Msft123
    exit
  exit

crypto ikev2 profile to-onprem-profile
  match address  local 10.1.0.4
  match identity remote address "Azure-VNGpubip" 255.255.255.255
  authentication remote pre-share
  authentication local  pre-share
  lifetime 3600
  dpd 10 5 on-demand
  keyring local  to-onprem-keyring
  exit

crypto ikev2 profile to-onprem-profile2
  match address  local 10.1.2.4
  match identity remote address "Azure-VNGpubip2" 255.255.255.255
  authentication remote pre-share
  authentication local  pre-share
  lifetime 3600
  dpd 10 5 on-demand
  keyring local  to-onprem-keyring2
  exit

crypto ipsec transform-set to-onprem-TransformSet esp-gcm 256 
  mode tunnel
  exit

crypto ipsec transform-set to-onprem-TransformSet2 esp-gcm 256 
  mode tunnel
  exit

crypto ipsec profile to-onprem-IPsecProfile
  set transform-set  to-onprem-TransformSet
  set ikev2-profile  to-onprem-profile
  set security-association lifetime seconds 3600
  exit

crypto ipsec profile to-onprem-IPsecProfile2
  set transform-set  to-onprem-TransformSet2
  set ikev2-profile  to-onprem-profile2
  set security-association lifetime seconds 3600
  exit

do term mon

int tunnel 11
  ip address 192.168.1.1 255.255.255.255
  tunnel mode ipsec ipv4
  ip tcp adjust-mss 1350
  tunnel source 10.1.0.4
  tunnel destination "Azure-VNGpubip"
  tunnel protection ipsec profile to-onprem-IPsecProfile
  exit

int tunnel 12
  ip address 192.168.1.2 255.255.255.255
  tunnel mode ipsec ipv4
  ip tcp adjust-mss 1350
  tunnel source 10.1.2.4
  tunnel destination "Azure-VNGpubip2"
  tunnel protection ipsec profile to-onprem-IPsecProfile2
  exit


  router bgp 65002
  bgp log-neighbor-changes
  bgp router-id 1.1.1.1
  neighbor 10.0.0.4 remote-as 65001
  neighbor 10.0.0.4 ebgp-multihop 255
  neighbor 10.0.0.4 update-source tunnel 11
  neighbor 10.0.0.5 remote-as 65001
  neighbor 10.0.0.5 ebgp-multihop 255
  neighbor 10.0.0.5 update-source tunnel 12

  address-family ipv4
    maximum-paths 2
    network 10.1.0.0 mask 255.255.0.0
network 1.1.1.1 mask 255.255.255.255
    neighbor 10.0.0.4 activate
    neighbor 10.0.0.5 activate     
    exit
  exit

!route BGP peer1 IP over the tunnel
ip route 10.0.0.4 255.255.255.255 Tunnel 11

!route BGP peer2 IP over the tunnel
ip route 10.0.0.5 255.255.255.255 Tunnel 12


Step 13: At this point you should have an 2x IKEv2 tunnels from on prem to the Azure VPN GW. Here are a few commands and expected outputs. It’s important you have reachability across the tunnels before moving on.

CSR1#sh ip bgp sum
BGP router identifier 1.1.1.1, local AS number 65002
BGP table version is 6, main routing table version 6
5 network entries using 1240 bytes of memory
8 path entries using 1152 bytes of memory
3 multipath network entries and 6 multipath paths
2/2 BGP path/bestpath attribute entries using 576 bytes of memory
1 BGP AS-PATH entries using 24 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 2992 total bytes of memory
BGP activity 5/0 prefixes, 8/0 paths, scan interval 60 secs
5 networks peaked at 19:05:14 Feb 28 2019 UTC (00:06:09.184 ago).

Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.0.0.4        4        65001       9      11        6    0    0 00:06:05        3
10.0.0.5        4        65001       9      11        6    0    0 00:06:08        3

CSR1#sh ip bgp neighbors 10.0.0.4   
BGP neighbor is 10.0.0.4,  remote AS 65001, external link
  BGP version 4, remote router ID 10.0.0.4
  BGP state = Established
*truncated*

CSR1#sh ip bgp neighbors 10.0.0.5
BGP neighbor is 10.0.0.5,  remote AS 65001, external link
  BGP version 4, remote router ID 10.0.0.5
  BGP state = Established
*truncated*

CSR1#sh ip bgp
BGP table version is 6, local router ID is 1.1.1.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, 
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, 
              x best-external, a additional-path, c RIB-compressed, 
              t secondary path, L long-lived-stale,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path
 *>   1.1.1.1/32       0.0.0.0                  0         32768 i
 *>   10.0.0.0/16      10.0.0.4                               0 65001 i
 *m                    10.0.0.5                               0 65001 i
 *>   10.1.0.0/16      0.0.0.0                  0         32768 i
 r>   192.168.1.1/32   10.0.0.4                               0 65001 i
 rm                    10.0.0.5                               0 65001 i
 r>   192.168.1.2/32   10.0.0.4                               0 65001 i
 rm                    10.0.0.5                               0 65001 i

CSR1#sh ip route bgp
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area 
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2, m - OMP
       n - NAT, Ni - NAT inside, No - NAT outside, Nd - NAT DIA
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       H - NHRP, G - NHRP registered, g - NHRP registration summary
       o - ODR, P - periodic downloaded static route, l - LISP
       a - application route
       + - replicated route, % - next hop override, p - overrides from PfR

Gateway of last resort is 10.1.0.1 to network 0.0.0.0

      10.0.0.0/8 is variably subnetted, 11 subnets, 3 masks
B        10.0.0.0/16 [20/0] via 10.0.0.5, 00:09:44
                     [20/0] via 10.0.0.4, 00:09:44
   
CSR1#sh run | s router bgp
router bgp 65002
 bgp router-id 1.1.1.1
 bgp log-neighbor-changes
 neighbor 10.0.0.4 remote-as 65001
 neighbor 10.0.0.4 ebgp-multihop 255
 neighbor 10.0.0.4 update-source Tunnel11
 neighbor 10.0.0.5 remote-as 65001
 neighbor 10.0.0.5 ebgp-multihop 255
 neighbor 10.0.0.5 update-source Tunnel12
 !
 address-family ipv4
  network 1.1.1.1 mask 255.255.255.255
  network 10.1.0.0 mask 255.255.0.0
  neighbor 10.0.0.4 activate
  neighbor 10.0.0.5 activate
  maximum-paths 2
 exit-address-family

CSR1#sh run | s route
router bgp 65002
 bgp router-id 1.1.1.1
 bgp log-neighbor-changes
 neighbor 10.0.0.4 remote-as 65001
 neighbor 10.0.0.4 ebgp-multihop 255
 neighbor 10.0.0.4 update-source Tunnel11
 neighbor 10.0.0.5 remote-as 65001
 neighbor 10.0.0.5 ebgp-multihop 255
 neighbor 10.0.0.5 update-source Tunnel12
 !
 address-family ipv4
  network 1.1.1.1 mask 255.255.255.255
  network 10.1.0.0 mask 255.255.0.0
  neighbor 10.0.0.4 activate
  neighbor 10.0.0.5 activate
  maximum-paths 2
 exit-address-family
ip route 10.0.0.4 255.255.255.255 Tunnel11
ip route 10.0.0.5 255.255.255.255 Tunnel12
ip route 10.1.0.0 255.255.0.0 Null0
ip route 10.1.10.0 255.255.255.0 10.1.1.1
ip route 13.92.135.131 255.255.255.255 10.1.2.1
ip route vrf GS 0.0.0.0 0.0.0.0 GigabitEthernet1 10.1.0.1 global

CSR1#sh tcp brief
TCB       Local Address               Foreign Address             (state)
7F1E2CA12AC8  192.168.1.1.52646          10.0.0.4.179                ESTAB
7F1DB4319370  192.168.1.2.31300          10.0.0.5.179                ESTAB

CSR1#sh ip bgp neighbors 10.0.0.4 advertised-routes
BGP table version is 6, local router ID is 1.1.1.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, 
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, 
              x best-external, a additional-path, c RIB-compressed, 
              t secondary path, L long-lived-stale,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path
 *>   1.1.1.1/32       0.0.0.0                  0         32768 i
 *>   10.0.0.0/16      10.0.0.4                               0 65001 i
 *>   10.1.0.0/16      0.0.0.0                  0         32768 i
 r>   192.168.1.1/32   10.0.0.4                               0 65001 i
 r>   192.168.1.2/32   10.0.0.4                               0 65001 i

Total number of prefixes 5 

CSR1#sh ip bgp neighbors 10.0.0.5 advertised-routes
BGP table version is 6, local router ID is 1.1.1.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, 
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, 
              x best-external, a additional-path, c RIB-compressed, 
              t secondary path, L long-lived-stale,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path
 *>   1.1.1.1/32       0.0.0.0                  0         32768 i
 *>   10.0.0.0/16      10.0.0.4                               0 65001 i
 *>   10.1.0.0/16      0.0.0.0                  0         32768 i
 r>   192.168.1.1/32   10.0.0.4                               0 65001 i
 r>   192.168.1.2/32   10.0.0.4                               0 65001 i

Total number of prefixes 5

CSR1#sh crypto ikev2  sa
 IPv4 Crypto IKEv2  SA 

Tunnel-id Local                 Remote                fvrf/ivrf            Status 
2         10.1.2.4/4500         13.92.135.131/4500    none/none            READY  
      Encr: AES-CBC, keysize: 256, PRF: SHA1, Hash: SHA96, DH Grp:2, Auth sign: PSK, Auth verify: PSK
      Life/Active Time: 3600/907 sec

Tunnel-id Local                 Remote                fvrf/ivrf            Status 
1         10.1.0.4/4500         13.92.173.131/4500    none/none            READY  
      Encr: AES-CBC, keysize: 256, PRF: SHA1, Hash: SHA96, DH Grp:2, Auth sign: PSK, Auth verify: PSK
      Life/Active Time: 3600/907 sec

 IPv6 Crypto IKEv2  SA

Azure CLI:
C:\Users\jewrigh>az network vpn-connection show --name to-onprem --resource-group Hub --query "{status: connectionStatus}"
{
  "status": "Connected"
  
C:\Users\jewrigh>az network vpn-connection show --name to-onprem2 --resource-group Hub --query "{status: connectionStatus}"
{
  "status": "Connected"
}

Step 14: Create VM in both Azure and on prem testVMsubnets:
az network public-ip create --name HubVMPubIP --resource-group Hub --location eastus --allocation-method Dynamic
az network nic create --resource-group Hub -n HubVMNIC --location eastus --subnet testVMSubnet --private-ip-address 10.0.10.10 --vnet-name Hub --public-ip-address HubVMPubIP
az vm create -n HubVM -g Hub --image UbuntuLTS --admin-username azureuser --admin-password Msft123Msft123 --nics HubVMNIC


az network public-ip create --name onpremVMPubIP --resource-group onprem --location eastus2 --allocation-method Dynamic
az network nic create --resource-group onprem -n onpremVMNIC --location eastus2 --subnet testVMSubnet --private-ip-address 10.1.10.10 --vnet-name onprem --public-ip-address onpremVMPubIP
az vm create -n onpremVM -g onprem --image UbuntuLTS --admin-username azureuser --admin-password Msft123Msft123 --nics onpremVMNIC


Step 15: Create route table for the onprem VNET and steer all necessary traffic to 10.100.1.4 (CSR# inside):

az network route-table create --name vm-rt --resource-group onprem
az network route-table route create --name vm-rt --resource-group onprem --route-table-name vm-rt --address-prefix 10.0.0.0/16 --next-hop-type VirtualAppliance --next-hop-ip-address 10.1.1.4
az network vnet subnet update --name testVMSubnet --vnet-name onprem --resource-group onprem --route-table vm-rt

Step 16: Test scenario ideas:
az network public-ip show -g onprem -n onpremVMPubIP --query "{address: ipAddress}"

SSH to on prem VM. Initiate ping to 10.0.10.10 (Azure VM) and drop tunnel 11. If traffic was traversing that tunnel, it will take ~30 seconds for route reconvergence. You may also see a couple minor errors bringing the BGP peer up and bringing tunnel 11 back up. Usually this is associated with bouncing the tunnel too fast. It should clear on it's own in a few seconds.


When both tunnels are active, the Azure side VM will see 2 valid next hops for 10.1.0.0/16 in the effective route table. When you drop a single tunnel, the effective route table will show 1 next hop.