This document will run you through a step by step configuration example of Inter VRF route leaking in Cisco Application Centric Infrastructure(ACI). This article describes the steps with screen shots of example configuration and some verification on the command line interface where relevant.
This procedure is applicable for 2 different VRF's in same tenant or separate tenant.
Note: This configuration has been tested on version 2.3(1f). Screen shots below may vary slightly with later versions.
In this document, we will be configuring and describing only about configurations relating to inter VRF leaking. Hence prior knowledge of ACI concepts and configurations like EPG, Bridge Domain, VRF and L3out is essential.
For the purpose of this document, below devices/components have been used.
1. ACI fabric with 3 leafs, 2 spines and 3 APICs running version 2.3(1f) 2. Nexus 7000, configured as a router running OSPF advertising routes to ACI fabric
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any configuration/command.
Below topology will be used for the purpose of this configuration example.
IP and other details:
Consumer VM : VM has the IP of 10.10.10.1/24 and has the gateway 10.10.10.254/24 in ACI deployed in VRF consumer-vrf. Provider VM: VM has the IP of 10.10.20.1/24 and has the gateway 10.10.20.254/24 in ACI deployed in VRF provider-vrf. L3out: L3out is deployed in ACI leaf 101 and 102 (vPC) connecting to Nexus 7K acting as router here. SVI interface vlan15 (188.8.131.52/24) is used for forming the OSPF adjacency and loopback 200 interface(184.108.40.206/32) will be used for testing the connectivity from Provider EPG to the l3out.
Before configuring anything related to inter-VRF route leaking, let's verify below configurations.
1. VRF provider-vrf has been created with Bridge Domain(BD) provider-bd. The endpoint 10.10.20.1 has been learned in EPG provider-epg through leaf 201/1/1
2. VRF consumer-vrf has been created with Bridge Domain(BD) consumer-bd. The endpoint 10.10.20.1 has been learned in EPG consumer-epg through leaf 201/1/17
3. Both above endpoints are able to ping their respective gateways deployed in ACI.
4. The l3out consumer-l3out has been deployed and ACI has learned the routes from the external router. 5. As you see below, consumer-vrf on leaf 201 has the route 10.10.10.0/24 which is local and learned 220.127.116.11/32 through the overlay-1 vrf from leaf 101 and 102 where the l3out is deployed.
6. Also, as you see below the provider-vrf just has its local route 10.10.20.0/24.
Now, let us start with the Inter-VRF configurations
Inter-VRF route leaking for communication between VMs in 2 different tenants/vrf's
Step1: Configure shared subnet under the provider-epg as opposed to configuring under BD.
Step2: Configure shared subnet under consumer-bd
Step3: Create a global contract in tenant provider-tenant
Step4: Export above contract to the consumer-tenant
Make sure to export the right contract and select the right tenant to export the contract to.
NOTE: If you are configuring inter-VRF routing between two VRFs in the same Tenant, there is no need to export/import the contract.
Step5: Apply the above contract as provided contract under provider-epg
Step6: Apply above exported global contract as consumed contract interface under the consumer-vrf
Now the communication between the VM's in provider-epg and consumer-epg should work based on the contract filter. In my case, I am using default filter which is allow-all.
As you can see below, the provider subnet 10.10.20.0/24 has been advertised to consumer-vrf and the consumer subnet 10.10.10.0/24 has been advertised to provider-vrf.
Inter-VRF route leaking for communication from a router connected to ACI through an l3out in consumer-vrf to the VM in provider-epg
In this case, I already have the l3out created in consumer-tenant and we saw earlier that the subnet 18.104.22.168/32 is being advertised from the external router to ACI. Let us make some changes to the l3out now so that we can advertise this route to the provider-vrf.
Note: Above EPG's are configured in leaf 201 and hence both consumer-vrf and provider-vrf has been deployed in leaf 201. However, we only have a l3out configured in leaf 101 and 102 which is in consumer-vrf. So, we would only see consumer-vrf deployed in leaf 101 and 102. Also, consumer-vrf here wouldn't learn the routes yet for the provider-vrf as it need not be programmed in leaf 101 and 102 as yet.
First, let's advertise subnet 22.214.171.124/32 learned from external router to provider-vrf
Step1: Under the l3out EPG in consumer-tenant, tick the check box for "Shared route control subnet" and "Shared security import subnet" along with the default "External subnet for External EPG"
Note: Shared route control subnet - Advertises the routes to another VRF based on where the contract is applied Shared security import subnet - Allows the communication between the subnet and another VRF.
In this example, I am configuring only 126.96.36.199/32 subnet to be advertised to ACI. This can be configured to 0.0.0.0/0 to allow all subnets from external router to be advertised to ACI.
Step2: Under the l3out EPG in consumer-tenant, apply the consumed contract interface that was imported from provider-tenant.
Now, the route 188.8.131.52/32 should have been advertised to the provider-vrfin compute leaf 201 and the provider subnet 10.10.20.0/24 should have been advertised to consumer-vrf in border leaf 101 and 102.
We need to complete 1 more step to actually establish the communication between a VM in provider-vrf to an interface in external router belonging to consumer-vrf. We now need to advertise the provider subnet 10.10.20.0/24 out of l3out to the external router so that the external router knows the path for the traffic to provider-vrf.
Step3: Advertise the provider subnet 10.10.20.0/24 out of l3out to the external router in consumer-vrf.
From the external router, if we check the routes now, we should be able to see ACI advertising the route 10.10.20.0/24. Ping to the VM 10.10.20.1 in provider-vrf should now work fine from the external router in consumer-vrf.
Check the routes in consumer-vrfand provider-vrf on compute leaf 201 to confirm the routes are leaked across the VRF.