Integrating new BGP routers into existing Nexus 5K network

[CiscoNutt]

Hi,

I am having a little trouble wrapping my head around how to do this.  We have an existing network that consists of Nexus 5K in the data center.  We are now in the process of designing and implementing a new site into the new network.

I have attached a diagram of the network segment in question.

To simplify management of the routes we want to integrate the new routers into the BGP network (both the 2900 routers are new).  What I am having trouble grasping is how to intigrate the new network with the existing network.

In a lab, I am only able to set up the VRF, BGP scenario using RD and RT.  From my understanding, and correct me if I am wrong, If I use RD and RT on the 2900s I would need the same setup on the Nexus for traffic to pass.  Under normal circumstances I could just add RDs and RTs to the N5K but then I would also need to go to all the other locations and configure them as well.

I would be greatful for some input on how I can integrate these new routers with the existing network and maintain dynamic routing using BGP.

Thanks

[Steven Williams]

You diagram is very vague. Is it necessary to implement BGP from the site to site?

[CiscoNutt]

it is not necessary, but would be the solution of choice.

[Steven Williams]

I am not sure i would consider it a solution when you dont need to. Running some other IGP and then redistributing would be a better choice in my mind. I run iBGP between my site to site, but again each site as eBGP peering to the MPLS cloud.

[CiscoNutt]

OK,  But is it possible to use VRF with BGP without the use of RD and RT?

[Steven Williams]

You can use VRF export maps.

http://packetlife.net/blog/2013/sep/26/vrf-export-maps/

[Jon Marshall]

In a lab, I am only able to set up the VRF, BGP scenario using RD and RT.  From my understanding, and correct me if I am wrong, If I use RD and RT on the 2900s I would need the same setup on the Nexus for traffic to pass

No you wouldn't. It is what we discussed in your previous thread.  It entirely depends on where you want to route between the VRFs. There are two issues here -

1) dark fiber interconnect. If you could use L3 subinterfaces on each 2900's fiber connection then you could run the VRFs between the sites without using BGP/MPLS at all.

If you cannot do this then you may need MPLS with MP-BGP to preserve the VPN separation across the fiber connection.

2) where do you want to route between VRFs ?  Is it on the local 2900, the Nexus switches or the firewall. If you want to route between VRFs and not go via the firewall you only have to use BGP to import/export routes on one device.

3) do you need to be able to route between VRFs at the remote site without having to go via the main site ?

So from your diagram it is impossible to answer any of the above. If you could tell us exactly how you wanted it to work in terms of inter VRF connectivity we may be able to help you but you do not necessarily need BGP/MPLS at all and even if you did you do not have to use it on every device in the network.

Jon

[CiscoNutt]

1) dark fiber interconnect. If you could use L3 subinterfaces on each  2900's fiber connection then you could run the VRFs between the sites  without using BGP/MPLS at all. 
If you cannot do this then you may need MPLS with MP-BGP to preserve the VPN separation across the fiber connection. 

They want to keep the routing protocol uniform across the network, meaning the company wants use to use BGP...if possible.  But this is not carved in stone.  I know I am able to do this using another IGP protocol.

2) where do you want to route between VRFs ?  Is it on the local 2900,  the Nexus switches or the firewall. If you want to route between VRFs  and not go via the firewall you only have to use BGP to import/export  routes on one device. 

All routing between VRFs will be done via the firewall.

3) do you need to be able to route between VRFs at the remote site without having to go via the main site ?

Inter VLAN routing will be handled by the firewall.

[Jon Marshall · Accepted solution]

If all inter VRF routing is handled by the firewall then your setup may be quite simple.  You probably don't want BGP/MPLS on the dark fibre link because -

lets say it was one vlan per VRF -

1) if you import/export routes on the 2911 at the main site then each VRF can see other VRFs routes. If you only want to route via the firewall there is no need to do this. In fact, as far as i can see you don't want to import/export any routes, you simply use subinterfaces on the firewall to control the flow of traffic.

The way to completely isolate the traffic is -

1) extend the VRFs all the way from the main site to the remote site. You would do this by creating subinterfaces on the dark fiber connections between the 2900s and then place each subinterface into the corresponding VRF.

2) on each LAN facing interface of the 2900s you again create the same subinterfaces and assign into the corresponding VRFs.

3) from each LAN interface you run a trunk back to the switch. (Not sure what you have in the remote site). In the main site that trunk would go to the Nexus switch and then there would be a trunk from the Nexus switch to the firewall. If it was one vlan per VRF there would be no need for SVIs on the Nexus switch.

Then per VRF you can run a routing protocol if needed eg. EIGRP or OSPF.

With the above the only way for any device in a particular VRF to communicate with any other device is via a firewall interface. There is no import/export of any routes. The subinterfaces on the firewall are not in VRFs and the firewall has a global routing table containing all the routes from each VRF and you strictly control access with stateful filtering.

If you couldn't use subinterfaces on the dark fiber connections you could always look at GRE tunnnels.

The above assumes it is one vlan per VRF. If it is multiple vlans then there would be a need for SVIs on the Nexus so routing between vlans in the same VRF could be done. Same applies at remote site.

If you do run BGP to import/export on the 2900s then you are then exchanging routes between VRFs. If the clients default gateway was set to the firewall then you should still get separation but if the client changed the gateway to the 2900 subinterface for example then that device would have routes within each VRF for other VRFs.

So based on what you have described i can't see the need for any importing/exporting of routes.

Does this make sense ?

Jon

[Jon Marshall]

I have assumed in the above that if any client in a VRF needs to talk to a non VRF device eg. some existing device in the network this would also have to be routed off the firewall ie. you have a firewall interface to get to existing non VRF devices.

If this is not the case and you only want the firewall to control inter VRF routing but not VRF to non VRF routing could you clarify exactly how you want that bit to work as what i have written in my last post may not be an applicable solution.

Jon

[CiscoNutt]

The firewall will also be responsible for routing VRF traffic into the global routing domain.

[Jon Marshall]

Well i would look at what i suggested then in my last but one post.

The only real issue left is do you have multiple vlans per VRF. That still wouldn't mean you need BGP for route import/export but you would need SVIs on the Nexus to do inter vlan routing with the same VRF.

I think MPLS/BGP would not only complicate the solution but actually provide less isolation than not using it.

Jon

[CiscoNutt]

Though we do not have any restrictions on using subinterfaces, we are considering using GRE tunnels already.  But might reconsider the subinterface option.

So we would also need an OSPF instance on the nexus for the redistribution.  I agree that your solution is most likely the best option we have here.

Thanks for your insight   will post again if we need more assistance .

[Jon Marshall]

So we would also need an OSPF instance on the nexus for the redistribution.

Not sure i follow. Redistribution from what to what.

One other thing i should have said. If you did want to run MPLS/BGP between the sites then it wouldn't actually make it less secure as long as you did not import/export between VRFs.

I didn't want to give the impression that it was the MPLS/BGP that created less isolation because on it's own it wouldn't but if you then imported/exported routes that would.

Jon

[CiscoNutt]

So we would also need an OSPF instance on the nexus for the redistribution.
Not sure i follow. Redistribution from what to what. 

the new network also needs to be available to the locations that connect to the WAN.  The diagram I posted was just what is going to be implemented there is much more that encompasses the network.  And also part of the reason I was looking in to using BGP.  As it stands with your suggested solution, I would also need to redistribute the ospf processes into BGP.