Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements

Welcome to Cisco Support Community. We would love to have your feedback.

For an introduction to the new site, click here. If you'd prefer to explore, try our test area to get started. And see here for current known issues.

New Member

Integrating new BGP routers into existing Nexus 5K network

Hi,

I am having a little trouble wrapping my head around how to do this.  We have an existing network that consists of Nexus 5K in the data center.  We are now in the process of designing and implementing a new site into the new network.

I have attached a diagram of the network segment in question.

To simplify management of the routes we want to integrate the new routers into the BGP network (both the 2900 routers are new).  What I am having trouble grasping is how to intigrate the new network with the existing network.

In a lab, I am only able to set up the VRF, BGP scenario using RD and RT.  From my understanding, and correct me if I am wrong, If I use RD and RT on the 2900s I would need the same setup on the Nexus for traffic to pass.  Under normal circumstances I could just add RDs and RTs to the N5K but then I would also need to go to all the other locations and configure them as well.

I would be greatful for some input on how I can integrate these new routers with the existing network and maintain dynamic routing using BGP.

Thanks

Everyone's tags (4)
1 ACCEPTED SOLUTION

Accepted Solutions
Hall of Fame Super Blue

Integrating new BGP routers into existing Nexus 5K network

If all inter VRF routing is handled by the firewall then your setup may be quite simple.  You probably don't want BGP/MPLS on the dark fibre link because -

lets say it was one vlan per VRF -

1) if you import/export routes on the 2911 at the main site then each VRF can see other VRFs routes. If you only want to route via the firewall there is no need to do this. In fact, as far as i can see you don't want to import/export any routes, you simply use subinterfaces on the firewall to control the flow of traffic.

The way to completely isolate the traffic is -

1) extend the VRFs all the way from the main site to the remote site. You would do this by creating subinterfaces on the dark fiber connections between the 2900s and then place each subinterface into the corresponding VRF.

2) on each LAN facing interface of the 2900s you again create the same subinterfaces and assign into the corresponding VRFs.

3) from each LAN interface you run a trunk back to the switch. (Not sure what you have in the remote site). In the main site that trunk would go to the Nexus switch and then there would be a trunk from the Nexus switch to the firewall. If it was one vlan per VRF there would be no need for SVIs on the Nexus switch.

Then per VRF you can run a routing protocol if needed eg. EIGRP or OSPF.

With the above the only way for any device in a particular VRF to communicate with any other device is via a firewall interface. There is no import/export of any routes. The subinterfaces on the firewall are not in VRFs and the firewall has a global routing table containing all the routes from each VRF and you strictly control access with stateful filtering.

If you couldn't use subinterfaces on the dark fiber connections you could always look at GRE tunnnels.

The above assumes it is one vlan per VRF. If it is multiple vlans then there would be a need for SVIs on the Nexus so routing between vlans in the same VRF could be done. Same applies at remote site.

If you do run BGP to import/export on the 2900s then you are then exchanging routes between VRFs. If the clients default gateway was set to the firewall then you should still get separation but if the client changed the gateway to the 2900 subinterface for example then that device would have routes within each VRF for other VRFs.

So based on what you have described i can't see the need for any importing/exporting of routes.

Does this make sense ?

Jon

21 REPLIES
New Member

Integrating new BGP routers into existing Nexus 5K network

You diagram is very vague. Is it necessary to implement BGP from the site to site?

New Member

Integrating new BGP routers into existing Nexus 5K network

it is not necessary, but would be the solution of choice.

New Member

Integrating new BGP routers into existing Nexus 5K network

I am not sure i would consider it a solution when you dont need to. Running some other IGP and then redistributing would be a better choice in my mind. I run iBGP between my site to site, but again each site as eBGP peering to the MPLS cloud.

New Member

Integrating new BGP routers into existing Nexus 5K network

OK,  But is it possible to use VRF with BGP without the use of RD and RT?

New Member

Integrating new BGP routers into existing Nexus 5K network

Hall of Fame Super Blue

Re: Integrating new BGP routers into existing Nexus 5K network

In a lab, I am only able to set up the VRF, BGP scenario using RD and RT.  From my understanding, and correct me if I am wrong, If I use RD and RT on the 2900s I would need the same setup on the Nexus for traffic to pass

No you wouldn't. It is what we discussed in your previous thread.  It entirely depends on where you want to route between the VRFs. There are two issues here -

1) dark fiber interconnect. If you could use L3 subinterfaces on each 2900's fiber connection then you could run the VRFs between the sites without using BGP/MPLS at all.

If you cannot do this then you may need MPLS with MP-BGP to preserve the VPN separation across the fiber connection.

2) where do you want to route between VRFs ?  Is it on the local 2900, the Nexus switches or the firewall. If you want to route between VRFs and not go via the firewall you only have to use BGP to import/export routes on one device.

3) do you need to be able to route between VRFs at the remote site without having to go via the main site ?

So from your diagram it is impossible to answer any of the above. If you could tell us exactly how you wanted it to work in terms of inter VRF connectivity we may be able to help you but you do not necessarily need BGP/MPLS at all and even if you did you do not have to use it on every device in the network.

Jon

New Member

Re: Integrating new BGP routers into existing Nexus 5K network

1) dark fiber interconnect. If you could use L3 subinterfaces on each  2900's fiber connection then you could run the VRFs between the sites  without using BGP/MPLS at all.

If you cannot do this then you may need MPLS with MP-BGP to preserve the VPN separation across the fiber connection.

They want to keep the routing protocol uniform across the network, meaning the company wants use to use BGP...if possible.  But this is not carved in stone.  I know I am able to do this using another IGP protocol.

2) where do you want to route between VRFs ?  Is it on the local 2900,  the Nexus switches or the firewall. If you want to route between VRFs  and not go via the firewall you only have to use BGP to import/export  routes on one device.

All routing between VRFs will be done via the firewall.

3) do you need to be able to route between VRFs at the remote site without having to go via the main site ?

Inter VLAN routing will be handled by the firewall.

Hall of Fame Super Blue

Integrating new BGP routers into existing Nexus 5K network

If all inter VRF routing is handled by the firewall then your setup may be quite simple.  You probably don't want BGP/MPLS on the dark fibre link because -

lets say it was one vlan per VRF -

1) if you import/export routes on the 2911 at the main site then each VRF can see other VRFs routes. If you only want to route via the firewall there is no need to do this. In fact, as far as i can see you don't want to import/export any routes, you simply use subinterfaces on the firewall to control the flow of traffic.

The way to completely isolate the traffic is -

1) extend the VRFs all the way from the main site to the remote site. You would do this by creating subinterfaces on the dark fiber connections between the 2900s and then place each subinterface into the corresponding VRF.

2) on each LAN facing interface of the 2900s you again create the same subinterfaces and assign into the corresponding VRFs.

3) from each LAN interface you run a trunk back to the switch. (Not sure what you have in the remote site). In the main site that trunk would go to the Nexus switch and then there would be a trunk from the Nexus switch to the firewall. If it was one vlan per VRF there would be no need for SVIs on the Nexus switch.

Then per VRF you can run a routing protocol if needed eg. EIGRP or OSPF.

With the above the only way for any device in a particular VRF to communicate with any other device is via a firewall interface. There is no import/export of any routes. The subinterfaces on the firewall are not in VRFs and the firewall has a global routing table containing all the routes from each VRF and you strictly control access with stateful filtering.

If you couldn't use subinterfaces on the dark fiber connections you could always look at GRE tunnnels.

The above assumes it is one vlan per VRF. If it is multiple vlans then there would be a need for SVIs on the Nexus so routing between vlans in the same VRF could be done. Same applies at remote site.

If you do run BGP to import/export on the 2900s then you are then exchanging routes between VRFs. If the clients default gateway was set to the firewall then you should still get separation but if the client changed the gateway to the 2900 subinterface for example then that device would have routes within each VRF for other VRFs.

So based on what you have described i can't see the need for any importing/exporting of routes.

Does this make sense ?

Jon

Hall of Fame Super Blue

Re: Integrating new BGP routers into existing Nexus 5K network

I have assumed in the above that if any client in a VRF needs to talk to a non VRF device eg. some existing device in the network this would also have to be routed off the firewall ie. you have a firewall interface to get to existing non VRF devices.

If this is not the case and you only want the firewall to control inter VRF routing but not VRF to non VRF routing could you clarify exactly how you want that bit to work as what i have written in my last post may not be an applicable solution.

Jon

New Member

Re: Integrating new BGP routers into existing Nexus 5K network

The firewall will also be responsible for routing VRF traffic into the global routing domain.

Hall of Fame Super Blue

Integrating new BGP routers into existing Nexus 5K network

Well i would look at what i suggested then in my last but one post.

The only real issue left is do you have multiple vlans per VRF. That still wouldn't mean you need BGP for route import/export but you would need SVIs on the Nexus to do inter vlan routing with the same VRF.

I think MPLS/BGP would not only complicate the solution but actually provide less isolation than not using it.

Jon

New Member

Integrating new BGP routers into existing Nexus 5K network

Though we do not have any restrictions on using subinterfaces, we are considering using GRE tunnels already.  But might reconsider the subinterface option.

So we would also need an OSPF instance on the nexus for the redistribution.  I agree that your solution is most likely the best option we have here.

Thanks for your insight   will post again if we need more assistance .

Hall of Fame Super Blue

Integrating new BGP routers into existing Nexus 5K network

So we would also need an OSPF instance on the nexus for the redistribution.

Not sure i follow. Redistribution from what to what.

One other thing i should have said. If you did want to run MPLS/BGP between the sites then it wouldn't actually make it less secure as long as you did not import/export between VRFs.

I didn't want to give the impression that it was the MPLS/BGP that created less isolation because on it's own it wouldn't but if you then imported/exported routes that would.

Jon

New Member

Integrating new BGP routers into existing Nexus 5K network

So we would also need an OSPF instance on the nexus for the redistribution.

Not sure i follow. Redistribution from what to what.

the new network also needs to be available to the locations that connect to the WAN.  The diagram I posted was just what is going to be implemented there is much more that encompasses the network.  And also part of the reason I was looking in to using BGP.  As it stands with your suggested solution, I would also need to redistribute the ospf processes into BGP.

Hall of Fame Super Blue

Re: Integrating new BGP routers into existing Nexus 5K network

But you are not running any L3 for the VRF networks on the Nexus so you can't redistribute there ie. the Nexus never sees any of the VRF routes, the only devices that see them are the 2900s per VRF and the firewall in it's global routing table.

The whole idea behind the suggestion i posted was that only the firewall will see all routes.

The WAN device running BGP needs to have the VRF routes in it's routing table to be able to advertise them into BGP.

So what is the WAN device running BGP that all the remote sites come in to ?

I suspect you may need a different solution as it would appear that you do actually need those VRF routes available to a device that is neither the 2900 nor the firewall.

Can you clarify how the rest of the WAN connectivity works.

Jon

New Member

Integrating new BGP routers into existing Nexus 5K network

The Nexus 5K pairs run BGP and connect to the WAN.  Some connections are over a IPVPN others over dark fiber.  Currently BGP is the only routing protocol in the network.

The firewall does route between VRFs but traffic within a VRF should be routed directly to its destination.  this is the reason I was looking into using BGP.

Hall of Fame Super Blue

Re: Integrating new BGP routers into existing Nexus 5K network

The firewall does route between VRFs but traffic within a VRF should be routed directly to its destination.  this is the reason I was looking into using BGP

Do you have multiple subnets per VRF then ? If you do you don't need BGP for this, you simply assign multiple SVIs into the same VRF.

Where it is more complicated is the existing WAN. I am assuming you do not want to extend the VRFs to existing sites which would mean configuration on all existing sites ?  If not then presumably you just want to advertise out the VRF networks to the existing sites with BGP and implement the isolation in the main site. You also wouldn't advertise the VRF networks out from the new remote site.

If so then if you do the route leaking on the Nexus then it will simply route directly between the existing network and the new VRF networks without going via the firewall which is not what you want. Two solutions spring to mind -

1) use static routes for the VRF networks on the Nexus switches pointing to the firewall production interface and then you can simply advertise these out via BGP.

or

2) run OPSF between the firewall and the Nexus switches and have the firewall advertise all the VRF routes and redistribute this into BGP. 

Both of the above would mean the Nexus had to send traffic to the firewall to get to the VRF networks which is what you want.

Jon

New Member

Re: Integrating new BGP routers into existing Nexus 5K network

Your second option is actually what I was just thinking about.  Thanks for the insight

New Member

Re: Integrating new BGP routers into existing Nexus 5K network

I think I have been down the road with VRF's on the 5k. I do not believe the 5ks support inter-VRF routing or route leaking. Dont quote me though.

New Member

Re: Integrating new BGP routers into existing Nexus 5K network

Nevermind. You can do this using RT's in each VRF on the 5k. You must be very careful not to route traffic over the peer-link?

New Member

Re: Integrating new BGP routers into existing Nexus 5K network

Not to worry, only intra VRF traffic is routed on the N5K, all inter VRF traffic goes through a firewall.

Thanks for the heads up though

596
Views
0
Helpful
21
Replies