Solved: L3 switch for interVLAN and router

ALIAOF_ · ‎11-14-2013

I'm working on a project where I need my layer 3 switch to handle all the interVLAN routing and DHCP

And all the internet and traffic to the corporate office needs to hit the router.

I have the router setup as 10.255.255.1 --> then a port on the switch as 10.255.255.6

Next I have a route on the switch, "ip route 0.0.0.0 0.0.0.0 10.255.255.1" and I did enable routing on the switch by "ip routing"

I have a loopback on the switch 172.16.11.1

Router has BGP setup to advertise the networks

I setup one port on a VLAN got a DHCP IP, but I can't ping the router 10.255.255.1 or the loopback. What am I missing here?

Jon Marshall · ‎11-29-2013

Mohammed

With scenario 1 and only about 500 routes i would try distributing BGP into EIGRP on your ASRs and let the 2960XR see 2 equal cost paths. If you want only one router to be used under normal operations then you can alter the metric of the backup router's EIGRP routes when you redistribute. Because you are redistibuting into EIGRP if you lose any part of the connection your 2960 will know ie.

WAN interface of ASR goes down, no BGP routes received so no routes redistributed into EIGRP so the 2960 switches to the backup router.

LAN interface of ASR goes down no EIGRP routes received by 2960 so again it switches to the backup router.

When the primary ASR gets either it's WAN or LAN interface back up then EIGRP routes with a better metric are again received by the 2960 and it switches back to the main router.

The 2960XR supports 24K of unicast routes using the ipv4 sdm template. You need the IP-Lite feature set.

Note - there is another post going on in this forum where someone else is doing a similiar thing and they are finding that when the primary comes back up and sends EIGRP routes with a better metric the L3 switch (3560) is ignoring them and staying with the backup router. They have to clear the BGP connection on the backup router for it to work. However they are doing mutual redistribution ie. they redistribute BGP into EIGRP but they also redistribute EIGRP into BGP to advertise the internal networks out via BGP. Adn we could always increase the delay to the outgoing interface on the link from the 2960 to the backup router but this shouldn't be needed

I'm assuming you are not proposing to redistribute EIGRP into BGP as well ? (can you confirm). Even if you did, with proper route filtering it should still work and we haven't got to the bottom of why their solution is not working. So what i have proposed should work but it needs testing when the primary fails and when the primary comes back.

It is also worth mentioning that if EIGRP did present problems you could always use OSPF and redistribute the routes as type 1s from the primary router and type 2s from the backup. Type 1 externals are preferred over type 2 externals. But i think the EIGRP solution should work fine.

Jon

View solution in original post

Jon Marshall · ‎11-14-2013

What IP is the DHCP client using ? The router will need to know how to reach this subnet so you can either run a routing protocol between the router and switch or add static route to router eg.

ip route 10.255.255.6

As for the loopback can you post switch config + the output of "sh ip int brief" from the same switch.

Jon

ALIAOF_ · ‎11-14-2013

So the VLAN is 10.139.132.0/23 and SVI is 10.139.132.1

I added the following command on the router but still was not able to ping that SVI from the router:

ip route 10.139.132.0 255.255.254.0 10.255.255.6

Router#show ip int br

Interface IP-Address OK? Method Status Protocol

Embedded-Service-Engine0/0 unassigned YES unset administratively down down

GigabitEthernet0/0 unassigned YES unset administratively down down

GigabitEthernet0/1 unassigned YES unset up up

GigabitEthernet0/2 unassigned YES unset up up

Loopback0 172.16.11.1 YES manual up up

Port-channel1 10.255.255.2 YES manual up up

And here is the switch:

Interface IP-Address OK? Method Status Protocol

Vlan1 10.139.128.1 YES manual up down

Vlan20 10.139.129.1 YES manual up down

Vlan40 10.139.132.1 YES manual up up

Vlan60 10.139.130.1 YES manual up down

FastEthernet0 unassigned YES unset down down

Jon Marshall · ‎11-14-2013

So the loopback is on the router not the switch ?

Also you have a port-channel on the router, so are you using etherchannel to connect the switch to the router ?

And the only interface on the switch that is up is vlan 40 - is this the 10.139.132.x network ?

Can you post full configs of the switch and router - it would be a lot easier.

Jon

ALIAOF_ · ‎11-14-2013

Yes loopback IP is on the router only for now. Yes I'm using etherchannel on the router and the switch to connect.

VLAN40 is the 10.139.132.x network. Here are the configs: NOTE: I do have HSRP config on the router but it is not in used for now.

SWITCH:

interface Port-channel1

no switchport

ip address 10.255.255.5 255.255.255.248

!

interface GigabitEthernet1/0/1

switchport access vlan 40

switchport mode access

!

interface GigabitEthernet1/0/23

no switchport

no ip address

channel-group 1 mode on

!

interface GigabitEthernet1/0/24

no switchport

no ip address

channel-group 1 mode on

!

interface Vlan1

ip address 10.139.128.1 255.255.255.128

no ip redirects

no ip proxy-arp

!

interface Vlan40

ip address 10.139.132.1 255.255.254.0

no ip redirects

no ip proxy-arp

!

ip route 0.0.0.0 0.0.0.0 10.255.255.1

ROUTER:

interface Loopback0

ip address 172.16.11.1 255.255.255.255

no ip redirects

no ip proxy-arp

!

interface Port-channel1

ip address 10.255.255.2 255.255.255.248

standby version 2

standby 1 ip 10.255.255.1

standby 1 priority 250

standby 1 track 1 decrement 6

hold-queue 150 in

!

interface GigabitEthernet0/1

description MEMBER OF PO1

no ip address

duplex auto

speed auto

channel-group 1

!

interface GigabitEthernet0/2

description MEMBER OF PO1

no ip address

duplex auto

speed auto

channel-group 1

!

ip route 10.139.132.0 255.255.254.0 10.255.255.6

ALIAOF_ · ‎11-14-2013

Eh I just realized Switch is 10.255.255.5 and I was adding a route with 10.255.255.6 duh .

I can ping the VLAN40 SVI from the router now and from the laptop I can ping the routers loopback as well as 10.255.255.1 and 10.255.255.5

So with this setup and BGP config on the router anything else that is a best practice that I might be missing?

Jon Marshall · ‎11-14-2013

If HSRP is not in use then you need to change default route on switch to point to the real IP on the router port-channel interface and not the VIP ie.

no ip route 0.0.0.0 0.0.0.0 10.255.255.1

ip route 0.0.0.0 0.0.0.0 10.255.255.2

Also your switch config shows the port-channel interface with an IP address of 10.255.255.5 but you have used 10.255.255.6 on the router. So change the following on the router -

no ip route 10.139.132.0 255.255.254.0 10.255.255.6

ip route 10.139.132.0 255.255.254.0 10.255.255.5

Jon

ALIAOF_ · ‎11-14-2013

Thank you John, yeah that was the issue. About HSRP I just haven't configured the second router yet, its in progress but wanted to get the first one up. So basically second router will connect to the switch the similar way:

Router 2: 10.255.255.3 --> VIP: 10.255.255.1

Interface on the switch for this router will be 10.255.255.6 and then I will have to add the static routes back to the VLAN's on Router2 via 10.255.255.6 sounds right?

Jon Marshall · ‎11-14-2013

I don't think that will work as you want it to. The problem is that for HSRP to work you need L2 adjacency for the HSRP messages. If you connected the routers to the switch with L2 etherchannels then this would work but you are using L3 etherchannels so there is no L2 adjacency.

You have a couple of options -

1) If you want to run HSRP change the L3 etherchannels to L2 etherchannels. The switch definitely supports this but i can't say for sure whether your router does. If it does you can still route the local vlans off the switch but the config would need modifying for the switch to router connections.

2) Don't run HSRP and just have 2 routed links between the switch and the 2 routers. The switch will see equal cost paths via both routers. You can use just default routes on the switch you would simply have 2 default routes one for each router.

Jon

ALIAOF_ · ‎11-14-2013

Ah that totally messes up things a bit for me.

I think the second scenario might work but do I need to put equal paths? Because the first router has a multilink bundle and second one has a backup internet using a GRE tunnel. So what if I put unequal routes. Goal is that all traffic will go out the main router but if something happens to the router or multilink bundle it will go over the second router. But then I think I'll need to also setup "ip sla and tracking" on the switch as well as the tracking on the router will be useless.

Or is it just easier and better to have two equal default routes ?

Now if I trunk the ports I know that makes it all easier, but will that keep all the interVLAN traffic absolutely on the switch including unicast and multicast?

Jon Marshall · ‎11-15-2013

If you make it a trunk link then yes the routers would see broadcasts/multicasts from the clients which are in the switch vlans. However there are ways around this -

1) use a new dedicated vlan for the switch to router connections. This vlan will only have the ports in it that connect to the routers.

2) if you use a trunk link only allow that new vlan on the link or alternatively don't make it a trunk link make it an access port link in only one vlan because those ports only need to be in that one vlan. Your switch will route between that vlan and the client vlans.

If you choose the routed links then yes you could influence which router to use. Note i am assuming that the switch supports floating statics but ihave only used them on a router before. So on the switch you would have -

ip route 0.0.0.0 0.0.0.0

ip route 0.0.0.0 0.0.0.0 250

note the 250 is important. This is an AD and so it won't be used unless the other route goes down.

If the router goes down physically it should go to the backup router but as you say if the WAN interface of the active router fails then this won't get communicated back to the switch so it will not know to use the other router. As you say you can use tracking but again i'm not sure whether you switch would support this, it depends on the switch.

It's difficult to come up with a solution wothout knowing how your routing is going to work so if you could supply a bit more info ie.

1) you say you are advertising BGP routes. Are you doing this from both routers or just the main one ? If only the main one how does the backup router take over if the main router fails

2) are you receviing any routes via BGP from your provider

3) if so what type of routes are they ie. full (very unlikely), partial, default route only. I'm assuming you don't want to pass any routes back to the L3 switch - is that correct ?

Jon

ALIAOF_ · ‎11-20-2013

Hello John sorry for the late reply I was in the field and got pulled into couple of another projects. So for this deployment here is what I ended up doing:

Connected the port channel1 to Router1

Connected the port channel2 to Router2

Setup a default route to the primary routers interface "ip route 0.0.0.0 0.0.0.0

"

Did not add the secondary route yet as adding the second route was causing packet loss with 250 AD. So the second phase would be to implement possibly EIGRP to handle that and then distribute it into BGP for the connection back to the corporate via MPLS. Or hopefully get rid of BGP altogether. About your questions:

1) you say you are advertising BGP routes. Are you doing this from both routers or just the main one ? If only the main one how does the backup router take over if the main router fails

*** Yes they are on both routers, initially they were setup with HSRP configuration and there was tracking to the MPLS PE. But there is no HSRP any longer so like you mentioned looks like this would require tracking on the switch and/or we can configure EIGRP I was thinking of.

2) are you receviing any routes via BGP from your provider

*** I'm new to BGP so bare with me, I believe we are. We peer with their ASN and then we have our own ASNs in the MPLS cloud.

3) if so what type of routes are they ie. full (very unlikely), partial, default route only. I'm assuming you don't want to pass any routes back to the L3 switch - is that correct ?

*** I believe they are full routes which we want to correct because not every site needs to know about every single other location but for now looks like they are full. Yes definitely do not want all those routes passing back to the L3 switch, just what is needed. It was just setup this way.

Now about your first two suggestions:

1) use a new dedicated vlan for the switch to router connections. This vlan will only have the ports in it that connect to the routers.

2) if you use a trunk link only allow that new vlan on the link or alternatively don't make it a trunk link make it an access port link in only one vlan because those ports only need to be in that one vlan. Your switch will route between that vlan and the client vlans.

*** I actually used this scenario at another location on a 2960X that can't do L3 interfaces but can do interVLAN routing. So I connected a port channel to the router and used an Access VLAN so:

Router po1: 10.255.255.9

Default route on the router: ip route 0.0.0.0 0.0.0.0 PE

Also routing to SVI's: 192.168.x.x/21 via 10.255.255.10

Switch po1: 10.255.255.10 (Access port to one VLAN)

Default route on the switch: ip route 0.0.0.0 0.0.0.0 10.255.255.9

What do you think? I believe this discussion with you helped me times 2

Jon Marshall · ‎11-20-2013

Okay, couple of thinsg to cover here.

1) Not sure why adding the second route with AD of 250 should have caused packet loss. Presumably you added these to the switch ? If so when you add the second default with AD of 250 and you do a "sh ip route" you should only see the one default - the main one in the routing table. The one with AD 250 should only come into the routing table if the first one is lost ie. the connection between the switch and the main router is lost. But not as pointed out if the main routers WAN interface goes down - for that you need some sort of tracking.

2) When you say full routes do you mean to the other sites or to the internet. I was assuming you were talking about imternet in which case i doubt it is full routes but if it to other sites on an MPLS cloud then yes full would make sense. Can you clarify

3) How exactly have you connected it up now. You say you used port-channels, are these L3 or L2 port-channels.

4) How many routes are we talking about and what is the model of the L3 switch ?

5) Is it possible to have a separate dedicated connection between the 2 routers that doesn't go via the switch.

Point 4 is important because if the L3 switch is capable and there are not that many routes then a much easier solution then tracking is simply to run a routing protocol such as EIGRP/OSPF between the routers and the switch and resitribute those routes into it. You can then if using EIGRP use an offset-list so the routes via the main router are preferred or use OSPF and redistribute the routes as external type 2 on the main router and type 1 on the secondary router. Type 2 routes do not include the cost to the ASBR (which would be the main router) whereas type 1 routes do include to the cost to the ASBR (the backup router) so the main router would be preferred.

This is all depends though on your switch and how many routes we are talking about.

Jon

ALIAOF_ · ‎11-20-2013

1- Yes when I did "sh ip route" it would only show the route via 10.255.255.1 active. I'll need to look into it a bit more and see why it was causing packet loss. Will have to lab it.

2- Full routes in the MPLS cloud you are correct.

3- First site that I did I used L3 port channel and second site had a 2960X so couldn't do a L3 interface so I ended up creating SVI's and enabling "ip routing" and setup a separate VLAN for connection to the router.

4- Separate connection to the router is that only for EIGRP implementation? Can't I just utilize the existing connection to the switch for EIGRP?

10.255.255.2 10.255.255.1 (Router1)

SWITCH -->

10.255.255.6 10.255.255.5 (Router2)

Jon Marshall · ‎11-20-2013

2) how amny routes though ?

3) but for the example we are talking about you have you used L3 port-channels - yes/no ?

4) I was think of perhaps using an IBGP peering between the 2 routers and then using local preference to influence which path to take but there may be easier solutions.

Jon