Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements

Welcome to Cisco Support Community. We would love to have your feedback.

For an introduction to the new site, click here. And see here for current known issues.

L3 switch for interVLAN and router

I'm working on a project where I need my layer 3 switch to handle all the interVLAN routing and DHCP

And all the internet and traffic to the corporate office needs to hit the router.

I have the router setup as 10.255.255.1 --> then a port on the switch as 10.255.255.6

Next I have a route on the switch, "ip route 0.0.0.0 0.0.0.0 10.255.255.1" and I did enable routing on the switch by "ip routing"

I have a loopback on the switch 172.16.11.1

Router has BGP setup to advertise the networks

I setup one port on a VLAN got a DHCP IP, but I can't ping the router 10.255.255.1 or the loopback.  What am I missing here?

1 ACCEPTED SOLUTION

Accepted Solutions
Hall of Fame Super Blue

Re: L3 switch for interVLAN and router

Mohammed

With scenario 1 and only about 500 routes i would try distributing BGP into EIGRP on your ASRs and let the 2960XR see 2 equal cost paths. If you want only one router to be used under normal operations then you can alter the metric of the backup router's EIGRP routes when you redistribute.  Because you are redistibuting into EIGRP if you lose any part of the connection your 2960 will know ie.

WAN interface of ASR goes down, no BGP routes received so no routes redistributed into EIGRP so the 2960 switches to the backup router.

LAN interface of ASR goes down no EIGRP routes received by 2960 so again it switches to the backup router. 

When the primary ASR gets either it's WAN or LAN interface back up then EIGRP routes with a better metric are again received by the 2960 and it switches back to the main router.

The 2960XR supports 24K of unicast routes using the ipv4 sdm template. You need the IP-Lite feature set.

Note - there is another post going on in this forum where someone else is doing a similiar thing and they are finding that when the primary comes back up and sends EIGRP routes with a better metric the L3 switch (3560) is ignoring them and staying with the backup router. They have to clear the BGP connection on the backup router for it to work. However they are doing mutual redistribution ie. they redistribute BGP into EIGRP but they also redistribute EIGRP into BGP to advertise the internal networks out via BGP.  Adn we could always increase the delay to the outgoing interface on the link from the 2960 to the backup router but this shouldn't be needed

I'm assuming you are not proposing to redistribute EIGRP into BGP as well ? (can you confirm).  Even if you did, with proper route filtering it should still work and we haven't got to the bottom of why their solution is not working. So what i have proposed should work but it needs testing when the primary fails and when the primary comes back.

It is also worth mentioning that if EIGRP did present problems you could always use OSPF and redistribute the routes as type 1s from the primary router and type 2s from the backup. Type 1 externals are preferred over type 2 externals. But i think the EIGRP solution should work fine.

Jon

22 REPLIES
Hall of Fame Super Blue

L3 switch for interVLAN and router

What IP is the DHCP client using ? The router will need to know how to reach this subnet so you can either run a routing protocol between the router and switch or add static route to router eg.

ip route 10.255.255.6

As for the loopback can you post switch config + the output of "sh ip int brief" from the same switch.

Jon

L3 switch for interVLAN and router

So the VLAN is 10.139.132.0/23 and SVI is 10.139.132.1

I added the following command on the router but still was not able to ping that SVI from the router:

ip route 10.139.132.0 255.255.254.0 10.255.255.6

Router#show ip int br

Interface                  IP-Address      OK? Method Status                Protocol

Embedded-Service-Engine0/0 unassigned      YES unset  administratively down down

GigabitEthernet0/0         unassigned      YES unset  administratively down down

GigabitEthernet0/1         unassigned      YES unset  up                    up

GigabitEthernet0/2         unassigned      YES unset  up                    up

Loopback0                  172.16.11.1     YES manual up                    up

Port-channel1              10.255.255.2    YES manual up                    up

And here is the switch:

Interface              IP-Address      OK? Method Status                Protocol

Vlan1                  10.139.128.1    YES manual up                    down

Vlan20                 10.139.129.1    YES manual up                    down

Vlan40                 10.139.132.1    YES manual up                    up

Vlan60                 10.139.130.1    YES manual up                    down

FastEthernet0          unassigned      YES unset  down                  down

Hall of Fame Super Blue

L3 switch for interVLAN and router

So the loopback is on the router not the switch ?

Also you have a port-channel on the router, so are you using etherchannel to connect the switch to the router ?

And the only interface on the switch that is up is vlan 40 - is this the 10.139.132.x network ?

Can you post full configs of the switch and router - it would be a lot easier.

Jon

L3 switch for interVLAN and router

Yes loopback IP is on the router only for now.  Yes I'm using etherchannel on the router and the switch to connect.

VLAN40 is the 10.139.132.x network.  Here are the configs: NOTE:  I do have HSRP config on the router but it is not in used for now.

SWITCH:

interface Port-channel1

no switchport

ip address 10.255.255.5 255.255.255.248

!

interface GigabitEthernet1/0/1

switchport access vlan 40

switchport mode access

!

interface GigabitEthernet1/0/23

no switchport

no ip address

channel-group 1 mode on

!

interface GigabitEthernet1/0/24

no switchport

no ip address

channel-group 1 mode on

!

interface Vlan1

ip address 10.139.128.1 255.255.255.128

no ip redirects

no ip proxy-arp

!

interface Vlan40

ip address 10.139.132.1 255.255.254.0

no ip redirects

no ip proxy-arp

!

ip route 0.0.0.0 0.0.0.0 10.255.255.1

ROUTER:

interface Loopback0

ip address 172.16.11.1 255.255.255.255

no ip redirects

no ip proxy-arp

!

interface Port-channel1

ip address 10.255.255.2 255.255.255.248

standby version 2

standby 1 ip 10.255.255.1

standby 1 priority 250

standby 1 track 1 decrement 6

hold-queue 150 in

!

interface GigabitEthernet0/1

description MEMBER OF PO1

no ip address

duplex auto

speed auto

channel-group 1

!

interface GigabitEthernet0/2

description MEMBER OF PO1

no ip address

duplex auto

speed auto

channel-group 1

!

ip route 10.139.132.0 255.255.254.0 10.255.255.6

L3 switch for interVLAN and router

Eh I just realized Switch is 10.255.255.5 and I was adding a route with 10.255.255.6 duh .

I can ping the VLAN40 SVI from the router now and from the laptop I can ping the routers loopback as well as 10.255.255.1 and 10.255.255.5

So with this setup and BGP config on the router anything else that is a best practice that I might be missing?

Hall of Fame Super Blue

Re: L3 switch for interVLAN and router

If HSRP is not in use then you need to change default route on switch to point to the real IP on the router port-channel interface and not the VIP ie.

no ip route 0.0.0.0 0.0.0.0 10.255.255.1

ip route 0.0.0.0 0.0.0.0 10.255.255.2

Also your switch config shows the port-channel interface with an IP address of 10.255.255.5 but you have used 10.255.255.6 on the router. So change the following on the router -

no ip route 10.139.132.0 255.255.254.0 10.255.255.6

ip route 10.139.132.0 255.255.254.0 10.255.255.5

Jon

L3 switch for interVLAN and router

Thank you John, yeah that was the issue.  About HSRP I just haven't configured the second router yet, its in progress but wanted to get the first one up.  So basically second router will connect to the switch the similar way:

Router 2: 10.255.255.3 --> VIP: 10.255.255.1

Interface on the switch for this router will be 10.255.255.6 and then I will have to add the static routes back to the VLAN's on Router2 via 10.255.255.6 sounds right?

Hall of Fame Super Blue

L3 switch for interVLAN and router

I don't think that will work as you want it to. The problem is that for HSRP to work you need L2 adjacency for the HSRP messages. If you connected the routers to the switch with L2 etherchannels then this would work but you are using L3 etherchannels so there is no L2 adjacency.

You have a couple of options -

1) If you want to run HSRP change the L3 etherchannels to L2 etherchannels. The switch definitely supports this but i can't say for sure whether your router does. If it does you can still route the local vlans off the switch but the config would need modifying for the switch to router connections.

2)  Don't run HSRP and just have 2 routed links between the switch and the 2 routers. The switch will see equal cost paths via both routers. You can use just default routes on the switch you would simply have 2 default routes one for each router.

Jon

Re: L3 switch for interVLAN and router

Ah that totally messes up things a bit for me.

I think the second scenario might work but do I need to put equal paths?  Because the first router has a multilink bundle and second one has a backup internet using a GRE tunnel.  So what if I put unequal routes.  Goal is that all traffic will go out the main router but if something happens to the router or multilink bundle it will go over the second router.  But then I think I'll need to also setup "ip sla and tracking" on the switch as well as the tracking on the router will be useless.

Or is it just easier and better to have two equal default routes ?

Now if I trunk the ports I know that makes it all easier, but will that keep all the interVLAN traffic absolutely on the switch including unicast and multicast?

Hall of Fame Super Blue

Re: L3 switch for interVLAN and router

If you make it a trunk link then yes the routers would see broadcasts/multicasts from the clients which are in the switch vlans. However there are ways around this -

1) use a new dedicated vlan for the switch to router connections. This vlan will only have the ports in it that connect to the routers.

2) if you use a trunk link only allow that new vlan on the link or alternatively don't make it a trunk link make it an access port link in only one vlan because those ports only need to be in that one vlan. Your switch will route between that vlan and the client vlans.

If you choose the routed links then yes you could influence which router to use. Note i am assuming that the switch supports floating statics but ihave only used them on a router before. So on the switch you would have -

ip route 0.0.0.0 0.0.0.0

ip route 0.0.0.0 0.0.0.0 250

note the 250 is important. This is an AD and so it won't be used unless the other route goes down.

If the router goes down physically it should go to the backup router but as you say if the WAN interface of the active router fails then this won't get communicated back to the switch so it will not know to use the other router. As you say you can use tracking but again i'm not sure whether you switch would support this, it depends on the switch. 

It's difficult to come up with a solution wothout knowing how your routing is going to work so if you could supply a bit more info ie.

1) you say you are advertising BGP routes. Are you doing this from both routers or just the main one ? If only the main one how does the backup router take over if the main router fails

2) are you receviing any routes via BGP from your provider

3) if so what type of routes are they ie. full (very unlikely), partial, default route only. I'm assuming you don't want to pass any routes back to the L3 switch - is that correct ?

Jon

Re: L3 switch for interVLAN and router

Hello John sorry for the late reply I was in the field and got pulled into couple of another projects.  So for this deployment here is what I ended up doing:

Connected the port channel1 to Router1

Connected the port channel2 to Router2

Setup a default route to the primary routers interface "ip route 0.0.0.0 0.0.0.0

"

Did not add the secondary route yet as adding the second route was causing packet loss with 250 AD.  So the second phase would be to implement possibly EIGRP to handle that and then distribute it into BGP for the connection back to the corporate via MPLS.  Or hopefully get rid of BGP altogether.  About your questions:

1) you say you are advertising BGP routes. Are you doing this from both routers or just the main one ? If only the main one how does the backup router take over if the main router fails

*** Yes they are on both routers, initially they were setup with HSRP configuration and there was tracking to the MPLS PE.  But there is no HSRP any longer so like you mentioned looks like this would require tracking on the switch and/or we can configure EIGRP I was thinking of.

2) are you receviing any routes via BGP from your provider

*** I'm new to BGP so bare with me, I believe we are. We peer with their ASN and then we have our own ASNs in the MPLS cloud.

3) if so what type of routes are they ie. full (very unlikely), partial, default route only. I'm assuming you don't want to pass any routes back to the L3 switch - is that correct ?

*** I believe they are full routes which we want to correct because not every site needs to know about every single other location but for now looks like they are full.  Yes definitely do not want all those routes passing back to the L3 switch, just what is needed.  It was just setup this way.

Now about your first two suggestions:

1) use a new dedicated vlan for the switch to router connections. This vlan will only have the ports in it that connect to the routers.

2) if you use a trunk link only allow that new vlan on the link or alternatively don't make it a trunk link make it an access port link in only one vlan because those ports only need to be in that one vlan. Your switch will route between that vlan and the client vlans.

*** I actually used this scenario at another location on a 2960X that can't do L3 interfaces but can do interVLAN routing.  So I connected a port channel to the router and used an Access VLAN so:

Router po1: 10.255.255.9

Default route on the router: ip route 0.0.0.0 0.0.0.0 PE

Also routing to SVI's: 192.168.x.x/21 via 10.255.255.10

Switch po1: 10.255.255.10 (Access port to one VLAN)

Default route on the switch: ip route 0.0.0.0 0.0.0.0 10.255.255.9

What do you think?  I believe this discussion with you helped me times 2

Hall of Fame Super Blue

Re: L3 switch for interVLAN and router

Okay, couple of thinsg to cover here.

1) Not sure why adding the second route with AD of 250 should have caused packet loss. Presumably you added these to the switch ?  If so when you add the second default with AD of 250 and you do a "sh ip route" you should only see the one default - the main one in the routing table. The one with AD 250 should only come into the routing table if the first one is lost ie. the connection between the switch and the main router is lost. But not as pointed out if the main routers WAN interface goes down - for that you need some sort of tracking.

2) When you say full routes do you mean to the other sites or to the internet. I was assuming you were talking about imternet in which case i doubt it is full routes but if it to other sites on an MPLS cloud then yes full would make sense. Can you clarify

3) How exactly have you connected it up now. You say you used port-channels, are these L3 or L2 port-channels.

4)  How many routes are we talking about and what is the model of the L3 switch ?

5) Is it possible to have a separate dedicated connection between the 2 routers that doesn't go via the switch.

Point 4 is important because if the L3 switch is capable and there are not that many routes then a much easier solution then tracking is simply to run a routing protocol such as EIGRP/OSPF between the routers and the switch and resitribute those routes into it. You can then if using EIGRP use an offset-list so the routes via the main router are preferred or use OSPF and redistribute the routes as external type 2 on the main router and type 1 on the secondary router. Type 2 routes do not include the cost to the ASBR (which would be the main router) whereas type 1 routes do include to the cost to the ASBR (the backup router) so the main router would be preferred.

This is all depends though on your switch and how many routes we are talking about.

Jon

Re: L3 switch for interVLAN and router

1- Yes when I did "sh ip route" it would only show the route via 10.255.255.1 active.  I'll need to look into it a bit more and see why it was causing packet loss.  Will have to lab it.

2- Full routes in the MPLS cloud you are correct.

3- First site that I did I used L3 port channel and second site had a 2960X so couldn't do a L3 interface so I ended up creating SVI's and enabling "ip routing" and setup a separate VLAN for connection to the router.

4- Separate connection to the router is that only for EIGRP implementation?  Can't I just utilize the existing connection to the switch for EIGRP?

10.255.255.2          10.255.255.1 (Router1)

SWITCH          -->

10.255.255.6          10.255.255.5 (Router2)

Hall of Fame Super Blue

L3 switch for interVLAN and router

2) how amny routes though ?

3) but for the example we are talking about you have you used L3 port-channels - yes/no ?

4) I was think of perhaps using an IBGP peering between the 2 routers and then using local preference to influence which path to take but there may be easier solutions.

Jon

Re: L3 switch for interVLAN and router

2) how amny routes though ?

*** I'm assuming 536 ?

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd

y.y.y.y     4 65113   52827   51111     1339    0    0 5d20h                               2

x.x.x.x    4 13979   51750   51232     1339    0    0 5d22h                             536

3) but for the example we are talking about you have you used L3 port-channels - yes/no ?

*** Yes first site I used a L3 port channel

4)  I was think of perhaps using an IBGP peering between the 2 routers and  then using local preference to influence which path to take but there  may be easier solutions.

*** Gotcha, with the HSRP setup without a L3 switch that is how it is currently setup i.e; ibgp peering between the 2 routers.

Hall of Fame Super Blue

L3 switch for interVLAN and router

Okay, lets take a step back here.  You initially were going to use HSRP between the routers and the switch but because you configured them as L3 HSRP wouldn't work. Now i initially stated that i didn't know if your routers supported L2 etherchannels (i suspect not unless you have a switching module). Do you know if your routers support L2 etherchannels ?

If so why not go that route with your setup ? You can use IBGP peering and how your failover works depends on whether you have a dedicated interconnect between the routers. I can explain fully how it would work with this if you need but it sounds like you already have this setup in use.

The other alternative as discussed is to use L3 links between the routers and the switch. 536 routes is not a huge number so you could redistribute these into either EIGRP or OSPF influencing the metrics as previously discussed.

But you still have not told me what switch it is ?  Also what IOS version and feature set (feature set is important as it may not run EIGRP/OSPF).

To be honest if the L3 switch supported EIGRP/OSPF and could handle those routes i think that is the cleanest solution because a failure on either the LAN or WAN links is automatically taken care of. But i wonder why you are not using your existing HSRP/IBGP solution or is it that it is not that reliable ?

Jon

Hall of Fame Super Blue

L3 switch for interVLAN and router

One final point. This is where if you can summarize your IPs from each site it really helps because then the routing table on your L3 switch would be greatly reduced.

All of what i have written assumes that you are not receiving a lot of internet routes.

Jon

Re: L3 switch for interVLAN and router

Okay, lets take a step back here.  You initially were going to use HSRP between the routers and the switch but because you configured them as L3 HSRP wouldn't work.

Now i initially stated that i didn't know if your routers supported L2 etherchannels (i suspect not unless you have a switching module). Do you know if your routers support L2 etherchannels ?

If so why not go that route with your setup ? You can use IBGP peering and how your failover works depends on whether you have a dedicated interconnect between the routers. I can explain fully how it would work with this if you need but it sounds like you already have this setup in use.

The other alternative as discussed is to use L3 links between the routers and the switch. 536 routes is not a huge number so you could redistribute these into either EIGRP or OSPF influencing the metrics as previously discussed.

*** Ok so the routers can support L3 and L2 etherchannels but what I used L3 etherchannel in the first scenario to keep any broadcast and multicast traffic away from hitting the router and just staying on the switch (2960XR)

*** In the second scenario I was not able to to do L3 etherchannel because the switch (2960X) does not support it, however it does support basic "IP Routing".  So I created SVI's on the switch for all the VLAN's and enabled IP Routing.  Then created a separate VLAN on the switch to connect the rotuer instead of doing a trunk port.  And connected an access port to the router that is part of this vlan.  For example:

SWITCH CONFIG:

conf t

!

interface vlan 999

ip address 10.10.10.2 255.255.255.252

!

vlan 999

!

interface Port-channel1

description 47/48-Router1

switchport access vlan 999

switchport mode access

!

interface range GigabitEthernet1/0/47-48

description Router 1

switchport access vlan 999

switchport mode access

srr-queue bandwidth share 10 10 60 20

queue-set 2

priority-queue out

mls qos trust cos

auto qos voip trust

channel-group 1 mode on

ROUTER CONFIG:

interface Port-channel1

ip address 10.10.10.1 255.255.255.252

!

interface GigabitEthernet0/1

description MEMBER OF PO1

no ip address

duplex auto

speed auto

channel-group 1

!

interface GigabitEthernet0/2

description MEMBER OF PO1

no ip address

duplex auto

speed auto

channel-group 1

Now I guess in this scenario I might be able to get HSRP working.

But you still have not told me what switch it is ?  Also what IOS version and feature set (feature set is important as it may not run EIGRP/OSPF).

*** SCENARIO 1 with L3 etherchannel: 2960XR (c2960x-universalk9-mz.150-2.EX1) - This supports EIGRP

*** SCENARIO 2 with L2 etherchannel: 2960X (c2960x-universalk9-mz.150-2.EX3) - Only supports basic InterVLAN Routing

To be honest if the L3 switch supported EIGRP/OSPF and could handle those routes i think that is the cleanest solution because a failure on either the LAN or WAN links is automatically taken care of. But i wonder why you are not using your existing HSRP/IBGP solution or is it that it is not that reliable ?

*** We are currently using it for the old setup in place but going on forward we will be utilizing the above mentioned scenarios instead of router on the stick situation. 

Hall of Fame Super Blue

Re: L3 switch for interVLAN and router

Mohammed

With scenario 1 and only about 500 routes i would try distributing BGP into EIGRP on your ASRs and let the 2960XR see 2 equal cost paths. If you want only one router to be used under normal operations then you can alter the metric of the backup router's EIGRP routes when you redistribute.  Because you are redistibuting into EIGRP if you lose any part of the connection your 2960 will know ie.

WAN interface of ASR goes down, no BGP routes received so no routes redistributed into EIGRP so the 2960 switches to the backup router.

LAN interface of ASR goes down no EIGRP routes received by 2960 so again it switches to the backup router. 

When the primary ASR gets either it's WAN or LAN interface back up then EIGRP routes with a better metric are again received by the 2960 and it switches back to the main router.

The 2960XR supports 24K of unicast routes using the ipv4 sdm template. You need the IP-Lite feature set.

Note - there is another post going on in this forum where someone else is doing a similiar thing and they are finding that when the primary comes back up and sends EIGRP routes with a better metric the L3 switch (3560) is ignoring them and staying with the backup router. They have to clear the BGP connection on the backup router for it to work. However they are doing mutual redistribution ie. they redistribute BGP into EIGRP but they also redistribute EIGRP into BGP to advertise the internal networks out via BGP.  Adn we could always increase the delay to the outgoing interface on the link from the 2960 to the backup router but this shouldn't be needed

I'm assuming you are not proposing to redistribute EIGRP into BGP as well ? (can you confirm).  Even if you did, with proper route filtering it should still work and we haven't got to the bottom of why their solution is not working. So what i have proposed should work but it needs testing when the primary fails and when the primary comes back.

It is also worth mentioning that if EIGRP did present problems you could always use OSPF and redistribute the routes as type 1s from the primary router and type 2s from the backup. Type 1 externals are preferred over type 2 externals. But i think the EIGRP solution should work fine.

Jon

Re: L3 switch for interVLAN and router

Thank you Jon for the excellent reply.  We will definitely have to do the un equal paths for the backup router. 

In my scenario I don't think I'll have to do mutual distribution.  ASR has BGP on it and I can distribute it into EIGRP.  EIGRP is configured between the routers and the 2960XR.  For the internal networks I will keep the way it is now i.e; use BGP to advertise them out.  So on the Router I have a static route pointing back to the L3 interface of the switch for the internal networks and then matching "network" statements under BGP.

I concur and I'm pretty confident that EIGRP should work I have a bit more experience with EIGRP vs OSPF and I like it better than OSPF too so most likely will go with that.

Hall of Fame Super Blue

Re: L3 switch for interVLAN and router

Just in case it got lost in the last post -

how are you proposing to advertise the internal subnets via BGP ie. network statements under BGP or redistributing the IGP into BGP ?

How many internal subnets are there and can they be summarised ? If so network statements are definitely the way to go.

Finally i'm assuming you don't receive a default route from BGP on both routers ?

Jon

Re: L3 switch for interVLAN and router

how are you proposing to advertise the internal subnets via BGP ie.  network statements under BGP or redistributing the IGP into BGP ?

*** Network statements under the BGP config and then static statements pointing to the L3 Switch interface

How many internal subnets are there and can they be summarised ? If so network statements are definitely the way to go.

*** Usually 4 or 5 so here is how it looks:

router bgp xxxxx

network 192.168.1.0 255.255.255.0

network 192.168.2.0 255.255.255.0

network 192.168.3.0 255.255.255.0

network 172.16.15.0 255.255.255.224

.......

Then static routes

ip route 192.168.1.0 255.255.255.0 10.255.255.2

ip route 192.168.2.0 255.255.255.0 10.255.255.2

and so on.

840
Views
25
Helpful
22
Replies
CreatePlease login to create content