Need Advice/Assistance w/ Resegmenting a Network

Unanswered Question
Oct 21st, 2009
User Badges:

I'm a new hire at my place of employment, in which I work at a branch office. Upon arriving, I was told that the network wasn't exactly running up to par with occasional hitches in the network. After doing some investigation, I found the that all the Cisco networking equipment is using VLAN 1 with multiple physical links between everything. Eventually, the connections go back to a C3640 router.


The C3640 router uses EIGRP to connect to other routers outside our location in which we also get our Internet connection. The main office has configured the router and I'm mainly responsible for everything internal to our branch. We have 4 subnets that we use internally:


176 - Datacenter


177 - Device Management


216 - Tech/Servers


229 - Users


The physical link to the router is defined as follows:


interface FastEthernet0/0

description LAN segment

ip address 172.24.216.254 255.255.255.128 secondary

ip address 172.24.229.254 255.255.255.128 secondary


ip address 172.24.216.254 255.255.255.0 secondary


ip address 172.24.217.254 255.255.255.0 secondary


I believe the problem is that all traffic, especially heavy traffic from the datacenter, has to go through this old router before reaching the other subnets. We have several gigabit capable switches here capable of routing. My idea is to move all the subnets to the C3750 & C4948 switches and allow the routing to be processed on those two switches. That leaves the C3640 to be only used for connections outside out network, such as to corporate or the Internet.


I want to resegment the network to 4 individual VLANs. I have a C3750 that I would like to be the main router for subnets 229 & 177 but would also provide a hub-spoke connectivity to all the other switches in the office. I also have a C4948 switch that I would like to be the router for subnets 176 & 216. If I am to do this, could I potentially just define everything via static routes or would you suggest using a routing protocol?


Any help would be appreciated.

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 4.9 (7 ratings)
Loading.
Jon Marshall Wed, 10/21/2009 - 05:27
User Badges:
  • Super Blue, 32500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

Your approach is a good one but i'm not clear why you want to route the internal subnets using 2 different switches.


Personally i would choose the 3750 and route all vlans off the switch. Then have a L3 routed connected to the 3640. This way you could indeed if you wanted use static routes and it would be relatively straightforward ie.


L3 connection between 3750 and 3640 -


3640

====


int fa0/0

ip address 192.168.5.1 255.255.255.252


3750

====


int gi0/1

ip address 192.168.5.2 255.255.255.252


then routing -


on the 3750


ip route 0.0.0.0 0.0.0.0 192.168.5.1


on the 3640


ip route 192.168.5.2


for each of the internal subnets being routed off the 3750.


You could just as easily run a routing protocol such as EIGRP although that does depend on the feature set in use on your 3750.


Jon

hyounkim80 Wed, 10/21/2009 - 05:37
User Badges:

Thanks Jon. Your reply was extremely helpful and I rated it accordingly.


The reason I wanted to split Datacenter traffic from the C3750 to another switch is because I didn't want the C3750 to get overloaded by the everyday traffic from the servers. I thought by isolating the traffic on its own switch, it could free up resources on the C3750.


Of course, if you think the C3750 can handle that with ease, then I can go ahead and just put all the VLANs on the C3750.


The C3750 is a C3750G-12S-E switch running c3750-ipservices-mz.122-52.SE.


With your example, it seems I'll just be using static routes. In your opinion, would it be necessary to use a routing protocol when only 2 "routers" are connected to each other?


Also is there anything else I need to do specifically in regards to routes and the interfaces? Or is your example all that is needed (for example do I need to put the interfaces on switchport mode access, switchport access vlan )?

Jon Marshall Wed, 10/21/2009 - 05:48
User Badges:
  • Super Blue, 32500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

"The reason I wanted to split Datacenter traffic from the C3750 to another switch is because I didn't want the C3750 to get overloaded by the everyday traffic from the servers. I thought by isolating the traffic on its own switch, it could free up resources on the C3750."


So you have servers in your branch office that talk to the data centre ??. If so be aware that it will very probably be the WAN connection and not the internal switch that becomes the bottleneck. If there was a lot of chatter between servers within your branch then yes i would connect them to the 4948 but if they were all in the same vlan then they could still route off the 3750.


Perhaps you clarify exactly how the setup works ie.


Do you know how much traffic is sent between servers


Is this traffic local to the servers within your branch or remote


Are the servers in your branch all in the same vlan


Where is the location of your data centre or is this just a label ?


"With your example, it seems I'll just be using static routes. In your opinion, would it be necessary to use a routing protocol when only 2 "routers" are connected to each other?"


Not necessary no. And with the number of vlans you have internally it is easily manageable with static routing. Dynamic routing would be more useful if you had a lot more


i) vlans

ii) routers


or you had redundant links and you let the routing protocol work out the best path.


One other option i didn't mention was that you could use the 4948 with the 3750 and


i) run HSRP between them

ii) dual connect your servers for NIC redundancy, but still need to understand server situation a little better


What i provided in config was for routing purposes. You would need to -


1) configure trunk links to other switches

2) configure STP root for vlans on 3750

3) as you say configure the ports with end devices into their respective vlans.


Jon

hyounkim80 Wed, 10/21/2009 - 06:22
User Badges:

Oh no... sorry for not being clear. All the subnets I mentioned earlier exist only in my office. The datacenter is in my office as well. All networking equipment is within walking distance and use MM fiber to connect to the C3750.


In the Datacenter, the majority of our servers use 2+ gigabit interfaces bonded by etherchannel. The servers themselves are various file servers, database servers, that are backed up daily to a backup server. We also utilize a fiber channel SAN setup. In addition, we also have a DR setup that is identical to the production servers. I just assumed that putting all those servers on the C3750 would eat up the link utilization and processor utilization.


A few weeks ago, every subnet was on VLAN 1 going to the C3640 router. I split all the subnets into their own VLAN, but they still go to the C3640. That's why I was looking for advice on how to migrate it off the C3640 to the C3750/C4948. But with your opinion & recommendation, I think I'll just use the C3750 as a router for the internal network and then the C3640 as the router to the outside.


I have set up HSRP before on 2 C3750s, so I could do it for the C4948 & the C3750 but... the C3750 is strictly fiber only (the C3750G-12S model), so I couldn't connect the servers redundantly. With that in mind, the HSRP would only be failover solution in case the C3750 goes down. But in that event, it wouldn't really make sense to do that because the C4948 doesn't have the capacity for being an uplink switch for the other switches in the office due to the lack of fiber ports.

Jon Marshall Wed, 10/21/2009 - 07:04
User Badges:
  • Super Blue, 32500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

Okay, thanks for clearing that up. I was a little confused because you said it was a branch office.


I would connect the servers to the 4948 switch but still route the server vlan(s) off the 3750. If you find that the utilization on the 3750 is too high you can migrate the L3 vlan interfaces for the servers onto the 4948 at a later date.


Agree with what you say about HSRP and the switches. Wouldn't make a lot of sense if you can't uplink the access-layer switches to the 4948.


Jon

hyounkim80 Wed, 10/21/2009 - 07:34
User Badges:

Thanks for all your help Jon.


For clarification purposes, on the C3640, you give the example of:

=========

int fa0/0

ip address 192.168.5.1 255.255.255.252


ip route 192.168.5.2

=========


When I log into my C3640, I only see the following available:

C3640(config)#ip route ?

A.B.C.D Destination prefix

profile Enable IP routing table profile

static Allow static routes

vrf Configure static route for a VPN Routing/Forwarding instance


I don't see an option to put a VLAN # in the ip route command, unless you're talking about the network itself. For example: 'ip route 172.24.176.0 255.255.255.0 192.168.5.2'. If this is the case, would specify the static route for each subnet on the C3750? I have 4 networks internally (172.24.176.0/24, 172.24.177.0/24, 172.24.216.0/28, 172.24.229.0/28).


Also, do I have to specify a VLAN interface on the C3750 for the link between the C3640 & the C3750? Or do I just strictly put what you mentioned in your example.


So if what I mentioned is correct, I am assuming that I have the following lines in my configuration:


=========

C3640

=========


int fa0/0

ip address 192.168.5.1 255.255.255.252

!

ip route 172.24.176.0 255.255.255.0 192.168.5.2

ip route 172.24.177.0 255.255.255.0 192.168.5.2

ip route 172.24.229.0 255.255.255.128 192.168.5.2

ip route 172.24.216.0 255.255.255.128 192.168.5.2


=========

C3750

=========


!

int gi0/1

ip address 192.168.5.2 255.255.255.252

!

ip route 0.0.0.0 0.0.0.0 192.168.5.1

Jon Marshall Wed, 10/21/2009 - 07:46
User Badges:
  • Super Blue, 32500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

"Thanks for all your help Jon."


No problem, glad to be of help.


"I don't see an option to put a VLAN # in the ip route command, unless you're talking about the network itself. For example: 'ip route 172.24.176.0 255.255.255.0 192.168.5.2'. If this is the case, would specify the static route for each subnet on the C3750?"


Correct, i mean you put the IP subnet and the subnet mask in the route statements on the 3640. Yes you would specify a route for each subnet on the 3640.


"Also, do I have to specify a VLAN interface on the C3750 for the link between the C3640 & the C3750? Or do I just strictly put what you mentioned in your example."


On the 3640 just remove all the existing IP addressing and add the new one.


On the 3750 you would do


int gi0/1

no switchport

ip address 192.168.5.2 255.255.255.252


also don't forget to enable ip routing on the 3750 ie.


3750(config)# ip routing


Your config is spot on with the addition of the "no switchport" under the gi0/1 interface.


Jon



hyounkim80 Wed, 10/21/2009 - 12:10
User Badges:

Hey Jon, I don't know if you're available, but I plan on doing this change tonight. I'm 99.9% certain this will go without a hitch.


Basically what I plan on doing is logging into the C3640 and performing a "show ip arp" so I can get the IPs of every machine/device we have on the network matched to their respective MAC addresses. Then I'll trace the MAC addresses across all our switches so when I do the change, I can just change the switchport VLAN from 1 to the new VLAN.


One thing to note though is that I notice that the router interface where all the subnets exist also has one other network on it that is not a part of our internal branch network. I believe the IP subnet belongs to devices the main office. I mentioned that our router is using EIGRP and is interconnected with other routers within the organization. So here is the config:


interface FastEthernet0/0

description LAN segment

ip address 10.10.1.254 255.255.255.0 secondary

ip address 10.100.131.254 255.255.255.0 secondary

ip address 172.16.216.254 255.255.255.128 secondary

ip address 172.16.229.254 255.255.255.128

ip address 172.16.176.253 255.255.255.0

ip address 172.16.177.253 255.255.255.0

ip access-group Virus in

ip helper-address 172.16.213.162

ip helper-address 172.16.208.34

ip accounting access-violations

ip flow ingress

ip pim dense-mode

load-interval 30

duplex auto

speed auto


Is it ok for me to leave the 10.xxx.xxx.xxx network intact on that interface and just add what we discussed earlier (the 192.xxx.xxx.xxx address) between the C3750 & the C3640? I shouldn't have any problems right?

Jon Marshall Wed, 10/21/2009 - 12:23
User Badges:
  • Super Blue, 32500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

"Hey Jon, I don't know if you're available, but I plan on doing this change tonight. I'm 99.9% certain this will go without a hitch"


I'm in UK so it's already evening here. I may be around as i have bad toothache at the moment so not sleeping too well but can't guarantee it.


The 10.x.x.x network, yes you can leave it on the interface but you should find out for sure what it is being used for before making the change. Doesn't sound right that you have this on your interface and yet the subnet belongs to a remote site.


Also a couple of other things -


1) you will need to move the ip helper-addresses to the L3 vlan interfaces you create on the 3750.


2) You have "ip pim dense-mode" on this interface. Do you know if that is actually in use because you will also need to enable this on the 3750 and then you do need the IP Services image to run multicast. If multicast is in use you will also need to migrate this commmand to each L3 vlan interface.


If you are comfortable with all the additional things above then good luck with the change tonight but -


if you are not it may well be worth postponing as this would also give you the time to track down all the mac-address to IP mappings and which switches they are on. This could be done during production hours and then you can have a text file with the correct config per switch which could just be cut and pasted in.


Personally i would do all this mac to IP mapping in advance of the change which will give you more time to troubleshoot other issues should it not go as planned.


Jon

hyounkim80 Fri, 10/23/2009 - 07:03
User Badges:

Hey Jon,


The network changes ended up being a failure. I configured both devices exactly how we discussed and the C3640 & C3750 were aware of the new routes. The routes for each network were displayed as directly connected and all devices were pingable. However, all subnets on the C3750 were no longer able to ping/connect to the main office or the Internet. Essentially, they were unable to get outside of our office network.


One thing to note is that the C3640 is connected to the next router (outside our network) via EIGRP. I'm not too keen on EIGRP, but here's how it's defined on the C3640 router (with IPs changed of course):

router eigrp 20

network 10.1.65.0 0.0.0.255

network 10.1.86.0 0.0.0.255

network 172.16.0.0

distribute-list Route_Distribution out FastEthernet0/1

no auto-summary


I know I'm missing something stupid… Because when I rolled back my changes and took out the lines I specifically mentioned above, everything goes back to normal. The only thing that changed with my changes is that the default router for our networks changes from the C3640 to the C3750. I'm 99.9% sure that something is wrong specifically with the EIGRP configuration. The C3640 is aware of the statically routed networks, but it's not being redistributed through EIGRP. That distribute-list line is an access list with a bunch of subnets (ours included) defined on it, so I'm pretty sure it has nothing to do with access.


What do you think?

Jon Marshall Fri, 10/23/2009 - 10:28
User Badges:
  • Super Blue, 32500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

Which interface connects to the main office on your WAN router ?


The reason you couldn't connect to the main office is because the 172.x.x.x networks have moved to the 3750 so your 3640 no longer advertises them to the main office.


Apologies, should have taken this into account.


There are 2 ways around this although i still need to know a bit more info about the distribute list.


1) On the 3640 under the EIGRP config add the following line -


router eigrp 20

redistribute static


this will make your EIGRP process tell the main office about the 172.x.x.x subnets that you have added static routes for on the 3640


2) Run EIGRP between the 3640 and the 3750. Again as i said this does depend on the image on your 3750. Perhaps best to try 1) first.


Good luck.


Jon

hyounkim80 Fri, 10/23/2009 - 10:38
User Badges:

Hey Jon. I assumed solution #1 after I wrote you and did some research. however, I notice some examples on Cisco.com specify the metric in EIGRP when redistributing across other routing protocols.


Specifically: default-metric 10000 100 255 1 1500


Do I have to specify the default metric? If so, does the above line look ok? I got it from Cisco's site for an ethernet link... however I'm wondering why the load is specified as a 1.


Thanks again!

hyounkim80 Fri, 10/23/2009 - 10:41
User Badges:

Oh, I forgot to mention that our WAN link goes out of interface f0/1, which only has an ip address specified in its configuration.


I'll try the suggested changes and let you know how it goes.


Thanks again for your help.

Jon Marshall Fri, 10/23/2009 - 10:47
User Badges:
  • Super Blue, 32500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

Just seen this additional post.


You need to check the distribute list because it applies to routes going out to the main office. Make sure it does not filter any of your internal networks although it shouldn't as it obviously worked before.


Jon

Jon Marshall Fri, 10/23/2009 - 10:45
User Badges:
  • Super Blue, 32500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

You usually do have to specify a metric but not with redistribute static. However if you want to then the above is fine. As there is only one path to your site then it's not such a big deal what metric you specify.


Jon

hyounkim80 Tue, 10/27/2009 - 12:03
User Badges:

Jon,


Just letting you know I have yet to make the changes. I'm waiting on the main office to give me TACACS access to the router (I was using my manager's access).


In the meantime, we got our replacement router for the C3640 (a C3845). Long before I decided to move the subnets to the C3750 switch, my boss recommended that we go ahead and get a new router (C3845). We didn't expect to get the new router until mid-to-late November, but it arrived today.


The router was spec'd out to 512MB RAM & 256 MB Flash (compared to the C3640's 64 MB RAM, 32 MB Flash). Now that we got it, I'm curious about the design aspect of our branch. Since we have a new router that is fairly powerful, should I move the subnets back to the C3845 (as with the original design with the C3640 router) or should I still use the C3750 for internal routing and use the new router specifically for outside access.


Any opinions would be appreciated.

Joseph W. Doherty Tue, 10/27/2009 - 12:34
User Badges:
  • Super Bronze, 10000 points or more

A 3845 is rated at 500 Kpps for minimum size packets (good for about 336 Mbps, Ethernet, 64 byte sized packets). I.e. 3845 should support 100 Mbps line-rate, duplex, but unable to guarantee single gig line-rate.


Both your 3750 and 4948 switch support line-rate all ports, so if there's more than about 100 Mbps LAN routing, you would want it on the L3 switches.

hyounkim80 Tue, 10/27/2009 - 13:27
User Badges:

Thanks Joseph. Much appreciated. I shall utilize the C3750 and still use the routers for routing out of office purposes.

Joseph W. Doherty Wed, 10/21/2009 - 09:34
User Badges:
  • Super Bronze, 10000 points or more

From what you've described, indeed it makes a whole lot of sense to get LAN routing off the 3640 and onto one or both of your L3 switches.


As to which L3 switch to select for routing (or both?) the 4848 is a wire-speed device, the 3750 models often not (although the 12 port fiber version you have is - it also supports special SDM templates). As both your switches support wire rate, and assuming most routing will be between server subnets and user subnets, it wouldn't make much of a difference which switch you route on but if there's also much traffic between the data center subnet and tech/server subnets, and these all will be connected to the 4948, it would make sense to route there (to avoid needlessly passing traffic up/down to the 3750).


Whether to use both L3 switches as routers would only make sense if there much traffic to route between subnets that naturally converge on the 3750. From what you describe, it doesn't seem like there would be much need to route on the 3750 (this assuming most traffic is from downlinked user switches to the port that connects to the 4948).


From later posts, unclear how you would use HSRP without some additional links between switches. If you do have those, then having both L3 switches active while using HSRP makes sense, but this also assumes you have sufficient ports to deal with lost of either the 3750 or 4948.


A good next step might be a dual stack of 3750 12 port SFP models and somewhere to connect the 4948 connected devices if it fails.

dgalati000 Wed, 10/28/2009 - 09:21
User Badges:

Hey Jose, I've been reading this thread and at the end where you discuss using L3 switches as routers; I was wondering if you could advise me as well...HERE's MY QUESTION >>



I need to determine if I should go with a (new) L2 switch and mult VLANS and use router-on-a-stick for intervlan routing. Its conducive to what the cust has at his multiple sites.


Or should I convince him to go with a L3 switch to the router instead ?


My concern is that its a mobile wireless site with two different radio signals (CDMA and GSM) operating in the same site...and I don't want a bottleneck on the trunk between the switch and the router when intervlan routing using the L2 switch. That is, one of the vlans consuming all the 10/100 trunk bandwidth would not work in the L2 switch env - or is it ?


Suggestions please ??

Joseph W. Doherty Wed, 10/28/2009 - 10:10
User Badges:
  • Super Bronze, 10000 points or more

Actually, I had already replied to these questions in your original post "Getting Started with LANs: bandwith considerations w/router-on-a-stick vs L3 switches" although I didn't reply to your 2nd (duplicate) post "Voice over IP: Help with Layer 2 vs Layer 3 switch in mobile wireless env". Was there something to my original reply that was unclear or confusing?

Actions

This Discussion