Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements

Welcome to Cisco Support Community. We would love to have your feedback.

For an introduction to the new site, click here. If you'd prefer to explore, try our test area to get started. And see here for current known issues.

ASK THE EXPERT – HIGH AVAILABILITY IN CAMPUS NETWORKS

Welcome to the Cisco Networking Professionals Ask the Expert conversation. This is an opportunity to discuss with Cisco expert Bobby Thekkekandam about the concepts and best practices for high availability and redundancy within a campus network, from redundant hardware, processors, and line cards. Bobby joined Cisco Systems, Inc. in the Customer Proof of Concept (CPOC) lab in 1998, and has been an engineer in the Technical Assistance Center (TAC) LAN switching group since 2002. His current responsibilities include escalations and troubleshooting complex issues related to the Cisco Catalyst series switches as well as providing training.

Remember to use the rating system to let Bobby know if you have received an adequate response.

Bobby might not be able to answer each question due to the volume expected during this event. Our moderators will post many of the unanswered questions in other discussion forums shortly after the event. This event lasts through May 19, 2006. Visit this forum often to view responses to your questions and the questions of other community members.

61 REPLIES

Re: ASK THE EXPERT – HIGH AVAILABILITY IN CAMPUS NETWORKS

A customer of us has requested to upgrade his network and to make it more redundant. He has two 3750 switches that are to be used as core switches. The discussion now focuses on the way in which these switches must be interconnected. They could either be stacked using the stacking cable on the rear or installed as separate L3-switches providing a HSRP solution. There are no physical restrictions that may limit choices here, the question is simply which configuration will provide the best redundancy.

If you have any advice about which solution is preferrable I would be glad to hear it.

Regards,

Leo

Cisco Employee

Re: ASK THE EXPERT – HIGH AVAILABILITY IN CAMPUS NETWORKS

Hi Leo,

Stacking will definitely provide the best redundancy here. From a management perspective, the two switches will function as one logical switch, and has a single IP address. Single IP management is supported for activities such as fault detection and VLAN creation and modification. Also, stacking creates a 32 Gbps switch interconnection and does not require user ports.

When switches are added or removed, the master switch automatically loads the Cisco IOS Software release running on the stack to the new switch, loads the global configuration parameters, and updates all the routing tables to reflect changes. Upgrades are applied universally and simultaneously to all members of the stack.

In, the event of an outage. When a troubled switch is removed from an existing stack of switches and replaced with another switch, the master switch will recognize this as a maintenance outage and automatically reload the port-level configuration that was on the previous switch without user intervention.

Also, compared with HSRP, you don't have to burn 3 IP addresses per VLAN for the subnet gateway, as a failover of the master switch will assume the same IP address as the original switch.

HTH,

Bobby

*Please rate helpful posts.

New Member

Re: ASK THE EXPERT – HIGH AVAILABILITY IN CAMPUS NETWORKS

Hi ,

are there are different type of links ? i know there are like OC3, DS3 , E1 , T1 etc . (physically different and capacity wise different ) Now can any traffic travel on these link IP , MPLS , ATM .What about the link which we terminate on MGX / IGX?

regards

JK

Cisco Employee

Re: ASK THE EXPERT – HIGH AVAILABILITY IN CAMPUS NETWORKS

Hi JK,

This question is out of the scope of this particular discussion. May I suggest posting this question in the "WAN, Routing and Switching" section?

thanks,

Bobby

New Member

Re: ASK THE EXPERT – HIGH AVAILABILITY IN CAMPUS NETWORKS

I have a question:

What is the meaning of multiple spanning trees? Why it was not supported by 803.1q?

Second Q:

How long it will take to reach CCIE wt is the best strategy u flow to be at that level? Is that hard

Best regards

Cisco Employee

Re: ASK THE EXPERT – HIGH AVAILABILITY IN CAMPUS NETWORKS

Hi Ahmed,

For your first question, Multiple Spanning Tree or MST (IEEE 802.1s) combines aspects from both the PVST+ and 802.1q. The idea is that several VLANs can be mapped to a reduced number of spanning tree instances because most networks do not need more than a few logical topologies. It's not necessarily accurate to say that it is "not supported" by 802.1q, but rather is a separate standard, developed after the IEEE ratified the 802.1q standard.

Here's a great document that explains MST in detail:

http://www.cisco.com/warp/customer/473/147.html

As for your second question, a better forum to ask questions related to certifications is in the NetPro Career Certifications section:

http://forums.cisco.com/eforum/servlet/NetProf?page=Career_Certifications_discussion

HTH,

Bobby

*Please rate helpful posts.

New Member

Re: ASK THE EXPERT – HIGH AVAILABILITY IN CAMPUS NETWORKS

in a L2 campus with RSPT in the distribution / access switches, access switches have redandant links to distribution 6509s 4500 with latest IOS. do we need any kind of tuning to get very fast SPT fallback to the blocking uplinks? cause on the documentations the uplink/backbone fast are not really needed for RTSP links since the new BPDU format takes care of that operation.

do you have any references how to tune the L2 uplinks to fail instantly considering Voice runs on this links?

thanks

Cisco Employee

Re: ASK THE EXPERT – HIGH AVAILABILITY IN CAMPUS NETWORKS

RSTP (IEEE 802.1w) natively includes most of Cisco's proprietary enhancements to the 802.1d spanning tree such as BackboneFast, UplinkFast, and PortFast. RSTP can achieve much faster convergence in a properly configured network, sometimes in the order of a few hundred milliseconds.

As far as tuning goes, 802.1w is not a timer-based protocol like 802.1D. Besides lowering the

hello interval, you can't do much.

Here is a general RSTP reference:

http://www.cisco.com/warp/customer/473/146.html

HTH,

Bobby

*Please rate helpful posts.

New Member

Re: ASK THE EXPERT – HIGH AVAILABILITY IN CAMPUS NETWORKS

Thanks, for the good link

however

I will need to know if there is anything I can do to speed up recovered links in RTSP

pls take a look at my scenario

6509A is running RTSP and is root for all L2 infrastructure (configured with the root macro as primary)

6509A is running RTSP and is backup root for all L2 infrastructure (configured with the root macro as secondary)

4510 has one trunk to 6509A and 6509B

there is no other configuration on these switches spanning tree others than the mode being RTSP

I am constantly pinging a host on the 4510 from a nework behind 6509s.

the delay of the backup L2 RTSP link is about 30seconds,

Can I tune that to recover faster?

C4510R> sh sp

VLAN0001

Spanning tree enabled protocol rstp

Root ID Priority 8192

Address 000a.f36a.fc01

Cost 4

Port 3 (GigabitEthernet1/3)

Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 32769 (priority 32768 sys-id-ext 1)

Address 0015.62b4.6b80

Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Aging Time 300

Interface Role Sts Cost Prio.Nbr Type

---------------- ---- --- --------- -------- --------------------------------

Gi1/3 Root FWD 4 128.3 P2p

Gi1/4 Altn BLK 4 128.4 P2p

Gi1/5 Desg FWD 4 128.5 P2p

Gi1/6 Desg FWD 4 128.6 P2p

Gi9/36 Desg FWD 19 128.548 P2p

C4510R>sh cdp nei

Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge

S - Switch, H - Host, I - IGMP, r - Repeater, P - Phone

Device ID Local Intrfce Holdtme Capability Platform Port ID

6509L3-A Gig 1/3 170 R S I WS-C6509 Gig 2/13

6509L3-B Gig 1/4 126 R S I WS-C6509 Gig 2/14

C4510R>

C4510R>

C4510R>

C4510R> sh sp vl 151

VLAN0151

Spanning tree enabled protocol rstp

Root ID Priority 8192

Address 000a.f36a.fc97

Cost 4

Port 3 (GigabitEthernet1/3)

Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 32919 (priority 32768 sys-id-ext 151)

Address 0015.62b4.6b80

Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Aging Time 300

Interface Role Sts Cost Prio.Nbr Type

---------------- ---- --- --------- -------- --------------------------------

Gi1/3 Root FWD 4 128.3 P2p

Gi1/4 Altn BLK 4 128.4 P2p

Gi1/5 Desg FWD 4 128.5 P2p

Gi1/6 Desg FWD 4 128.6 P2p

Gi8/47 Desg FWD 19 128.495 P2p

Gi9/36 Desg FWD 19 128.548 P2p

C4510R>en

unpluging root port

New Member

Re: ASK THE EXPERT – HIGH AVAILABILITY IN CAMPUS NETWORKS

Bobby,

any response on my comment?

basically the RSTP in my scenario takes about 30 sec to congerve in a very simple SPT environment. How do I tune for hast convergence?

thanks

h

Silver

Re: ASK THE EXPERT – HIGH AVAILABILITY IN CAMPUS NETWORKS

Hello Bobby,

This question is somewhat related to one asked by another member asking for recommendations on deployment of a Cisco 3750 stack as distribution or core switches. Your recommendation is to deploy them as a stack and to be honest I do not disagree with you but I do have one question:

Can a stack be upgraded from IOS perspective without introducing any downtime? In a two core switch scenario one can upgrade one of the core switches while still maintining connectivity but so far I have been unable to find documentation stating that a stack upgrade will be completely hitless.

New Member

Re: ASK THE EXPERT – HIGH AVAILABILITY IN CAMPUS NETWORKS

hi bobby,

regarding to the first questing about the stacking of catalyst 3750 and "software-upgrade redundancy".

we have a customer who has connected his servers to a "two-switch" cat3750 stack. for redundancy reasons they use two ethernet connections from the server to the stack (one to the first and one to the second switch in the stack).

as far as know, the normal behavior of a cat3750 stack when upgrading the software is that after downloading and distributing the image (or tar file) the whole stack reloads. as the servers of our customer are mission-critical, a reboot of the complete stack at once is unacceptable.

so after spending a lot of time in the lab, is there any procedure to make a software upgrade without rebooting the whole stack all at once (and completly lose the connection to devices behind this stack, even when they are "dual-homed" connected to the stack)?

kind regards,

bernhard

Cisco Employee

Re: ASK THE EXPERT – HIGH AVAILABILITY IN CAMPUS NETWORKS

Hi Bernhard,

This is possible, but to do so has certain requirements and work only with a limited number of IOS versions.

If you are running the 3750 as a L2 device with dual attachments from two different stack members, *AND* you are upgrading to a new version that is compatible with the version currently running on the stack, you may be able to do something like this to minimize downtime:

1) Use "archive download-sw" to download the new code to all stack members without reloading the stack

2) Use "reload slot " to reload the stack member which one of the redundant links is connected to.

3) Wait until switch comes back up, rejoins the stack and become fully operational (i.e. STP state fully converges on all ports) - this is possible only if the new version running on switch is compatible with the old version running on the rest of the stack. Otherwise switch will stay in Version Mismatch state, and automatically downgraded back to the old version unless auto-upgrade is disabled, so this scheme will not work.

4) Use "reload slot " to reload the stack members still running the old version - you will need to issue this command multiple times, once for each of the remaining stack members.

Please note there will be a master switchover during upgrade, which may cause STP to reconverge, in which case there might be some traffic disruption.

Another big caveat with this approach is that the old version and new version must be compatible. Typically only rebuilds are compatible (but not guaranteed - which should be indicated on the front page of the Release Note). So this will only works if the customer is upgrading from, say 12.1(19)EA1c to 12.1(19)EA1d, but not from 12.1(19)EA1d to 12.2(20)SE. So this approach might be of limited usefulness.

HTH,

Bobby

*Please rate helpful posts.

Cisco Employee

Re: ASK THE EXPERT – HIGH AVAILABILITY IN CAMPUS NETWORKS

Hi Atif,

Unfortunately, we cannot achieve zero downtime with a 3750 stack upgrade. That may be one potential disadvantage to a stacked 3750 setup versus an individual switch setup, although I feel even with that caveat, the benefits outweigh this cost, but of course that is a highly subjective point of debate.

HTH,

Bobby

*Please rate helpful posts.

New Member

Re: ASK THE EXPERT – HIGH AVAILABILITY IN CAMPUS NETWORKS

hi

i have more than 12 cabinet arround my site.each cabinet contain 1 shelf fiber;.this shelf fiber must be connected to the main cabinet.this main cabinet contain a main fibershelf;every port fiber in this shelf its connected to the shelf in each cabinet.distance between the cabinet up to 500M.i need help about the switchs in each cabinet that support fiber;note the switch in the DIST MUST HAVE UP TO 24 PORT FIBER..what u advise me about switches(35xx for the distr & 29xx for the access)that meet this situation

10xs

Cisco Employee

Re: ASK THE EXPERT – HIGH AVAILABILITY IN CAMPUS NETWORKS

Hi,

Can you clarify your question some more? Are you trying to determine which switches support fiber uplinks to your central cabinet? What type of switch are these distribution switches connecting to?

thanks,

Bobby

New Member

Re: ASK THE EXPERT – HIGH AVAILABILITY IN CAMPUS NETWORKS

10xs for ur reply!

yes,with ur prospective how to determine the best equipment the best design that meet this situation.in the central cabinet we should install a distribution switch that handel up t 24 port fiber

10xs'Ali

New Member

Re: ASK THE EXPERT – HIGH AVAILABILITY IN CAMPUS NETWORKS

Hi,

i am working within a network with one 4506 switch for the core. Now we want to add a second 4506 switch for redundancy and load balancing. My question is, what is the best way to add this second switch? Is it best to add the second one using HSRP so if one fails the second takes over? Or is it possible to add the second to archieve load balancing and redundancy so if one fails that the second one takes over?

Some more information to the first switch, it does OSPF Routing and is the VTP Server.

Regards

Patrick

Cisco Employee

Re: ASK THE EXPERT – HIGH AVAILABILITY IN CAMPUS NETWORKS

Hi Patrick,

It is certainly possible to add the second switch to achieve both load balancing and redundancy. HSRP provides layer 3 redundancy, and you'll also want to have layer 2 redundancy.

For layer 3 redundancy, one common method is to set your HSRP priorities such that one switch is the active for some vlans (such as even), and the other is active for the remaining vlans (odd). Of course, you can divide that up however you want.

For layer 2 redundancy, there are a lot of ways to accomplish this. Generally, you'll want to have your connecting switches dual homed to both cores, and have a trunk and/or etherchannel between the cores. The etherchannel will provide redundancy as well as aggregate the bandwidth of the multiple links.

A couple of excellent documents cover the best practices for designing a network with high availability:

http://www.cisco.com/application/pdf/en/us/guest/netsol/ns432/c649/cdccont_0900aecd801a8a2d.pdf

http://www.cisco.com/application/pdf/en/us/guest/netsol/ns432/c649/ccmigration_09186a00805fccbf.pdf

http://www.cisco.com/en/US/customer/netsol/ns340/ns394/ns431/ns432/networking_solutions_design_guidances_list.html

HTH,

Bobby

*Please rate helpful posts.

New Member

Re: ASK THE EXPERT – HIGH AVAILABILITY IN CAMPUS NETWORKS

i am using a vpn connection to another company and for some reason while the vpn connection is connected i cannot use my local network is there any reason for this.

Cisco Employee

Re: ASK THE EXPERT – HIGH AVAILABILITY IN CAMPUS NETWORKS

Hi Sean,

This question can better be answered in the VPN forums. Here is a link to that section:

http://forums.cisco.com/eforum/servlet/NetProf?page=Virtual_Private_Networks_discussion

HTH,

Bobby

New Member

Re: ASK THE EXPERT – HIGH AVAILABILITY IN CAMPUS NETWORKS

Hi Bobby

Thanks for allowing this conversation to happen. I'd like to know how to accomplish the following:

I have two datacentres at the moment. What I'd like to do is convert one into a DR site. Currently our live website points to IP x.x.x.x the equivalent ip on our DR site would be y.y.y.y. The problem that comes up is that if I were to failover to the DR site, my DNS would need to be updated to reflect the change in IP. Is there someway of keeping the same public IP's on the firewalls, switches and routers the same on each site.

Just to add my two sites are BGP'ing with our ISP, so failure on our live site link or router will cause traffic to be sent to the new DR site. Connecting the two sites is our internal LAN link. The two sites are in two different geographic locations.

What problems could you see arising from doing this?

How do places like google manage to move their traffic from live to DR without changing their IP's?

Thanks in advance

Dan

Cisco Employee

Re: ASK THE EXPERT – HIGH AVAILABILITY IN CAMPUS NETWORKS

Hi Dan,

Your question is a little out of the scope of this discussion (High availability and redundancy in campus networks). May I point you to the WAN, Routing and Switching section, which is better suited to your scenario?

http://forums.cisco.com/eforum/servlet/NetProf?page=netprof&forum=Network%20Infrastructure&topic=WAN%2C%20Routing%20and%20Switching&CommCmd=MB%3Fcmd%3Ddisplay_messages%26mode%3Dnew%26location%3D.ee71a06

thanks,

Bobby

New Member

Re: ASK THE EXPERT – HIGH AVAILABILITY IN CAMPUS NETWORKS

Hello all.

This is my first time in my life to post a message on this forum.

I have done my CCNA a few years ago and after I decided to practice it now I have bought 2 Cisco Router 2600 serious. my question is a bit silly, but I hope you can help me anyway.

now I have got 2 second hand cisco router

2610 & 2620 serious

I have just received them and have not been able to connect the cables so that I can use them as a home lab for practicing my studying and configuring them.

I would appreciate any help from you .

Thank you very much

Cisco Employee

Re: ASK THE EXPERT – HIGH AVAILABILITY IN CAMPUS NETWORKS

Hi,

Your question is out of the scope of this discussion (High Availability and Redundancy in Campus Networks). Your question can best be answered the LAN, Switching and Routing section:

http://forums.cisco.com/eforum/servlet/NetProf?page=netprof&forum=Network%20Infrastructure&topic=LAN%2C%20Switching%20and%20Routing&CommCmd=MB%3Fcmd%3Ddisplay_messages%26mode%3Dnew%26location%3D.ee71a04

thanks,

Bobby

New Member

Re: ASK THE EXPERT – HIGH AVAILABILITY IN CAMPUS NETWORKS

Regarding 10GE redundancy.

Hi Bobby, what are popular topology approaches for link resliance with 10GE. For example the 4948-10GE Switch can have a max of two 10GE. If I want to deploy a pair of 4948-10GE for dual nic servers. That could give me a single 10GE uplink from each 4948 and a single 10GE link between the switches. I want to avoid a split LAN in the event of the loss of the switch to switch link, but can I bundle the 10GE in a GEC along with a 1000Base TX? The data sheet says a max of 40Gbps for a GEC.

Depending on the Data flows I guess another option is a 40Gbps GEC between the switches and even the standby uplink?.

Cisco Employee

Re: ASK THE EXPERT – HIGH AVAILABILITY IN CAMPUS NETWORKS

Hi Les,

The 40Gbps in the data sheet is "marketing math," meaning that it is referring to full duplex throughput. Since there are two 10GE links on the 4948-10GE switch, bundling the two into an etherchannel will give you 10Gbps tx and rx, for a total for 40Gbps.

You won't be able to bundle a 10GE link with a 1000baseTX link, as part of the requirements for etherchannel are that the links be of the same speed.

However, what you can do is connect the 10GE links together, and connect a separate link with one of the copper ports, and simply allow STP to block it. In the event of a link failure, the 1000baseTX port will become active, preventing the LAN from becoming segmented, although at a reduced bandwidth.

HTH,

Bobby

*Please rate helpful posts.

New Member

Re: ASK THE EXPERT – HIGH AVAILABILITY IN CAMPUS NETWORKS

Hello, I posted this in the regular forum aswell but no on seems to know what's wrong.

I'll try this post aswell.

I'm experiencing a rather annoying thing in a clients network.

Here's the setup:

2 Cisco Catalyst 6509 with SUP720

Cisco Internetwork Operating System Software

IOS (tm) s72033_rp Software (s72033_rp-JK9SV-M), Version 12.2(18)SXD5, RELEASE SOFTWARE (fc3)

Technical Support: http://www.cisco.com/techsupport

Copyright (c) 1986-2005 by cisco Systems, Inc.

Compiled Fri 13-May-05 19:15 by ssearch

Image text-base: 0x4002100C, data-base: 0x42698000

ROM: System Bootstrap, Version 12.2(17r)S2, RELEASE SOFTWARE (fc1)

BOOTLDR: s72033_rp Software (s72033_rp-JK9SV-M), Version 12.2(18)SXD5, RELEASE SOFTWARE (fc3)

Here's the essentials out of my configuration:

mls ip multicast flow-stat-timer 9

no mls flow ip

no mls flow ipv6

mls cef error action freeze

spanning-tree mode pvst

no spanning-tree optimize bpdu transmission

redundancy

mode sso

main-cpu

auto-sync running-config

interface Vlan7

description XXXXX

ip vrf forwarding XXXXX

ip address XXX.XXX.XXX.61 255.255.255.0

ip access-group XXXXX in

no ip redirects

standby 1 ip XXX.XXX.XXX.60

standby 1 authentication XXXXX

router bgp XXXXX

bgp log-neighbor-changes

address-family ipv4 vrf XXXXX

neighbor XXX.XXX.XXX.62 remote-as XXXXX

neighbor XXX.XXX.XXX.62 activate

no auto-summary

no synchronization

network XXX.XXX.XXX.0 mask 255.255.255.0

exit-address-family

ip vrf XXXXX

rd 1:7

route-target export 1:7

My problem is this, when I am at one VLAN and ping's a device on another VLAN i get response times like this:

Response from XXX.XXX.XXX.XXX: byte=32 tid < 1 ms TTL=127

Response from XXX.XXX.XXX.XXX: byte=32 tid < 1 ms TTL=127

Response from XXX.XXX.XXX.XXX: byte=32 tid < 1 ms TTL=127

Response from XXX.XXX.XXX.XXX: byte=32 tid=248ms TTL=127

Response from XXX.XXX.XXX.XXX: byte=32 tid < 1 ms TTL=127

Response from XXX.XXX.XXX.XXX: byte=32 tid < 1 ms TTL=127

Response from XXX.XXX.XXX.XXX: byte=32 tid < 1 ms TTL=127

Response from XXX.XXX.XXX.XXX: byte=32 tid=97ms TTL=127

Response from XXX.XXX.XXX.XXX: byte=32 tid < 1 ms TTL=127

Response from XXX.XXX.XXX.XXX: byte=32 tid < 1 ms TTL=127

Response from XXX.XXX.XXX.XXX: byte=32 tid < 1 ms TTL=127

Response from XXX.XXX.XXX.XXX: byte=32 tid=209ms TTL=127

Response from XXX.XXX.XXX.XXX: byte=32 tid=9ms TTL=127

Response from XXX.XXX.XXX.XXX: byte=32 tid=48ms TTL=127

Response from XXX.XXX.XXX.XXX: byte=32 tid < 1 ms TTL=127

Response from XXX.XXX.XXX.XXX: byte=32 tid < 1 ms TTL=127

Response from XXX.XXX.XXX.XXX: byte=32 tid < 1 ms TTL=127

Response from XXX.XXX.XXX.XXX: byte=32 tid=3ms TTL=127

Response from XXX.XXX.XXX.XXX: byte=32 tid=9ms TTL=127

I have tried to connect two PC's to the same physical card in one off my 6509's just at two seperate VLANs and I still have the same problem.

If anyone has any ideas I would be more than happy to hear, if you need to see some more config just post a replie here.

Thanks in advance

New Member

Re: ASK THE EXPERT – HIGH AVAILABILITY IN CAMPUS NETWORKS

Hi,

I assume both VLANs are in the VRF, right?

From my previous expereince, you can never really complain when you get those spikes in ICMP echo-replies. It also does not indicate that you have a bad network connection. PCs are running many processes, so it's possible that at that instance there a process with higher priority which caused the ICMP echo-reply to be delayed.

Did you try to ping from interface-to-interface (use extended ping to set source IP)?

David

79
Views
53
Helpful
61
Replies