Redundancy options when you run out of 10G links

branfarm1 · ‎04-07-2010

Hi everybody,

Here's my scenario: I have a two 6509's connected to 3 stacks of three 3750 switches, with 2 10G links per stack. The 10G links connect to the first and last switch in each stack. Each 10G link is a L3 point to point connection. I want to add a backup link to the middle switch in each stack, but I don't have any more 10G links available. I'm looking for feedback on my options.

I figure that my first option is to leave it alone, because I already have two links to each stack Just humor me on this and assume I need another link.

My second option is to use a 1G L3 link between 6500 and 3750, which would have the advantage of utilizing the rapid convergence inherent in dynamic routing protocols.

My third option is to use flex links, by moving the L3 interface to a vlan interface, and configuring the 1G interface as a backup to one of the 10G links.

I haven't seen much on here about people using flex-links -- can I assume that this would be the least preferred option?

Thanks in advance for the help -- all replies will be rated, so feel free to chime in.

--Brandon

Jon Marshall · ‎04-07-2010

Brandon

I like the idea of leaving it alone

However if you must add another link ! , then i would simply make it a routed link although it does beg the question when would it be used ie. both 10Gbps links would need to fail before this one kicked in assuming you are not looking to run EIGRP unequal cost load-balancing.

Is there a specific reason you want a 3rd link ?

Jon

branfarm1 · ‎04-07-2010

Jon,

Thanks for the quick reply. I agree that leaving it alone is the best solution but I had an issue last week that is causing the higher-ups to ask what more can be done. The situation itself was really odd -- one of the 10G links to a stack failed due to a bad 10G GBIC on the master switch. This should have had no effect other than taking down that particular link, but, in this case, it caused the whole stack to become unresponsive. When the stack went sideways on me, the second 10G link also decided to become unresponsive. I had to pull the power on the stack and reboot the devices to get everything running again, other than the failed GBIC.

So now we're supposed to come up with something to prevent this from happening in the future. I don't know -- to me it was a fluke thing that the stack froze up entirely, but I could be missing something. What do you think?

--Brandon

Jon Marshall · ‎04-07-2010

branfarm1 wrote:

Jon,

Thanks for the quick reply. I agree that leaving it alone is the best solution but I had an issue last week that is causing the higher-ups to ask what more can be done. The situation itself was really odd -- one of the 10G links to a stack failed due to a bad 10G GBIC on the master switch. This should have had no effect other than taking down that particular link, but, in this case, it caused the whole stack to become unresponsive. When the stack went sideways on me, the second 10G link also decided to become unresponsive. I had to pull the power on the stack and reboot the devices to get everything running again, other than the failed GBIC.

So now we're supposed to come up with something to prevent this from happening in the future. I don't know -- to me it was a fluke thing that the stack froze up entirely, but I could be missing something. What do you think?

--Brandon

Brandon

Difficult to say what caused it without logs or more details but if the 10Gbps link failing froze the whole stack then wouldn't that mean that even with the new 1Gbps link you would still have a frozen stack and would need to reboot ?

I guess you have to be seen to do something though but i'm not sure how the 1Gbps link is going to help if the entire stack freezes. Have you the opportunity to do testing on one of these stacks ie. shutdown one of the 10Gbps interfaces to make sure it all routes correctly the other way ?

Jon

branfarm1 · ‎04-07-2010

You're definitely right on that -- the stack freezing up trumps everything else. I did have a chance to test routing failover while I was trying to determine if it was a bad cable, GBIC, or port that caused the issue in the first place. Everything worked great throughout all my connecting/disconneting/swapping. How have you dealt with these situations when they arise? In the past I've worked for managers who were more technical and they understood the finer technical aspects... but sometimes, well, you know...

Maybe Cisco should start offering a "Dealing with Management 101" class!

Jon Marshall · ‎04-07-2010

Brandon

Everything worked great throughout all my connecting/disconneting/swapping. How have you dealt with these situations when they arise? In the past I've worked for managers who were more technical and they understood the finer technical aspects... but sometimes, well, you know...

Yes, it can be a tricky one sometimes. Sometimes these things just happen but that isn't what management really want to hear. That's why i understand the idea behind adding the 3rd link, you are right it is not needed but it does show you doing something.

I have had similiar issues before. That's why i asked about the testing because often this is a way to prove that what happened was more of a one off accident than anything to do with the config. What i would do would be to draw up a testing plan, do a thorough bug check etc.. which shows you being as proactive as you can be. This is really what non-technical managers want to see more often than not.

Sorry to be so vague but with non-technical managers it is often really more about being seen to do something rather than actually doing anything worthwhile.

Jon

Leo Laohoo · ‎04-07-2010

Doesn't the 3750E come with a management 10/100 Fast0? I recommend you use this. It's not meant for data traffic but more so in troubleshooting.

If you are out of 10Gb links on your 6500 and I guess it doesn't help much until you get another blade. But the risk of adding a 3rd 10Gb link would it solve the problem or would it make it worst? I'd be looking at what caused the stack to freeze than getting a 3rd link.