cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1569
Views
0
Helpful
3
Replies

vPC query : possible to force "up" with a single switch?

amenh
Level 1
Level 1

I believe it may not be possible, but I thought I'd ask around to see if anyone else had run into this. I had an interesting (understatement!) failure today. Our data center experienced a classic cascading power failure (circuit failure shunts load to "backup" or "secondary" circuit, overloading that circuit). The initial end result was a fully down N5K that had been a member of a functioning vPC pair. The other N5K remained up. However, apparently the data center operators, in their zeal to troubleshoot, repeatedly cycled power on the down N5K AND N5K that had initially stayed up.

The combo joy was a dead N5K (green lights, no boot, TAC case already opened for RMA) and the other N5K that booted without ever having a functioning vPC peer.

The end result was a production network completely down, even though 1 of the N5Ks was up and available. Since it couldn't talk to its peer, it wouldn't bring any of the vPCs up (uplinks, server links, nor the peer-link). I poked around, didn't see any obvious way to "force" it to bring them up. I'll be opening a TAC case for that, just to be complete. I've also sent a query to my SE team.

Nevertheless, I wondered if anyone else had experienced this scenario and had come up with a solution. I was fortunate in that I had a spare N5K that I could configure identically to the failed unit and migrate the connections; the vPCs all came up fine after that. When I first installed this system, I tested single switch failures and that worked as expected (once the vPC is up, it survives a partner failure just fine). I just didn't consider the possibility of both switches rebooting, with 1 failing to power back up.

Thoughts?

Hagen

1 Accepted Solution

Accepted Solutions

Darren Ramsey
Level 4
Level 4

Yes I have seen this in our lab environment.

http://tools.cisco.com/Support/BugToolKit/search/getBugDetails.do?method=fetchBugDetails&from=myNotification&bugId=CSCte95521

Looks like it's fixed in a pending release.

The only way I could make it work was to remove the "vpc xx" statement from all the port-channels until the dead 5K recovered.

View solution in original post

3 Replies 3

Darren Ramsey
Level 4
Level 4

Yes I have seen this in our lab environment.

http://tools.cisco.com/Support/BugToolKit/search/getBugDetails.do?method=fetchBugDetails&from=myNotification&bugId=CSCte95521

Looks like it's fixed in a pending release.

The only way I could make it work was to remove the "vpc xx" statement from all the port-channels until the dead 5K recovered.

Darren, that's it exactly. I guess I'm pending the release of software! I'm hopeful it won't happen again, but nice to know I'm not crazy (at least this time). Dumping the vPC config popped into my head, but since I had the shelf spare N5K, I went for that "fix".

Thanks for the bug ID.

Hagen

This could be a scary situation for sure in a production data center environment with the exact conditions you described.

I'm hoping this will be fixed in 5.0(2)N1(1), and you'll also get some new cool features like FEX pre-provisioning, config-sync, and config rollback

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: