VSS on 4500 Testing failing!

[Mohit Chauhan]

Hi friends

This is the first time I am configuring VSS on 4500 Sup7R+E (quad)

I configured the VSS setup following this link below which looked fairly straight-forward:

http://www.labminutes.com/blog/2013/09/cisco-4500-4500x-6500-vss-configuration 

I used two tengig ports one from SUP1 of each and second from 10G card using X-2 module as part of the VSL. Dual active from Gig card (copper). I had upgraded the switches to 3.0.5 to have the new feature of dual-active using fast-hellos.

So below were the configured ports:

T1/2/1-T2/2/1 (VSL)

T1/3/1-T2/3/1 (VSL)

G1/5/48-G2/5/48 (Dual-active)

But then I was having a hard time to figure out how to configure the Sup2's on both switches for quad support (ICS). I tried to follow the below link

http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst4500/15-1-2/XE_340/configuration/guide/config/vss.html#wp1204229

And finally I did this by shutting down the complete VSS cluster and then unplugging the SUP1's from both switches. And re-doing the same VSS config but using different 10G port on the SUP2 of each since SUP1 was not available. 

So basically the ports were like this:

T1/2/1-T2/2/1 (VSL)

T1/4/1-T2/4/1 (VSL)

G1/5/48-G2/5/48 (Dual-active)

Also I set config-register of both SUP2's to value 0 (auto-boot disabled)

All was fine till here.

However when I started testing, like say even having just one SUP1 on each switch, and if I unplug the SUP1 from the SW1, all the cards from that switch move to provisioning mode and I lose the connectivity to them. All the VSL links goes down and also the dual-active link.

Is that normal and expected?

SW1#sh module
Switch Number: 1 Role: Virtual Switch Standby

Chassis Type : WS-C4507R+E

Power consumed by backplane : 40 Watts

Mod Ports Card Type                              Model              Serial No.
---+-----+--------------------------------------+------------------+-----------
 1    24  1000BaseX (SFP)
 2    18  10GE (X2), 1000BaseX (SFP)
 3     6  Sup 7L-E 10GE (SFP+), 1000BaseX (SFP)
 4     6  Sup 7L-E 10GE (SFP+), 1000BaseX (SFP)
 5    48  10/100/1000BaseT Premium POE E Series
 6    48  10/100/1000BaseT Premium POE E Series

 M MAC addresses                    Hw  Fw           Sw               Status
--+--------------------------------+---+------------+----------------+---------
 1 bc16.65e7.84c0 to bc16.65e7.84d7                                   Provision
 2 f872.eab0.0f5e to f872.eab0.0f6f                                   Provision
 3 6c41.6a77.79c0 to 6c41.6a77.79c5                                   Provision
 4 6c41.6a77.79c6 to 6c41.6a77.79cb                                   Provision
 5 f872.eadc.8f7c to f872.eadc.8fab                                   Provision
 6 bc16.65e7.6fcc to bc16.65e7.6ffb                                   Provision

Mod  Redundancy role     Operating mode      Redundancy status
----+-------------------+-------------------+----------------------------------
 3   Standby Supervisor  SSO                 Disabled

System Failures:
----------------
Power Supply:   bad/off (see 'show power')

Switch Number: 2 Role: Virtual Switch Active

Chassis Type : WS-C4507R+E

Power consumed by backplane : 40 Watts

Mod Ports Card Type                              Model              Serial No.
---+-----+--------------------------------------+------------------+-----------
 1    24  1000BaseX (SFP)                        WS-X4624-SFP-E     JAE17280093
 2    18  10GE (X2), 1000BaseX (SFP)             WS-X4606-X2-E      JAE17270AQ0
 3     6  Sup 7L-E 10GE (SFP+), 1000BaseX (SFP)  WS-X45-SUP7L-E     CAT1727L00F
 5    48  10/100/1000BaseT Premium POE E Series  WS-X4648-RJ45V+E   JAE17270CCD
 6    48  10/100/1000BaseT Premium POE E Series  WS-X4648-RJ45V+E   JAE172802NS

 M MAC addresses                    Hw  Fw           Sw               Status
--+--------------------------------+---+------------+----------------+---------
 1 6c41.6acb.61bc to 6c41.6acb.61d3 2.0                               Ok
 2 f872.eab0.1012 to f872.eab0.1023 2.0                               Ok
 3 6c41.6a77.7cc0 to 6c41.6a77.7cc5 3.0 15.0(1r)SG10 03.05.01.E       Ok
 5 f872.ea36.18e4 to f872.ea36.1913 3.1                               Ok
 6 6c41.6acb.6204 to 6c41.6acb.6233 3.1                               Ok

Mod  Redundancy role     Operating mode      Redundancy status
----+-------------------+-------------------+----------------------------------
 3   Active Supervisor   SSO                 Active

System Failures:
----------------
Power Supply:   bad/off (see 'show power')

Any suggestions? I need to get this working properly ASAP. Thanks in advance guys!

[Mohit Chauhan]

Just to add here, below was the strange output which I wasnt sure about. So thought would be worth mentioning it here.

Switch1#sh switch virtual link port-channel

Executing the command on VSS member switch role = VSS Active, id = 1

Flags:  D - down        P - bundled in port-channel
        I - stand-alone s - suspended
        H - Hot-standby (LACP only)
        R - Layer3      S - Layer2
        U - in use      N - not in use, no aggregation
        f - failed to allocate aggregator

        M - not in use, no aggregation due to minimum links not met
        m - not in use, port not aggregated due to minimum links not met
        u - unsuitable for bundling
        d - default port

        w - waiting to be aggregated

Group  Port-channel  Protocol    Ports
------+-------------+-----------+-------------------
1      Po1(SU)          -        Te1/2/1(P)  Te1/3/1(P)
2      Po2(SU)          -        Te2/2/1(P)  Te2/3/1(P)  Te2/4/1(w)

Executing the command on VSS member switch role = VSS Standby, id = 2

Flags:  D - down        P - bundled in port-channel
        I - stand-alone s - suspended
        H - Hot-standby (LACP only)
        R - Layer3      S - Layer2
        U - in use      N - not in use, no aggregation
        f - failed to allocate aggregator

        M - not in use, no aggregation due to minimum links not met
        m - not in use, port not aggregated due to minimum links not met
        u - unsuitable for bundling
        d - default port

        w - waiting to be aggregated

Group  Port-channel  Protocol    Ports
------+-------------+-----------+-------------------
1      Po1(SU)          -        Te1/2/1(P)  Te1/3/1(P)
2      Po2(SU)          -        Te2/2/1(P)  Te2/3/1(P)  Te2/4/1(D)

[Jon Marshall]

However when I started testing, like say even having just one SUP1 on each switch, and if I unplug the SUP1 from the SW1, all the cards from that switch move to provisioning mode and I lose the connectivity to them. All the VSL links goes down and also the dual-active link.

Is that normal and expected?

Yes this is normal behaviour for a single suprevisor chassis.

With VSS for the control plane you have one active and one standby switch and the active switch is responsible for things like STP, building the L3 routing tables and the forwarding tables.

The forwarding tables are then downloaded to the supervisor in the standby switch and any DFC enabled linecards.

In terms of the data/forwarding plane both switches are active and can forward data independantly of each other. The supervisor in each chassis is responsible for it's own linecards so if the supervisor in that chassis fails then so do the linecards ie. the linecards cannot be run by the other switch's supervisor.

So when you unplugged the supervisor all the linecards failed which means the VSL and your dual-active link also failed.

Jon

[Mohit Chauhan]

Hi Jon

Thanks for your valuable inputs.

In our case we do have dual SUPs for each switch, is there a better way to have better resiliency so that when I do unplug the SUP-1 from SW1, the line cards of SW1 are still available?

All I have found so far that we can only use the SUP-2 in both switches as ICS and have to manually boot them from ROM mode.

For Quad-SUP environment, how can we have exact same VSS on SUP-2's matching that was on SUP-1's of the switches. The way I explained I have configured is correct? Is there a better way?

I have attached the run config of the switch. Let me know if you need any more output for clarification.

Thanks!

Mohit

[Jon Marshall]

Mohit

As far as i know the standby supervisor in the same chassis will not take over if the active supervisor fails ie. it cannot take over the running of the linecards in the chassis. This is because SSO (or RPR) is only run between the two active supervisor, one in each chassis and not between the active and standby supervisors in the same chassis.

And it does not boot automatically either ie. you must manually do it.

In addition you must make sure that the standby supervisor (ICS) is configured to only boot to ROMMON mode otherwise it will just keep trying to reload.

Jon

[Mohit Chauhan]

Thanks Jon.
I have disabled auto boot on the ICS SUPs. 
But as I mentioned in my earlier comments, my original VSS config using the SUP1's of both chassis were by using the following ports as below:

T1/2/1-T2/2/1 (VSL)

T1/4/1-T2/4/1 (VSL)

G1/5/48-G2/5/48 (Dual-active)

However, when I configured the sec SUPs on each chassis to run in VSS mode for the first time by keeping the fIrst SUPs unplugged, the guidelines said to configure the VSS in exactly the same way like we did for the first SUPs. But what I realized here was that I could not make exactly the same 10g links as part of the VSL group as the port on SUP1s we're not available. So the config of the VSS on second SUPs were slightly different. ie instead of T1/3/1-T2/3/1, I used T1/4/1-T2/4/1

So my ports were like this:

T1/2/1-T2/2/1 (VSL)

T1/4/1-T2/4/1 (VSL)

G1/5/48-G2/5/48 (Dual-active)

I am hoping that should not create any mismatch?

In the testing scenario when I unplug the SUP1 from the SW1, and manually boot the SUP2 on the same chassis, it should form VSS with the other chassis which is running on SUP1? And that way, all the cards on the chassis one should come back?

And in that case the only active VSL ports will be only one, ie T1/2/1-T2/2/1. 

If the above  theory seems to be true, then I think I should use the third VSL port using the 10Gline card only, ie T1/2/x-T2/2/y and not the SUP2's 10G port.
What is your suggestion on this? 

[Jon Marshall · Accepted solution]

Mohit

I understand what you are asking but i have never done quad supervisor support so i can't say for sure what is recommended and not.

The ports on the ICS are capable of forwarding data so i don't know whether you can include those in the VSL configuration ie.

your VSL link is configured using all the ports on both supervisors for each chassis plus the linecard ports. If the ICA fails then you lose the opposite ports on the other ICA in the other chassis but you still get -

1) the ports between the new ICA and the ICS in the other chassis

and

2) the ports on the linecards

i would try testing with this and see if it works. This way you could have the exact same configuration on both supervisors per chassis.

Sorry i can't be more specific.

Hopefully someone who has implemented quad supervisors can verify whether this will work or not.

Jon

[Mohit Chauhan]

Hi Jon

I did test by unplugging the active (ICA-ie SUP1) from the chassis 1 (which was VSS active). This triggered the VSS SSO so the standby Chassis 2 took over the active role on its ICA (SUP1).

I saw as expected the line cards failing on chassis1.

Now I manually booted the SUP2 (ICS) on chassis 1 and after a long time when it became VSS standby.

It was now when i could the line cards come up and after sometime my pings to one of the cards started working.

All this time, these pings were not working.

I also tried dual-homing a basic switch and using MEC on the VSS side. This was the failover were like 2 ping drops only.

thanks for your help in getting me to have a better understanding. 

Cheers,

Mohit