01-01-2009 05:52 PM - edited 03-04-2019 03:17 AM
Good evening everyone. Well after doing an IOS upgrade from s72033-advipservicesk9_wan-mz.122-33.SRA5 to c7600s72033-advipservicesk9-mz.122 SRB4 (Highest level approved by our org) on our 7606. The standby supervisor
is in a "Current Software state = STANDBY COLD" state vs HOT. Here is the log after the standby SUP was reset with new IOS, The primary SUP is already running the new IOS when this SUP was reset. I have included results from "sh redundancy" also.
Please let me you what you think might be the issue...
Thanks All!
Dec 31 16:02:10: %SNMP-5-MODULETRAP: Module 5 [Down] Trap
Dec 31 16:02:09: %OIR-SP-3-PWRCYCLE: Card in module 5, is being power-cycled (Mo
dule reset)
Dec 31 16:02:10: %RED_MODE-3-RED_MODE_MSG_NOT_OK: Red Mode ISSU msg type (100) i
s not ok
Dec 31 16:02:10: %PFREDUN-SP-6-ACTIVE: Standby processor removed or reloaded, ch
anging to Simplex mode
Dec 31 16:04:04: %ISSU-SP-3-INCOMPATIBLE_PEER_UID: Peer image (c7600s72033_sp-AD
VIPSERVICESK9-M), version (12.2(33)SRB4) on peer uid (5) is incompatible
Dec 31 16:05:03: %PFREDUN-SP-4-INCOMPATIBLE: Defaulting to RPR mode (Runtime inc
ompatible)
Dec 31 16:05:08: %FABRIC-SP-5-CLEAR_BLOCK: Clear block option is off for the fab
ric in slot 5.
Dec 31 16:05:08: %FABRIC-SP-5-FABRIC_MODULE_BACKUP: The Switch Fabric Module in
slot 5 became standby
Dec 31 16:05:09: %DIAG-SP-6-RUN_MINIMUM: Dec 31 16:05:09: %DIAG-SP-6-DIAG_OK: Module 5: Passed Online Diagnostics
Dec 31 16:05:10: %OIR-SP-6-INSCARD: Card inserted in slot 5, interfaces are now
online
Dec 31 16:07:34: %PFREDUN-SP-6-ACTIVE: Standby initializing for RPR mode
Dec 31 16:07:34: %SYS-SP-3-LOGGER_FLUSHED: System was paused for 00:00:00 to ens
ure console debugging output.
Dec 31 16:07:35: %PFINIT-SP-5-CONFIG_SYNC: Sync'ing the startup configuration to
the standby Router.
Dec 31 16:07:39: %RF-SP-5-RF_TERMINAL_STATE: Terminal state reached for (RPR)
sh red
Redundant System Information :
------------------------------
Available system uptime = 19 weeks, 4 days, 27 minutes
Switchovers system experienced = 1
Standby failures = 5
Last switchover reason = user initiated
Hardware Mode = Duplex
Configured Redundancy Mode = sso
Operating Redundancy Mode = rpr
Maintenance Mode = Disabled
Communications = Up
Current Processor Information :
-------------------------------
Active Location = slot 6
Current Software state = ACTIVE
Uptime in current state = 2 days, 1 hour, 11 minutes
Image Version = Cisco IOS Software, c7600s72033_rp Software (c7
600s72033_rp-ADVIPSERVICESK9-M), Version 12.2(33)SRB4, RELEASE SOFTWARE (fc3)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2008 by Cisco Systems, Inc.
Compiled Thu 24-Jul-08 01:47 by prod_rel_team
BOOT = sup-bootdisk:c7600s72033-advipservicesk9-mz.122
-33.SRB4.bin,1;sup-bootdisk:s72033-advipservicesk9_wan-mz.122-33.SRA5.bin,1;
CONFIG_FILE =
BOOTLDR =
Configuration register = 0x2102
Peer Processor Information :
----------------------------
Standby Location = slot 5
Current Software state = STANDBY COLD
Uptime in current state = 1 day, 3 hours, 4 minutes
Image Version = Cisco IOS Software, c7600s72033_rp Software (c7
600s72033_rp-ADVIPSERVICESK9-M), Version 12.2(33)SRB4, RELEASE SOFTWARE (fc3)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2008 by Cisco Systems, Inc.
Compiled Thu 24-Jul-08 01:47 by prod_rel_team
BOOT = sup-bootdisk:c7600s72033-advipservicesk9-mz.122
-33.SRB4.bin,1;sup-bootdisk:s72033-advipservicesk9_wan-mz.122-33.SRA5.bin,1;
CONFIG_FILE =
BOOTLDR =
Configuration register = 0x2102
01-02-2009 12:33 AM
Hello Robert,
Configured Redundancy Mode = sso
Operating Redundancy Mode = rpr
for this reason the state is STANDBY COLD
http://www.cisco.com/en/US/docs/routers/7600/ios/12.2SR/configuration/guide/nsfsso.html#wp1119694
What we can see from logs is:
Dec 31 16:02:10: %RED_MODE-3-RED_MODE_MSG_NOT_OK: Red Mode ISSU msg type (100) i
s not ok
Dec 31 16:02:10: %PFREDUN-SP-6-ACTIVE: Standby processor removed or reloaded, ch
anging to Simplex mode
Dec 31 16:04:04: %ISSU-SP-3-INCOMPATIBLE_PEER_UID: Peer image (c7600s72033_sp-AD
VIPSERVICESK9-M), version (12.2(33)SRB4) on peer uid (5) is incompatible
Dec 31 16:05:03: %PFREDUN-SP-4-INCOMPATIBLE: Defaulting to RPR mode (Runtime inc
ompatible)
for some reasons the active supervisor didn't like the peer and so now they are in RPR mode.
I would try to reboot again the standby supervisor.
the error message decoder says for the first message :
1. %RED_MODE-3-RED_MODE_MSG_NOT_OK: Red Mode ISSU msg type ([dec]) is not ok
The redundancy mode ISSU message type has not negotiated correctly.
Recommended Action: Copy the error message exactly as it appears on the console or in the system log. Research and attempt to resolve the error using the Output Interpreter https://www.cisco.com/cgi-bin/Support/OutputInterpreter/home.pl. Issue the show message types command to gather data that may help identify the nature of the error. Also perform a search of the Bug Toolkit http://tools.cisco.com/Support/BugToolKit/action.do?hdnAction=searchBugs. If you still require assistance, open a case with the Technical Assistance Center via the Internet http://www.cisco.com/cgi-bin/front.x/case_tools/caseOpen.pl, or contact your Cisco technical support representative and provide the representative with the gathered information.
Related documents- No specific documents apply to this error message.
How have you resetted the standby supervisor ?
Hope to help
Giuseppe
01-02-2009 07:27 PM
Thanks for the sugestions! I have reset the standby a couple of times now. I came to he same conclusion as as as why I was in standby cold. The log also mentions incompatible image... Maybe it is because the old image was not deleted and it does not support ISSU? Anyone else have thoughts on this idea?
01-03-2009 02:55 AM
Hello Robert,
the strange aspect is the standby Sup has the right image see from your first post:
Dec 31 16:04:04: %ISSU-SP-3-INCOMPATIBLE_PEER_UID: Peer image (c7600s72033_sp-AD
VIPSERVICESK9-M), version (12.2(33)SRB4) on peer uid (5) is incompatible
So I suspect it is a question of timing.
connect the console to the standby Supervisor and try to see what it says (not sure the console will work on standby you may need to use the master Supervisor and then switch console command)
Hope to help
Giuseppe
01-13-2009 05:56 PM
Did you find the issue? I have a set of 7609 12.2.33SRC2 doing the same thing...
10-15-2009 09:07 AM
Hi,
I have the same issue on the router 7613 with IOS 12.2.33SRD2. Could you please tell me how you solved the problem?
Thanks in the advance,
Georgina
10-15-2009 10:27 AM
Hello Georgina,
have you got a NAM module on your chassis?
I've found a field notice that should apply to 12.2(33)SRC2
see
http://www.cisco.com/en/US/partner/ts/fn/631/fn63179.html
or
http://www.cisco.com/en/US/ts/fn/631/fn63179.html
it may be good if you can provide a sh module
Edit:
there are also some bugs that may apply like
CSCta36814 Bug Details
After final reload of ISSU SRB3 -> SRD1 the standby loads only to RPR mode and
reports MCL configuration mistmatch.
Hope to help
Giuseppe
10-16-2009 04:50 AM
Hi,
Thanks for the inputs.
The problem seems to be related with the NAM module.
After removing the NAM card the redundancy started to work perfectly.
My Best Regards,
Georgina Cunha
10-15-2009 10:45 AM
We felt it was mis-matched configs between the Sups.We can issued the following command at the enable prompt: "redundancy config-sync ignore mismatched-commands". After issuing this command, we saved the configuration to memory and then reloaded the standby. Assuming that any mis-matched commands was the culprit, this process should revert the standby sup back to sso mode. It did for us. Other input told to us to take ssh out of config before first boot as the keys will always be a "mismatched config"
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide