Re: CAT 6506 vs. 7.6.12

82netops · ‎04-25-2006

Are there any issues regarding sup. module WS-X6K-SUP1A-2GE and image cat6000-supk8.7-6-12.bin? We have recently discovered problems with this combination and random reboots. This may be caused by a bad image which is being resolved w/ a new push but I just wanted to check. Any help would be appreciated, thank you.

Shannon

Bobby Thekkekandam · ‎04-25-2006

Hi Shannon,

One thing that jumps out from the top of my head is that it requires 128MB of RAM. The Sup1A comes with 64MB by default. If you try to run later 7.6 versions with 64MB of RAM, you'll run into the type of problems that you've described.

HTH,

Bobby

*Please rate helpful posts.

82netops · ‎04-26-2006

Hey Bobby, thanks for the response. We verified that we have 128K DRAM, running at 37% utilization. Any other ideas? Thanks.

Bobby Thekkekandam · ‎04-26-2006

Could you post a "show log" and "show ver"?

There may be something useful in a stack trace that will point to the problem.

-Bobby

82netops · ‎04-27-2006

B237_NCC_6506> (enable) sh log

Network Management Processor (ACTIVE NMP) Log:

Reset count: 6

Re-boot History: Apr 27 2006 08:20:03 0, Apr 27 2006 08:11:54 0

Apr 27 2006 08:03:38 0, Apr 25 2006 20:00:54 0

Apr 25 2006 13:43:10 0, Apr 25 2006 13:34:26 0

Bootrom Checksum Failures: 0 UART Failures: 0

Flash Checksum Failures: 0 Flash Program Failures: 0

Power Supply 1 Failures: 0 Power Supply 2 Failures: 1

Swapped to CLKA: 0 Swapped to CLKB: 0

Swapped to Processor 1: 0 Swapped to Processor 2: 0

DRAM Failures: 0

Exceptions: 6

Loaded NMP version: 7.6(12)

Software version: bootflash:cat6000-supk8.7-6-12.bin

Reload same NMP version count: 27

Last software reset by user: 4/25/2006,20:00:00

Last Exception occurred on Apr 27 2006 08:02:44 ...

Software version = 7.6(12)

Error Msg:

Cache Error Exception occurred

EPC: BFC08518

Stack content: Not Available

Register content:

Status: 3051C005 Cause: 00000008

AT: 00000000

V0: 00000000 V1: FFFFFFFF

A0: A0002032 A1: 00000080

A2: 80005E04 A3: A0002030

T0: 00100000 T1: A0002F9A

T2: 00000008 T3: 00000002

T4: 7F76FFDF T5: 00000008

T6: 3041C080 T7: 3041C000

S0: 00000000 S1: A0002032

S2: 7BF79FFF S3: 0000007F

S4: A0002032 S5: 0000000D

S6: 00000016 S7: 00000008

T8: 3041C001 T9: 85333128

K0: 80005FC8 K1: 80005DB0

GP: 815527E0 SP: 80005DB0

S8: 00000003 RA: BFC08554

HIGH: BFC01540 LOW: 00A80000

BADVADDR: 20202030 ERR EPC: BFC08518

EOBC Exceptions/Hang: 0

Heap Memory Log:

Corrupted Block = none

NVRAM log:

01. 4/25/2006,13:36:37: StartupConfig:System config started

02. 4/25/2006,13:39:57: StartupConfig:System config ended

03. 4/25/2006,13:39:58: StartupConfig:System config started for module 3

04. 4/25/2006,13:40:01: StartupConfig:System config ended for module 3

05. 4/25/2006,13:45:21: StartupConfig:System config started

06. 4/25/2006,13:48:50: StartupConfig:System config ended

07. 4/25/2006,13:48:52: StartupConfig:System config started for module 2

08. 4/25/2006,13:48:53: StartupConfig:System config ended for module 2

09. 4/25/2006,20:03:06: StartupConfig:System config started

10. 4/25/2006,20:06:26: StartupConfig:System config ended

11. 4/25/2006,20:06:27: StartupConfig:System config started for module 3

12. 4/25/2006,20:06:29: StartupConfig:System config ended for module 3

13. 4/27/2006,08:14:06: StartupConfig:System config started

14. 4/27/2006,08:17:31: StartupConfig:System config ended

15. 4/27/2006,08:17:37: StartupConfig:System config started for module 3

16. 4/27/2006,08:17:38: StartupConfig:System config ended for module 3

17. 4/27/2006,08:21:36: StartupConfig:System config started

18. 4/27/2006,08:21:36: StartupConfig:System config ended

19. 4/27/2006,08:21:45: StartupConfig:System config started for module 1

20. 4/27/2006,08:21:45: StartupConfig:System config ended for module 1

21. 4/27/2006,08:25:06: StartupConfig:System config started for module 2

22. 4/27/2006,08:25:18: StartupConfig:System config ended for module 2

23. 4/27/2006,08:27:05: StartupConfig:System config started for module 3

24. 4/27/2006,08:27:20: StartupConfig:System config ended for module 3

25. 4/27/2006,08:29:53: StartupConfig:System config started for module 4

26. 4/27/2006,08:30:08: StartupConfig:System config ended for module 4

82netops · ‎04-27-2006

B237_NCC_6506> (enable) sh ver

WS-C6506 Software, Version NmpSW: 7.6(12)

NMP S/W compiled on May 4 2005, 14:59:23

System Bootstrap Version: 5.3(1)

System Boot Image File is 'bootflash:cat6000-supk8.7-6-12.bin'

System Configuration register is 0xf

Hardware Version: 3.0 Model: WS-C6506 Serial #: SAL082295FL

PS1 Module: WS-CAC-1300W Serial #: ABC08272240

PS2 Module: WS-CAC-1300W Serial #: SNI06330755

Mod Port Model Serial # Versions

--- ---- ------------------- ----------- --------------------------------------

1 2 WS-X6K-SUP1A-2GE SAL063244HY Hw : 7.3

Fw : 5.3(1)

Fw1: 5.4(2)

Sw : 7.6(12)

Sw1: 7.6(12)

WS-F6K-PFC SAL0632437Y Hw : 2.0

Sw :

WS-X6K-SUP1A-2GE SAL063244HY Hw : 7.3

Sw :

2 48 WS-X6148-RJ-45 SAL0824A6CZ Hw : 3.1

Fw : 5.4(2)

Sw : 7.6(12)

3 48 WS-X6148-RJ-45 SAL0827BESC Hw : 3.1

Fw : 5.4(2)

Sw : 7.6(12)

4 48 WS-X6348-RJ-45 SAL06313ZNA Hw : 6.1

Fw : 5.4(2)

Sw : 7.6(12)

DRAM FLASH NVRAM

Module Total Used Free Total Used Free Total Used Free

------ ------- ------- ------- ------- ------- ------- ----- ----- -----

1 130944K 49472K 81472K 16384K 10961K 5423K 512K 122K 390K

Uptime is 0 day, 1 hour, 6 minutes

Bobby Thekkekandam · ‎04-27-2006

Hi,

Thanks for posting the outputs. The stack itself seems to be corrupted, so I wasn't able to extract the functions that were on the stack at the time of crash. However, the switch crashed due to a TLB Exception.

A TLB exception occurs when the CPU tries to access an invalid virtual address. Most often, this occurs when accessing data ( as opposed to code/instructions). The cause is usually a software bug.

I was able to find a number of TLB related crashes that affect 7.6(12).

CSCsc01175 - TLB Exception - PID = ssh_flush

CSCsb60960 - Crash in dot1x when one of the data structure not initalized

CSCsb24936 - Standby crashes upon switchover with snmpget caqAggPolicerPackets

There have been several maintenance releases since 7.6(!2)-- now up to 7.6(17). I would recommend moving to that version to eliminate any of the various bugs that exist in 7.6(12). Here are the release notes for the 7.x releases, I'd also check that before upgrading for any caveats.

If for some reason, that doesn't mitigate the problems, I'd recommend opening a TAC SR do do some in-depth troubleshooting.

HTH,

Bobby

*Please rate helpful posts.