What is this error mean? %PLATFORM_IPC-3-COMMON: Unknown IPC message type

[Rudy R.]

Errors started popping up on the console (making it difficult to even use console as they come in pretty frequently).  I have never seen them before.  I have no IPC configurations on this switch.

Info on IOS:

Cisco IOS Software, C3560E Software (C3560E-IPBASEK9-M), Version 15.0(1)SE, RELEASE SOFTWARE (fc1)

BOOTLDR: C3560E Boot Loader (C3560X-HBOOT-M) Version 12.2(53r)SE1, RELEASE SOFTWARE (fc1)

License Level: ipbase

cisco WS-C3560X-24 (PowerPC405) processor (revision A0) with 262144K bytes of memory.

Here are the errors:

Dec  2 16:27:52.793: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 58674 size 126

-Traceback= 5370E8z 21EF97Cz 23E20C0z 23E5A38z 23E5E1Cz 22D7E3Cz 22D7DE4z 22DDA0Cz 22DDF14z 22D7EECz 1B2E650z 227DF8Cz 2289730z 220E69Cz 22179C8z 286A948z

Dec  2 16:27:56.559: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 58674 size 126

-Traceback= 5370E8z 21EF97Cz 23E20C0z 23E5A38z 23E5E1Cz 22D7E3Cz 22D7DE4z 22DDA0Cz 22DDF14z 22D7EECz 1B23D08z 28653A0z 2862940z

Dec  2 16:27:59.042: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 58674 size 126

-Traceback= 5370E8z 21EF97Cz 23E20C0z 23E5A38z 23E5E1Cz 22D7E3Cz 22D7DE4z 22DDA0Cz 22DDF14z 22D7EECz 2865750z 2865410z 2862940z

Dec  2 16:28:02.791: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 58674 size 126

-Traceback= 5370E8z 21EF97Cz 23E20C0z 23E5A38z 23E5E1Cz 22D7E3Cz 22D7DE4z 22DDA0Cz 22DDF14z 22D7EECz 2864B14z 286492Cz 2866054z 2862940z

Dec  2 16:28:04.209: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 58674 size 126

-Traceback= 5370E8z 21EF97Cz 23E20C0z 23E5A38z 23E5E1Cz 22D7E3Cz 22D7DE4z 22DDA0Cz 22DDF14z 22D7EECz 1B232F4z

Dec  2 16:28:08.084: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 58674 size 126

-Traceback= 5370E8z 21EF97Cz 23E20C0z 23E5A38z 23E5E1Cz 22D7E3Cz 22D7DE4z 22DDA0Cz 22DDF14z 22D7EECz 2865300z 2864B80z 2862940z

Dec  2 16:28:11.053: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 58674 size 126

-Traceback= 5370E8z 21EF97Cz 23E20C0z 23E5A38z 23E5E1Cz 22D7E3Cz 22D7DE4z 22DDA0Cz 22DDF14z 22D7EECz 1ACA310z

Dec  2 16:28:14.845: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 58674 size 126

-Traceback= 5370E8z 21EF97Cz 23E20C0z 23E5A38z 23E5E1Cz 22D7E3Cz 22D7DE4z 22DDA0Cz 22DDF14z 22D7EECz 2862BA4z

Dec  2 16:28:18.401: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 58674 size 126

-Traceback= 5370E8z 21EF97Cz 23E20C0z 23E5A38z 23E5E1Cz 22D7E3Cz 22D7DE4z 22DDA0Cz 22DDF14z 22D7EECz 1ACD2E0z 1ACD2D0z 28653A0z 2862940z

Dec  2 16:28:21.060: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 58674 size 126

-Traceback= 5370E8z 21EF97Cz 23E20C0z 23E5A38z 23E5E1Cz 22D7E3Cz 22D7DE4z 22DDA0Cz 22DDF14z 22D7EECz 2708DE0z 22D6E24z 23D3D88z 1DF4250z 239BB18z 239BBE8z

Dec  2 16:28:25.347: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 58674 size 126

-Traceback= 5370E8z 21EF97Cz 23E20C0z 23E5A38z 23E5E1Cz 22D7E3Cz 22D7DE4z 22DDA0Cz 22DDF14z 22D7EECz 1ACD320z 28653A0z 2862940z

Dec  2 16:28:28.761: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 58674 size 126

-Traceback= 5370E8z 21EF97Cz 23E20C0z 23E5A38z 23E5E1Cz 22D7E3Cz 22D7DE4z 22DDA0Cz 22DDF14z 22D7EECz 286492Cz 286492Cz 286540Cz 2862940z

Dec  2 16:28:31.059: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 58674 size 126

-Traceback= 5370E8z 21EF97Cz 23E20C0z 23E5A38z 23E5E1Cz 22D7E3Cz 22D7DE4z 22DDA0Cz 22DDF14z 22D7EECz 285EF44z

Dec  2 16:28:37.484: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 58674 size 126

-Traceback= 5370E8z 21EF97Cz 23E20C0z 23E5A38z 23E5E1Cz 22D7E3Cz 22D7DE4z 22DDA0Cz 22DDF14z 22D7EECz 2865320z 2864B80z 2862940z

Dec  2 16:28:38.759: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 58674 size 126

-Traceback= 5370E8z 21EF97Cz 23E20C0z 23E5A38z 23E5E1Cz 22D7E3Cz 22D7DE4z 22DDA0Cz 22DDF14z 22D7EECz 1B23328z

Dec  2 16:28:49.940: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 58674 size 126

-Traceback= 5370E8z 21EF97Cz 23E20C0z 23E5A38z 23E5E1Cz 22D7E3Cz 22D7DE4z 22DDA0Cz 22DDF14z 22D7EECz 22DE160z 2398D20z 22D6E24z 23D3D7Cz 1DE1F30z 1DF21A4z

Dec  2 16:28:57.355: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 58674 size 126

-Traceback= 5370E8z 21EF97Cz 23E20C0z 23E5A38z 23E5E1Cz 22D7E3Cz 22D7DE4z 22DDA0Cz 22DDF14z 22D7EECz 285EE1Cz

Dec  2 16:29:03.588: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 58674 size 126

-Traceback= 5370E8z 21EF97Cz 23E20C0z 23E5A38z 23E5E1Cz 22D7E3Cz 22D7DE4z 22DDA0Cz 22DDF14z 22D7EECz 28652E0z 2864B80z 2862940z

Dec  2 16:29:08.033: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 58674 size 126

-Traceback= 5370E8z 21EF97Cz 23E20C0z 23E5A38z 23E5E1Cz 22D7E3Cz 22D7DE4z 22DDA0Cz 22DDF14z 22D7EECz 1B9F374z 286492Cz 107D0A8z 107F09Cz 107F834z 1919524z

Thanks for any pointers on how to debug this.

Rudy

[Rudy R.]

Here is some more info... I can shutdown a port and it stops the IPC messages (of type unknown)...  Note 'term monitor' is on.

piojo(config)#interface

Dec  2 16:40:08.642: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 58674 size 126

-Traceback= 5370E8z 21EF97Cz 23E20C0z 23E5A38z 23E5E1Cz 22D7E3Cz 22D7DE4z 22DDA0Cz 22DDF14z 22D7EECz 107EFF4z 107D100z 107F114z 107F834z 1919524z 1ACA338zte

piojo(config)#interface tenGigabitEthernet

Dec  2 16:40:09.775: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 58674 size 126

-Traceback= 5370E8z 21EF97Cz 23E20C0z 23E5A38z 23E5E1Cz 22D7E3Cz 22D7DE4z 22DDA0Cz 22DDF14z 22D7EECz 28657DCz 2865410z 2862940z1/1

piojo(config)#interface tenGigabitEthernet 1/1

piojo(config-if)#

piojo(config-if)#

piojo(config-if)#shut

Dec  2 16:40:12.769: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 58674 size 126

-Traceback= 5370E8z 21EF97Cz 23E20C0z 23E5A38z 23E5E1Cz 22D7E3Cz 22D7DE4z 22DDA0Cz 22DDF14z 22D7EECz 486B40zdown

piojo(config-if)#

Dec  2 16:40:15.445: %IFDAMP-5-UPDOWN: interface TenGigabitEthernet1/1 update IP Routing state to DOWN, interface is not suppressed

Dec  2 16:40:15.445: %OSPF-5-ADJCHG: Process 1, Nbr 10.8.254.107 on TenGigabitEthernet1/1 from FULL to DOWN, Neighbor Down: Interface down or detached

Dec  2 16:40:15.445: %LINK-5-CHANGED: Interface TenGigabitEthernet1/1, changed state to administratively down

Dec  2 16:40:16.452: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/1, changed state to down

piojo(config-if)#

piojo(config-if)#

piojo(config-if)#no shutdown

piojo(config-if)#

Dec  2 16:40:55.431: %LINK-3-UPDOWN: Interface TenGigabitEthernet1/1, changed state to down

Dec  2 16:40:55.540: %LINK-3-UPDOWN: Interface TenGigabitEthernet1/1, changed state to up

Dec  2 16:40:55.548: %IFDAMP-5-UPDOWN: interface TenGigabitEthernet1/1 update IP Routing state to UP, interface is not suppressed

Dec  2 16:40:56.547: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/1, changed state to up

piojo(config-if)#

Dec  2 16:41:41.481: %OSPF-5-ADJCHG: Process 1, Nbr 10.8.254.107 on TenGigabitEthernet1/1 from LOADING to FULL, Loading Done

Dec  2 16:42:32.883: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 58674 size 126

-Traceback= 5370E8z 21EF97Cz 23E20C0z 23E5A38z 23E5E1Cz 22D7E3Cz 22D7DE4z 22DDA0Cz 22DDF14z 22D7EECz 286605Cz 2866054z 2862940z

Dec  2 16:42:33.990: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 58674 size 126

-Traceback= 5370E8z 21EF97Cz 23E20C0z 23E5A38z 23E5E1Cz 22D7E3Cz 22D7DE4z 22DDA0Cz 22DDF14z 22D7EECz 28652D4z 2864B80z 2862940z

Dec  2 16:42:35.853: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 58674 size 126

-Traceback= 5370E8z 21EF97Cz 23E20C0z 23E5A38z 23E5E1Cz 22D7E3Cz 22D7DE4z 22DDA0Cz 22DDF14z 22D7EECz 2865330z 2865410z 2862940z

Dec  2 16:42:43.771: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 58674 size 126

-Traceback= 5370E8z 21EF97Cz 23E20C0z 23E5A38z 23E5E1Cz 22D7E3Cz 22D7DE4z 22DDA0Cz 22DDF14z 22D7EECz 2864B0Cz 286492Cz 286540Cz 2862940z

Dec  2 16:42:45.851: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 58674 size 126

-Traceback= 5370E8z 21EF97Cz 23E20C0z 23E5A38z 23E5E1Cz 22D7E3Cz 22D7DE4z 22DDA0Cz 22DDF14z 22D7EECz 285EE0Cz

Dec  2 16:42:53.996: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 58674 size 126

-Traceback= 5370E8z 21EF97Cz 23E20C0z 23E5A38z 23E5E1Cz 22D7E3Cz 22D7DE4z 22DDA0Cz 22DDF14z 22D7EECz 191954Cz 1919524z 1ACA338z 1ACD2D0z 28653A0z 2862940z

Dec  2 16:42:55.850: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 58674 size 126

-Traceback= 5370E8z 21EF97Cz 23E20C0z 23E5A38z 23E5E1Cz 22D7E3Cz 22D7DE4z 22DDA0Cz 22DDF14z 22D7EECz 1ACD2CCz 1ACD2BCz 28653A0z 2862940zexit

piojo(config)#exit

Here are the interface configs for the two switches (both 3560X with same IOS version):

interface TenGigabitEthernet1/1

description To MOSCA

no switchport

dampening

ip address 10.8.254.125 255.255.255.252

ip ospf authentication message-digest

ip ospf message-digest-key 1 md5 XXXXX

logging event link-status

carrier-delay msec 0

end

And on the other side, there is a similar config:

interface TenGigabitEthernet1/1

description To PIOJO

no switchport

dampening

ip address 10.8.254.126 255.255.255.252

ip ospf authentication message-digest

ip ospf message-digest-key 1 md5 XXXXX

logging event link-status

carrier-delay msec 0

end

[Amit Singh]

Please could you paste the following from the switch :

1. Show version

2. Dir flash:

3. If you have any crachinfo file in the flash, please paste the output as well. command : more flash:crashinfo/

-amit singh

[Rudy R.]

piojo#show version

Cisco IOS Software, C3560E Software (C3560E-IPBASEK9-M), Version 15.0(1)SE, RELEASE SOFTWARE (fc1)

Technical Support: http://www.cisco.com/techsupport

Copyright (c) 1986-2011 by Cisco Systems, Inc.

Compiled Wed 20-Jul-11 09:21 by prod_rel_team

ROM: Bootstrap program is C3560E boot loader

BOOTLDR: C3560E Boot Loader (C3560X-HBOOT-M) Version 12.2(53r)SE1, RELEASE SOFTWARE (fc1)

piojo uptime is 4 weeks, 10 hours, 58 minutes

System returned to ROM by power-on

System restarted at 23:02:02 PDT Thu Nov 3 2011

System image file is "flash:/c3560e-ipbasek9-mz.150-1.SE/c3560e-ipbasek9-mz.150-1.SE.bin"

This product contains cryptographic features and is subject to United

States and local country laws governing import, export, transfer and

use. Delivery of Cisco cryptographic products does not imply

third-party authority to import, export, distribute or use encryption.

Importers, exporters, distributors and users are responsible for

compliance with U.S. and local country laws. By using this product you

agree to comply with applicable laws and regulations. If you are unable

to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:

http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to

export@cisco.com.

License Level: ipbase

License Type: Permanent

Next reload license Level: ipbase

cisco WS-C3560X-24 (PowerPC405) processor (revision A0) with 262144K bytes of memory.

Processor board ID FDO1451P05J

Last reset from power-on

4 Virtual Ethernet interfaces

1 FastEthernet interface

28 Gigabit Ethernet interfaces

2 Ten Gigabit Ethernet interfaces

The password-recovery mechanism is enabled.

512K bytes of flash-simulated non-volatile configuration memory.

Base ethernet MAC Address       : 50:3D:E5:32:82:80

Motherboard assembly number     : 73-12554-04

Motherboard serial number       : FDO145118KQ

Model revision number           : A0

Motherboard revision number     : A0

Model number                    : WS-C3560X-24T-S

Daughterboard assembly number   : 800-32786-01

Daughterboard serial number     : FDO14510M3M

System serial number            : FDO1451P05J

Top Assembly Part Number        : 800-31331-01

Top Assembly Revision Number    : A0

Version ID                      : V01

CLEI Code Number                : COMJU00ARA

Hardware Board Revision Number  : 0x02

Switch Ports Model              SW Version            SW Image                

------ ----- -----              ----------            ----------              

*    1 30    WS-C3560X-24       15.0(1)SE             C3560E-IPBASEK9-M       

Configuration register is 0xF

piojo#dir flash:

Directory of flash:/

    2  -rwx        6051   Dec 1 2011 20:07:40 -08:00  config.text

    3  -rwx        3820   Dec 1 2011 20:07:40 -08:00  private-config.text

  461  -rwx        2844   Dec 1 2011 20:02:31 -08:00  vlan.dat

    4  -rwx        3096   Dec 1 2011 20:07:40 -08:00  multiple-fs

    5  drwx         512  Feb 28 1993 16:15:37 -08:00  c3560e-ipbasek9-mz.150-1.SE

57671680 bytes total (34187776 bytes free)

[Rudy R.]

I tried removing dampening, logging event link-status, and carrier-delay msec 0; however, the errors continue.

Is there some global command that will stop IPC messages?  I don't have any master/slave setups.

[acampbell]

Hi,

As you pointed out these are MASTER/SLAVE alerts

You are not running a stacked setup.

I think you should try upgrading the IOS to

15.0.1SE1

If this does not solve the issue then

collect a show tech and raise a TAC case

Regards

Alex

[Ron Arroyo]

I am seeing the same error in my logs.  I upgraded to

Cisco IOS Software, C3750E Software (C3750E-UNIVERSALK9-M), Version 15.0(1)SE1, RELEASE SOFTWARE (fc3)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2011 by Cisco Systems, Inc.
Compiled Thu 01-Dec-11 14:06 by prod_rel_team

ROM: Bootstrap program is C3750E boot loader
BOOTLDR: C3750E Boot Loader (C3750X-HBOOT-M) Version 12.2(53r)SE1, RELEASE SOFTWARE (fc1)

but the errors are still logging.  Atleast now with SE1, switches in my stack are no longer crashing.  Submitted all info to TAC.  The last error recieved was the following:

Dec 12 09:53:59: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 25738 size 130

-Traceback= 545B9Cz 21DD084z 23CCA04z 23D037Cz 23D0760z 22C77A4z 22C774Cz 22CD380z 22CD888z 22C7854z 1B154A8z F4238z 284606Cz 2846B4Cz 2844080z

[gene.uhl]

Did they ever discover a fix????   I have the same issue.  Have a Nexus with two M type cards, 10G.   Have a 3750 stack with two 3750G and two 3750X stacked with the latest FW 15.x    Have a portchannel defined between the Nexus and Stack using two 10G ports on the stack and two 10G ports on the Nexus  L3 portchannel.  When I bring one or both ports up I get:

*Mar  3 22:06:40: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 7591 size 61 (CS3750X-IT01-4)

-Traceback= 545B9Cz 21DD084z 23CCA04z 23D037Cz 23D0760z 22C77A4z 22C774Cz 22CD380z 22CD888z 22C7854z 28477BCz 2847794z 2844080z (CS3750X-IT01-4)

*Mar  3 22:06:45: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 7591 size 61 (CS3750X-IT01-4)

-Traceback= 545B9Cz 21DD084z 23CCA04z 23D037Cz 23D0760z 22C77A4z 22C774Cz 22CD380z 22CD888z 22C7854z 28477BCz 2847794z 2844080z (CS3750X-IT01-4)

*Mar  3 22:06:50: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 7591 size 60 (CS3750X-IT01-4)

-Traceback= 545B9Cz 21DD084z 23CCA04z 23D037Cz 23D0760z 22C77A4z 22C774Cz 22CD380z 22CD888z 22C7854z 28477BCz 2847794z 2844080z (CS3750X-IT01-4)

*Mar  3 22:06:55: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 7591 size 60 (CS3750X-IT01-4)

-Traceback= 545B9Cz 21DD084z 23CCA04z 23D037Cz 23D0760z 22C77A4z 22C774Cz 22CD380z 22CD888z 22C7854z 28477BCz 2847794z 2844080z (CS3750X-IT01-4)

*Mar  3 22:07:01: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 7591 size 60 (CS3750X-IT01-4)

-Traceback= 545B9Cz 21DD084z 23CCA04z 23D037Cz 23D0760z 22C77A4z 22C774Cz 22CD380z 22CD888z 22C7854z 28477BCz 2847794z 2844080z (CS3750X-IT01-4) *Mar  3 22:06:40: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 7591 size 61 (CS3750X-IT01-4)
-Traceback= 545B9Cz 21DD084z 23CCA04z 23D037Cz 23D0760z 22C77A4z 22C774Cz 22CD380z 22CD888z 22C7854z 28477BCz 2847794z 2844080z (CS3750X-IT01-4)
*Mar  3 22:06:45: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 7591 size 61 (CS3750X-IT01-4)
-Traceback= 545B9Cz 21DD084z 23CCA04z 23D037Cz 23D0760z 22C77A4z 22C774Cz 22CD380z 22CD888z 22C7854z 28477BCz 2847794z 2844080z (CS3750X-IT01-4)
*Mar  3 22:06:50: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 7591 size 60 (CS3750X-IT01-4)
-Traceback= 545B9Cz 21DD084z 23CCA04z 23D037Cz 23D0760z 22C77A4z 22C774Cz 22CD380z 22CD888z 22C7854z 28477BCz 2847794z 2844080z (CS3750X-IT01-4)
*Mar  3 22:06:55: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 7591 size 60 (CS3750X-IT01-4)
-Traceback= 545B9Cz 21DD084z 23CCA04z 23D037Cz 23D0760z 22C77A4z 22C774Cz 22CD380z 22CD888z 22C7854z 28477BCz 2847794z 2844080z (CS3750X-IT01-4)
*Mar  3 22:07:01: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 7591 size 60 (CS3750X-IT01-4)
-Traceback= 545B9Cz 21DD084z 23CCA04z 23D037Cz 23D0760z 22C77A4z 22C774Cz 22CD380z 22CD888z 22C7854z 28477BCz 2847794z 2844080z (CS3750X-IT01-4)

Have no idea what this means, so I have to shut down those links....

ANy idea....

[Ron Arroyo]

I have not recieved a fix.  Cisco said that my outputs were too generic to isolate and havent proceed further.  I am currently still on 15.0(1)SE1 and plan to upgrade to SE2.  I have almost the same config. 3750X stack L2 portchannel to VPC port channel accross two Nexus 7Ks.

I will update my TAC case and see if anything has changed.  I am not seeing any network issues other than the logging of this error on the 3750X.

Jan 27 10:01:16: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 25738 size 130

-Traceback= 545B9Cz 21DD084z 23CCA04z 23D037Cz 23D0760z 22C77A4z 22C774Cz 22CD380z 22CD888z 22C7854z 22CD260z

Jan 27 10:01:19: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 25738 size 130

-Traceback= 545B9Cz 21DD084z 23CCA04z 23D037Cz 23D0760z 22C77A4z 22C774Cz 22CD380z 22CD888z 22C7854z 22C6708z 22C678Cz 22C678Cz 23D25B4z 23D3514z 265EA40z

Jan 27 10:01:51: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 25738 size 130

-Traceback= 545B9Cz 21DD084z 23CCA04z 23D037Cz 23D0760z 22C77A4z 22C774Cz 22CD380z 22CD888z 22C7854z 22C6728z

Jan 27 10:14:35: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 25738 size 130

-Traceback= 545B9Cz 21DD084z 23CCA04z 23D037Cz 23D0760z 22C77A4z 22C774Cz 22CD380z 22CD888z 22C7854z 22C66F8z 238358Cz 22C678Cz 23BE5ECz 1DD4080z 1DE44E8z Jan 27 10:01:16: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 25738 size 130
-Traceback= 545B9Cz 21DD084z 23CCA04z 23D037Cz 23D0760z 22C77A4z 22C774Cz 22CD380z 22CD888z 22C7854z 22CD260z
Jan 27 10:01:19: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 25738 size 130
-Traceback= 545B9Cz 21DD084z 23CCA04z 23D037Cz 23D0760z 22C77A4z 22C774Cz 22CD380z 22CD888z 22C7854z 22C6708z 22C678Cz 22C678Cz 23D25B4z 23D3514z 265EA40z
Jan 27 10:01:51: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 25738 size 130
-Traceback= 545B9Cz 21DD084z 23CCA04z 23D037Cz 23D0760z 22C77A4z 22C774Cz 22CD380z 22CD888z 22C7854z 22C6728z
Jan 27 10:14:35: %PLATFORM_IPC-3-COMMON: Unknown IPC message type 25738 size 130
-Traceback= 545B9Cz 21DD084z 23CCA04z 23D037Cz 23D0760z 22C77A4z 22C774Cz 22CD380z 22CD888z 22C7854z 22C66F8z 238358Cz 22C678Cz 23BE5ECz 1DD4080z 1DE44E8z

[Ron Arroyo]

This is what Cisco TAC initially isolated the errors of this problem.  This resolved my switches from crashing once I upgraded to SE1.  I am still seeing the errors in the log however there is no crashes or impact to traffic at this time.

CSCtr31957   IOSD crash @ ipc_check_qtime_process with RP CPU 99 percent

Symptom:

IPC use ipc_check_qtime_process() to handle IPC messages. So all the platform/release having this API with fix of CSCto09059 may hit this issue very frequent. If fix of CSCto09059 is not there then frequency will be very less and even we may not see the issue also. For other details please refer enclosures "RCA" and "Eng-note".

Traffic model is 130 CPS & 180 seconds CHT SIP ipv4 traffic and enable both of XML and radius billing.

Try to let SBC RP CPU usage reaching 99 percent, and below replication congestion observed on the RP console:

The system has become congested. Dialog creating or out of dialog messages may get rejected.

   Congestion statistics:

   Total accepted packets     = 636359

   Total rejected packets     = 5383

   Current congestion level   = 0

   Current queue length       = 2

   Time since last reset     = 45454s

   Current oldest element age = 13ms

   Max item delay             = 1393ms

Actions when problem was observed:

After running about 6 hours, find below traceback and active RP crash and reloads:

UNIX-EXT-SIGNAL: Segmentation fault(11), Process = IPC Check Queue Time Process

Conditions:

It is a race condition. ipc_check_qtime_process() is processing some ipc message and getting ack for that message at interrupt level, getting ack means that message is no more valid because interrupt handler returns that message to message cache so accessing the same message in process context(after coming out of Interrupt Handler) is causing crash.

Workaround:

No workaround

The release that will contain the fix for this bug will be 15.0(1)SE1

[gene.uhl]

Thanks!!   Don't put any confidense in SE2 as I am running that now.  The only issue I have to battle now is the Twinax cables I am running are not supported in my N7K-M132XP-12   I did purchase the correct SFP-H10GB-ACU7M but I guess I did not get Version 2.  I hope they don't latch on to that as the issue.  Your setup is a little more complicated witht eh VPC between two boxes, but nontheless should work. I am just going to one chasis but two different cards for redundancy so a basic PC configuration.   

I wonder if I swap out the Twinax and use standard SFP+ will I get the same errors...may be going down the wrong path here....I will update if I hear anything too...

Thanks

Gene

[Ron Arroyo]

Thanks for the info.   I have N7K-M108X2-12L with X2-10GB-LRM 10GBASE-LRM X2 Module on one side and 3750X with

SFP-10G-LRM= 10GBASE-LRM SFP Module on the other.

[gene.uhl]

Well, that blows my SFP theory,  Thanks  :-))

I just received a reponse and the knucklehead at Cisco said he thinks it is a hardware problem and wants to replace teh 3750 so I need to send him the info on it.     I love when they don't read all the info I sent them....as I asked which one, there are 4 in the stack.   I asked him to escalate the call as I don't see it being a hardware problem as if I disable both ports, then enable one at a time either to the top 3750x or the bottom 3750x I get the same errors.  We shall see...

[Ron Arroyo]

Right there with you.  I am somewhat thinking it may be a hw issue on the 3750X side.  SE1 didnt really resolve the issue, it just prevented the SBC RP CPU from hitting 99% and crashing the switch.  I had 2 of 4 switches crash in my 3750X stack.  one with the 10G and one without. 

[Rudy R.]

Cisco needs to get an answer out on this one...  If I shut down my te1/1 the errors go away (and everything has to route over a 1Gbps interface), and when enabled, the errors return.

Can anyone (who has the problem) reply and let us know if 15.0(1)SE2 fixes the issue?  Also, the 12.x -> 15.0 upgrade was 30 minutes... do the bug fixes of  '15.0(1)SE bugs' require a 30 minute upgrade when moving to SE2?