cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1797
Views
5
Helpful
1
Replies

3750X crash

adamsutton75
Level 1
Level 1

Hi,

We have a 3750X stack of two switches running software 12.2(58)SE2 which has crashed after logging memory allocation errors.  We have this software deployed on numerous other stacks but they are Layer 2 only.  This particular stack is in a development lab and configured for Layer 3 for about a dozen VLANs with minimal traffic.

As such being in a dev lab this switch doesn't have support, hence posting here for any possible answers while we look to get cover and log a call.  Our concern being is this error indicative of a hardware fault, or a software bug that is only relavent to Layer 3 configuration.  A concern as elsewhere in the network we are in the process of commissioning another 3750X stack with this software which will be live and running Layer 3 IP routing.

The error logged is;

              

*Aug 13 18:06:13.367: %SYS-2-MALLOCFAIL: Memory allocation of 65536 bytes failed from 0x1A72728, alignment 0

Pool: Processor  Free: 219960  Cause: Memory fragmentation

Alternate Pool: None  Free: 0  Cause: No Alternate pool

-Process= "RedEarth Rx Manager", ipl= 0, pid= 86

-Traceback= 533568z 1A1E294z 1A2F910z 1A7272Cz 1A1DF0Cz 1A2A484z 2838518z 1C7C798z 21DC4CCz 21DA52Cz 24E4404z 24E49A0z 26D5768z 26CF5DCz

*Aug 13 18:06:43.382: %SYS-2-MALLOCFAIL: Memory allocation of 65536 bytes failed from 0x1A72728, alignment 0

Pool: Processor  Free: 207688  Cause: Memory fragmentation

Alternate Pool: None  Free: 0  Cause: No Alternate pool

-Process= "hrpc <- response", ipl= 0, pid= 94

*Aug 13 18:07:13.413: %SYS-2-MALLOCFAIL: Memory allocation of 65536 bytes failed from 0x1A72728, alignment 0

Pool: Processor  Free: 196592  Cause: Memory fragmentation

Alternate Pool: None  Free: 0  Cause: No Alternate pool

-Process= "hrpc <- response", ipl= 0, pid= 94

-Traceback= 533568z 1A1E294z 1A2F910z 1A7272Cz 1A1DF0Cz 1A2A484z 2838518z 207BE9Cz 207B490z 21825E4z 21823C0z 2182B2Cz 26D5768z 26CF5DCz

*Aug 13 18:07:41.280: %PLATFORM_RPC-0-RESOURCE_CRASH: System is unable to alloc memory for RPC

*Aug 13 18:06:13.367: %SYS-2-MALLOCFAIL: Memory allocation of 65536 bytes failed from 0x1A72728, alignment 0
Pool: Processor  Free: 219960  Cause: Memory fragmentation
Alternate Pool: None  Free: 0  Cause: No Alternate pool
-Process= "RedEarth Rx Manager", ipl= 0, pid= 86
-Traceback= 533568z 1A1E294z 1A2F910z 1A7272Cz 1A1DF0Cz 1A2A484z 2838518z 1C7C798z 21DC4CCz 21DA52Cz 24E4404z 24E49A0z 26D5768z 26CF5DCz
*Aug 13 18:06:43.382: %SYS-2-MALLOCFAIL: Memory allocation of 65536 bytes failed from 0x1A72728, alignment 0
Pool: Processor  Free: 207688  Cause: Memory fragmentation
Alternate Pool: None  Free: 0  Cause: No Alternate pool
-Process= "hrpc <- response", ipl= 0, pid= 94

Many thanks

Adam

1 Reply 1

Partha Dasgupta
Cisco Employee
Cisco Employee

Hello Adam,

As per the logs pasted , it looks like we are hitting software defect

CSCtt94924
Switch Crashes with "Unable to allocate memory for RPC" Error messages
Symptom: -- 3750 running 12.2(58)SE2 crashes repeatedly with "%PLATFORM_RPC-0-RESOURCE_CRASH: System is unable to alloc memory for RPC" Error messages. Conditions: -- sh proc mem showing *Malloclite* holding higher value and the same is incremented when checking on constant intervals.Workaround:

which is duplicate to another software defect

CSCuc18521 Bug Details

Memory Leak by "MallocLite" process

Symptom:
Memory leak in process pool memory - first seen as caused by *MallocLite* process , after disabling "no memory lite" by "hrpc < - response"

Conditions:

Occurs on stacked switches, ie, 3750, 3750e, 3750x, 2960S with 58.SE and later images
DENY configu in ACL with "log" option plus traffic

Workaround:
55.SE images have no such issue or
Remove "log" keyword in deny acl. A stack reload after removing the log keyword is suggested.

Regards

Partha

Review Cisco Networking products for a $25 gift card