Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements
Webcast-Catalyst9k
New Member

RedEarth Tx Mana - High CPU usage on Cisco 3560-X

Hi Everyone,

We are doing a testing on Cisco 3560-X switch by generating 100M traffic. Now CPU usage of RedEarth Tx Mana

is high around 40%.

We have tweak the SDM templates for routing.

Can any one help to sort this out ??

Cisco IOS Software, C3560E Software (C3560E-UNIVERSALK9-M), Version 12.2(58)SE2, RELEASE SOFTWARE (fc1)

Technical Support: http://www.cisco.com/techsupport

Copyright (c) 1986-2011 by Cisco Systems, Inc.

Compiled Thu 21-Jul-11 01:34 by prod_rel_team

ROM: Bootstrap program is C3560E boot loader

BOOTLDR: C3560E Boot Loader (C3560X-HBOOT-M) Version 12.2(53r)SE2, RELEASE SOFTWARE (fc1)

NMC-0.77-3560 uptime is 1 day, 1 hour, 42 minutes

System returned to ROM by power-on

System restarted at 09:34:34 LKR Thu Aug 22 2013

System image file is "flash:/c3560e-universalk9-mz.122-58.SE2.bin"

This product contains cryptographic features and is subject to United

States and local country laws governing import, export, transfer and

use. Delivery of Cisco cryptographic products does not imply

third-party authority to import, export, distribute or use encryption.

Importers, exporters, distributors and users are responsible for

compliance with U.S. and local country laws. By using this product you

agree to comply with applicable laws and regulations. If you are unable

to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:

http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to

export@cisco.com.

License Level: ipservices

License Type: Evaluation

Next reload license Level: ipservices

cisco WS-C3560X-24 (PowerPC405) processor (revision A0) with 262144K bytes of memory.

Processor board ID FDO1622R1X9

Last reset from power-on

1 Virtual Ethernet interface

1 FastEthernet interface

28 Gigabit Ethernet interfaces

2 Ten Gigabit Ethernet interfaces

The password-recovery mechanism is enabled.

512K bytes of flash-simulated non-volatile configuration memory.

Base ethernet MAC Address       : D8:67:D9:CA:0D:80

Model revision number           : A0

Motherboard revision number     : C0

Model number                    : WS-C3560X-24T-L

Top Assembly Revision Number    : E0

Version ID                      : V02

CLEI Code Number                : COMJU00ARB

Hardware Board Revision Number  : 0x03

Switch Ports Model              SW Version            SW Image                

------ ----- -----              ----------            ----------              

*    1 30    WS-C3560X-24       12.2(58)SE2           C3560E-UNIVERSALK9-M    

Configuration register is 0xF


Everyone's tags (1)
10 REPLIES
Hall of Fame Super Gold

RedEarth Tx Mana - High CPU usage on Cisco 3560-X

Version 12.2(58)SE2

This is most likely your culprit.

Use 12.2(55)SE8 if you are running Layer 3, PoE and/or 802.1x.

You could also try 15.0(2)SE4 although I'd much prefer 12.2(55)SE8.

New Member

Re: RedEarth Tx Mana - High CPU usage on Cisco 3560-X

Hi Leo,

Is this one will work ?

c3560e-universalk9-tar.150-2.SE4

Hall of Fame Super Gold

RedEarth Tx Mana - High CPU usage on Cisco 3560-X

Without knowing what you've got, give it a try.

New Member

Re: RedEarth Tx Mana - High CPU usage on Cisco 3560-X

I have tried with the new IOS , still the same issue.

Hall of Fame Super Gold

RedEarth Tx Mana - High CPU usage on Cisco 3560-X

I have tried with the new IOS , still the same issue

Do you have the following:  Layer 3 operation, PoE or 802.1x.

If you have any one of the above, then try 12.2(55)SE8.

New Member

Re: RedEarth Tx Mana - High CPU usage on Cisco 3560-X

Is there any other work round to tackle the RedEarth Tx Mana high utilization issue ??

New Member

RedEarth Tx Mana - High CPU usage on Cisco 3560-X

I have same issue with our  WS-C3750X-48 running  15.0(2)SE3 [c3750e-universalk9npe-mz.150-2.SE2] IOS.

------------------show process memory-------------------

PID TTY  Allocated      Freed    Holding    Getbufs    Retbufs Process

!

  89   0          0          0       7200          0          0 RedEarth I2C dri

  90   0      34872      34816       7160          0          0 RedEarth Tx Mana

  91   0      28600      35368       7160       2268          0 RedEarth Rx Mana

!

------------------ show process cpu ------------------

PID Runtime(ms)     Invoked      uSecs   5Sec   1Min   5Min TTY Process

!

  89      545506     2571645        212  0.15%  0.20%  0.17%   0 RedEarth I2C dri

  90     9762656     1715940       5689  2.24%  2.46%  2.55%   0 RedEarth Tx Mana

  91       93631     1285767         72  0.00%  0.01%  0.00%   0 RedEarth Rx Mana

!

Please advice!!!


New Member

RedEarth Tx Mana - High CPU usage on Cisco 3560-X

This HIGH CPU Usage might be due to TCAM Exhaustion. In our case we saw the same in our 3560 used in Router Mode with BGP + OSPF.

Use this command to check the utilization of TCAM.

show platform tcam utilization

You will get similar Output :

sh platform tcam utilization

CAM Utilization for ASIC# 0                      Max            Used

                                             Masks/Values    Masks/values

Unicast mac addresses:                       2268/2268         39/39   

IPv4 IGMP groups + multicast routes:         1072/1072          1/1    

IPv4 unicast directly-connected routes:      2048/2048         34/34   

IPv4 unicast indirectly-connected routes:    2048/2048       2047/2047 

IPv6 Multicast groups:                       1072/1072         11/11   

IPv6 unicast directly-connected routes:      2048/2048          0/0    

IPv6 unicast indirectly-connected routes:    3072/3072          3/3    

IPv4 policy based routing aces:               128/128          13/13   

IPv4 qos aces:                                512/512           6/6    

IPv4 security aces:                           640/640          44/44   

IPv6 policy based routing aces:               128/128           8/8    

IPv6 qos aces:                                128/128           5/5    

IPv6 security aces:                           128/128          17/17   

Note: Allocation of TCAM entries per feature uses

a complex algorithm. The above information is meant

to provide an abstract view of the current TCAM utilization

in our testing the above high utilization of "IPv4 unicast indirectly-connected routes:    2048/2048       2047/2047"was the issue that high CPU usage occour.

As we have found, Router will forward packet to CPU when TCAM resources are full. We didn't found a good explanation for this, but i hope this will give you some idea.

New Member

RedEarth Tx Mana - High CPU usage on Cisco 3560-X

CAM Utilization for ASIC# 0                       Max            Used

                                             Masks/Values     Masks/values

  Unicast mac addresses:                       6364/6364         189/189 

IPv4 IGMP groups + multicast routes:          1072/1072          1/1   

IPv4 unicast directly-connected routes:       6144/6144          0/0   

IPv4 unicast indirectly-connected routes:     2048/2048         34/34  

IPv6 Multicast groups:                         112/112          11/11  

IPv6 unicast directly-connected routes:         74/74            0/0   

IPv6 unicast indirectly-connected routes:       32/32            3/3   

IPv4 policy based routing aces:                412/412          12/12  

IPv4 qos aces:                                 512/512          21/21  

IPv4 security aces:                            924/924          44/44  

IPv6 policy based routing aces:                 18/18            8/8   

IPv6 qos aces:                                  22/22           18/18  

IPv6 security aces:                             60/60           17/17  

Note: Allocation of TCAM entries per feature  uses

a complex algorithm. The above information is  meant

to provide an abstract view of the current TCAM  utilization

I do not see any thing abnormal here. Can you please advice from the above command ?

New Member

Re: RedEarth Tx Mana - High CPU usage on Cisco 3560-X

Seems like any of your TCAM resorces are not much utilized. I did not find any good explanation about what RedEarth Rx Mana process actually does.

Better do some more search on this process.

5280
Views
0
Helpful
10
Replies
CreatePlease to create content