Cisco 3745 high cpu usage

[IgorHamzic]

Hi. For some time on one of ours Cisco 3745 routers we have been having a very high CPU usage around 70%. I know this is very high and wondered if you could help me to find the cause. Below is the output from the sh proc cpu | exclude 0.00%__0.00%__0.00% command.

CPU utilization for five seconds: 60%/25%; one minute: 69%; five minutes: 72%

PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process

4 22848424 2914987 7838 0.00% 0.05% 0.05% 0 Check heaps

22 3151572 24641326 127 0.00% 0.02% 0.00% 0 Per-Second Jobs

35 6275548 425586 14745 0.00% 0.01% 0.00% 0 Per-minute Jobs

47 3490448562875681874 0 0.49% 0.74% 0.66% 0 IP Input

73 357226642514347994 0 0.08% 0.04% 0.05% 0 Socket Timers

115 9046748 21118779 428 0.08% 0.03% 0.02% 0 SAA Event Proces

126 310612044 34928714 8892 32.42% 25.41% 26.10% 0 FRF9 manager

131 10119524 139770053 72 0.00% 0.02% 0.03% 0 IP-EIGRP Hello

138 11579950442514347994 0 1.39% 2.08% 2.15% 0 Rtt Responder

149 5092728 74037445 68 0.00% 0.01% 0.00% 0 IP-EIGRP Router

The router has 128 MBs of memory and IOS version c3745-is-mz.122-13.T1.bin.

Any help is greatly appreciated.

[Joseph W. Doherty]

RTR (or now known as SAA, prior name still in command syntax), will add load; not sure if it would be enough to account for your CPU load especially at the interrupt level. If no one is monitoring or analyzing the stats, something you could turn off and see the impact.

[IgorHamzic]

Seems that disabling the rtr did the trick. The CPU usage dropped visibly after disabling the rtr.

The result of sh proc cpu | exclude 0.00%__0.00%__0.00%:

CPU utilization for five seconds: 1%/0%; one minute: 0%; five minutes: 12%

PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process

3 15220 530 28716 0.00% 0.00% 0.23% 162 Virtual Exec

4 33746820 3566160 9463 0.65% 0.11% 0.06% 0 Check heaps

47 4060059163096394853 0 0.08% 0.01% 0.13% 0 IP Input

115 13374740 22568152 592 0.08% 0.02% 0.01% 0 SAA Event Proces

126 3593591000 179190592 20054 0.16% 0.01% 0.00% 0 FRF9 manager

138 13547657362727439431 0 0.00% 0.00% 0.41% 0 Rtt Responder

149 6738796 89963122 74 0.00% 0.01% 0.00% 0 IP-EIGRP Router

I'll keep monitoring for a while and report the results. If this keeps up I'll disable rtr on the other router and switch too.

[Joseph W. Doherty · Accepted solution]

Quite a difference, especially if it holds!

If no one is actually using RTR/SAA, deactivation might make sense. Or, check the parameters and have the tests sample much, much less frequently.

[IgorHamzic]

Sure is quite a difference.I believe no one is using the RTR/SAA currently but I'll wait for a couple of days and see if someone complains.

I sure hope it holds. The real test will come in about a couple of hours when people in remote offices start work.

I think it will be a much smoother ride as there won't be any interrupts now that will use CPU resources.

[ariesc_33]

Hi,

Can you post the complete output of the "show proc cpu"?

The output above shows that the overall CPU is 81 percent, but summing up the output you posted, it is not equal to 81 percent.

The output should be a clear clue on what really causes the high CPU by looking of how much CPU of each service consumes.

The first output on your first post shows that one of the problem has something to do with frame relay.

[IgorHamzic]

Hi all. The problem was with RTR. After I removed it from the configuration the CPU usage dropped to around 1% even during peak times.

The problem is solved now.

Thank you guys for the great advice that helped me solve this.