3640 High CPU load

Unanswered Question
Dec 20th, 2006
User Badges:

I have 3640 router configured for MLPPP with 2 E1 links.

Currently the IOS used is c3640-i-mz.122-40.bin

When the bandwidth usage is high the CPU load reach 100%

Any suggestion to upgrade the IOS to resolve the CPU load issue.

3640 has Flash:8MB and RAM:96MB

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
spremkumar Wed, 12/20/2006 - 20:59
User Badges:
  • Red, 2250 points or more

Hi


Are you only running multilink or having any other routing protocol or any QOS config done in the box ?


Also can you post the output of show process cpu & show version here ?


The outputs will help to find out the process which hogs up the cpu as well as to check the ios code for any bugs..


Do make sure that you have cef enabled on your box which will also solve the high cpu utilisation problem..


you can enable the same using ip cef under global config mode..



regds


sourabhagarwal Wed, 12/20/2006 - 21:32
User Badges:
  • Gold, 750 points or more

please find attachment which has possible causes and sloution of the high CPU utilization.


hoope it helps ... rate if it does ...



ckuriyar74 Wed, 12/20/2006 - 21:58
User Badges:

Fast switching enabled on all interfaces and still no luck & high CPU load.

jackyoung Wed, 12/20/2006 - 23:31
User Badges:
  • Gold, 750 points or more

Can you post the config ?


Are you hit this bug ?


***Quoted Start

CSCdv68743


Symptoms: The performance of a router may be lower than you would expect, and CPU utilization may be high during packet forwarding.


Conditions: These symptoms are observed on a nondistributed Cisco router that runs Cisco IOS Release 12.2, 12.2 S, 12.2 T, 12.3, or 12.3 T under the following circumstances:


?The router has a service policy attached to one or more interfaces.


?The policy map of the service policy contains one or more class maps that are configured with one or more match access-group name access-group-name class-map configuration commands.


?There is a large number of named extended IP access control list (ACLs) configured on the router, and the packets that are passing through the router match these ACLs.


Workaround: If this is an option, enter the match access-group access-group class-map configuration command in the class maps, that is, enter the access-group argument instead of the access-group-name argument.

*** Quoted End


I cannot suggest the IOS unless there is more info.


Any PBR, debug are running on the router ?


If there is maintenance window, it is fine to upgrade the IOS to test it but be backup all config. and original IOS before carry the upgrade. Also need to check the memory requirement for the new IOS.


Hope this helps.



royalblues Thu, 12/21/2006 - 00:01
User Badges:
  • Green, 3000 points or more

i agree with Jack.


I have seen similar symptoms when i had PBR configured on a 3640 router.

Try upgrading the IOS to the latest one but iam very doubtful that you would be able to load a new one with just 8 MB capacity


HTH

Narayan

ckuriyar74 Thu, 12/21/2006 - 01:43
User Badges:

I agree with Jack.

This router has a service policy attached to one or more interfaces and also the policy map of the service policy contains one or more class maps.

Also there is a large number of named extended IP access control list (ACLs) configured on the router.


The problem like i just have 8MB flash to upgrade the IOS.

We have ordered new 2811 router and probably replacing in day or two.


Thanks to every one for the suggestions.

jackyoung Thu, 12/21/2006 - 05:34
User Badges:
  • Gold, 750 points or more

If you have chance, try to remove those PBR command in interface then check the CPU loading.


Please advise the result if you took the test. :)

ckuriyar74 Mon, 12/25/2006 - 21:27
User Badges:

I have replaced the router with 2811.

The IOS I have uploaded is

c2800nm-advipservicesk9-mz.124-11.T.bin


and also attached the router config.



royalblues Tue, 12/26/2006 - 03:14
User Badges:
  • Green, 3000 points or more

Hi


I see NBAR configured on the routers. This process is CPU intensive and can definitely choke a 3640 router.


The 2811 should be able to take the load.


Are you still seeing high utilization BTW


HTH

Narayan

jackyoung Tue, 12/26/2006 - 17:02
User Badges:
  • Gold, 750 points or more

I can't download the config file, can you reattach it. However, I agree w/ Narayan that any packet inspection feature will increase the router loading, so try to remove / disable those commands and test again.


Hopt this helps.

ckuriyar74 Wed, 12/27/2006 - 03:04
User Badges:

Again attached the config file.

I tried disabling those commands and 10% of CPU load reduced.


jackyoung Wed, 12/27/2006 - 17:26
User Badges:
  • Gold, 750 points or more

Please try to remove the "ip accounting" commands. It use lots of CPU resources. Only enable the ip accounting if there is a need, e.g. verify packet flow in interface.


If the CPU still high, it means the QoS commands may spend the router loading too. Try to remove the service policy command at the interface one by one to test. However, I believe the QoS commands are required, so if the ip accounting commands are removed but still high router loading, you may require to upgrade the router.



Hope this helps.

ckuriyar74 Tue, 01/02/2007 - 23:36
User Badges:

Probably multilink configuration causing high CPU load?


Any suggestions to configure instead of MLPPP.


IP Cef per destination or per packet?

jackyoung Tue, 01/02/2007 - 23:57
User Badges:
  • Gold, 750 points or more

Could you check my reponse on 27 Dec ? IP accounting is a CPU intensive feature. More packet pass through, more cpu loading is used.


Please try to remove this command and check again.


Hope this helps.



ckuriyar74 Wed, 01/03/2007 - 00:06
User Badges:

I have already removed IP accounting but still CPU high.

jackyoung Wed, 01/03/2007 - 00:46
User Badges:
  • Gold, 750 points or more

Thanks for the update. It is quite strangth, could you post the updated config. ?


The MLPPP should not be the factor. I suspect there is bug or hardware limitation.

ckuriyar74 Wed, 01/03/2007 - 01:39
User Badges:

I have other 2801 router with

IOS c2801-ipbase-mz.124-1c.bin and have the same configuration in both 2801 & 2811

The output in 2801 is on ethernet not on serial and this is my backup router.

If the traffic is on 2801 router the CPU load does not go more than 20% but when the same amount of traffic on 2811 router with MLPPP goes upto 50%.

I have attached the config.



Attachment: 
royalblues Wed, 01/03/2007 - 02:31
User Badges:
  • Green, 3000 points or more

When you do MLPPP, all packets are processed by CPU.


If you can spare a few IPs and configure each link wth a different one, try loadbalancing on CEF instead and check


HTH

Narayan

jackyoung Wed, 01/03/2007 - 06:53
User Badges:
  • Gold, 750 points or more

In addition to Naranya;s suggestion. If you tried to disable MLPPP and the CPU loading still high, you can try to remove those QoS and filtering command one-by-one to determine the root of cause.


However, 2811 do have higher processing power than 3640. Let me check the figure and return to you tomorrow.



jackyoung Wed, 01/03/2007 - 17:01
User Badges:
  • Gold, 750 points or more

In addition to my last post, please check below link that the 2811 perform 120,000PPS and 3640 perform 50,000 - 70,000PPS under CEF. Therefore, the process power of 3640 is much lower than 2811.


I suggest to make the 2811 as primary router w/ MLPPP and 3640 as the backup router to provide better service.


http://www.cisco.com/warp/public/765/tools/quickreference/routerperformance.pdf


Hope this helps.

rabeder Wed, 01/03/2007 - 07:43
User Badges:

hi,


i did some testing with llq and 3640, 3825, 28xx routers.

!

the 3640 is not able to handle more than 4mb/s with llq.


it depends a little bit which encapsulation on layer 2 you are using.


i testet with frf16 (two e1 fr interfaces) - as i said, the router was overloaded.

if you use ethernet as layer 2 - you can say that a 3640 can the double of traffic than frame-relay

royalblues Thu, 01/04/2007 - 22:40
User Badges:
  • Green, 3000 points or more

I Agree with Kurt.


I have seen similar symptoms in a 2621 routers where the CPU would become a problem when you pump about 4 MB traffic on FR along with IPSec


The router just works fine when 10Mbps link is terminated on ethernet with the same features.


Narayan



jackyoung Fri, 01/05/2007 - 00:17
User Badges:
  • Gold, 750 points or more

I agreed too, this is why I ask to remove the function one-by-one to determine which one consume the CPU loading the most.


However, if the 2811 (high cpu power) can be the primary then it will solve the problem.


In IPSEC case, we have to consider the hardware-based encryption module to speed up the process and load-share the on-board CPU loading.


Just my 2 cents.

Actions

This Discussion