High cpu interrupts on 2811 using NAT on 100Mbit link

Unanswered Question
Jun 12th, 2008
User Badges:

I've experience 99% interrupts on mine 2811 when using nat on 100Mbit link and usually it's not getting more than 45Mbit. Without NAT, cpu load not more than 3%.

That's quite surprising me because 1801 at my other site can handle 100Mbit just fine with NAT interrupts just take about 50-60%.

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
sirdudesly Thu, 06/12/2008 - 01:56
User Badges:

we'll need to know a bit more about your NAT configuration before we can troubleshoot the problem

nikolay-shopik Thu, 06/12/2008 - 02:03
User Badges:

There nothing special in NAT configuration.

I have two outside interfaces (one not used right now), and one inside.

And I overloading my outside interface. Both pools unused right now. below you can find nat confirutaion



ip nat translation tcp-timeout 7200

ip nat pool MX X.X.X.146 X.X.X.146 prefix-length 28

ip nat pool MTU-pool X.X.X.147 X.X.X.148 prefix-length 28

ip nat pool corbina Y.Y.Y.67 Y.Y.Y.78 prefix-length 28

ip nat inside source list 100 interface FastEthernet0/0.4 overload

ip nat inside source list 151 pool MX overload

ip nat inside source static tcp 10.0.1.5 587 interface FastEthernet0/0.4 587

ip nat inside source static tcp 10.0.1.2 389 interface FastEthernet0/0.4 389

ip nat inside source static tcp 10.0.1.6 21 interface FastEthernet0/0.4 21

ip nat inside source static tcp 10.0.1.5 143 interface FastEthernet0/0.4 143

ip nat inside source static tcp 10.0.1.253 25 interface FastEthernet0/0.4 25

ip nat inside source static tcp 10.0.1.251 80 interface FastEthernet0/0.4 33389

Pravin Phadte Thu, 06/12/2008 - 03:37
User Badges:
  • Silver, 250 points or more

Hi,


When we say that router 1081 has cpu of 50 to 60%. In this design and config what are the NAT statements? Is it doing 1 or more than 1 overload?.


I would not compare the NAT or PAT to be done by the bandwidth. They work fine if configured logically and will not affect the bandwidth of the link.


Cisco says that one NAT is been enabled the CPU will be utilized.


Once the NAT starts working it will use the cpu once the cpu shoots high the more the packet drops on the link.

This will happen in NAT overload.


You have 3 nat pool in which you may can make it to 2. Once its overload i don't feel it will help much but if your cpu is at 99% it will bring it down to 30%.

Again this will depend on how many ip address you will be overloading from internal network through the nat. If there are too many of them it will increase the cpu in turn.


show ip nat translation will show you how may ip address are been natted.


I would request to re-modify the nat overload statements as per you requirements and check.


regards,


Pravin


nikolay-shopik Thu, 06/12/2008 - 04:07
User Badges:

1801 have even little more complex configuration than 2811 because of two outside interfaces in works and therefore have little more complex access-list. Here is part of config.

ip nat translation tcp-timeout 7200

ip nat inside source list 100 interface FastEthernet0 overload

ip nat inside source list 101 interface Virtual-PPP1 overload

ip nat inside source static tcp 192.168.0.11 21 interface Virtual-PPP1 21

ip nat inside source static tcp 192.168.0.11 5500 interface Virtual-PPP1 5500

ip nat inside source static udp 192.168.0.11 60000 interface Virtual-PPP1 60000

ip nat inside source static tcp 192.168.0.11 60000 interface Virtual-PPP1 60000


As I said before I don't use pool currently only overloading 1 IP interface. Currently I'm overloading only 1 IP from my inside address, downloading from FTP server test file. And removing all pools doesn't helps in this case.


Pravin Phadte Thu, 06/12/2008 - 04:36
User Badges:
  • Silver, 250 points or more

Which of the pools is configured right now.


ip nat pool MX X.X.X.146 X.X.X.146 prefix-length 28

ip nat pool MTU-pool X.X.X.147 X.X.X.148 prefix-length 28

ip nat pool corbina Y.Y.Y.67 Y.Y.Y.78 prefix-length 28


Also can you provide the output of


sh memory

sh processes cpu

sh flash: to check any crashinfo and ios


Regrads

Pravin

nikolay-shopik Thu, 06/12/2008 - 04:50
User Badges:

I don't use pools. see by yourself


dodge#sh ip nat s

Total active translations: 35 (0 static, 35 dynamic; 35 extended)

Outside interfaces:

FastEthernet0/0.4

Inside interfaces:

FastEthernet0/0.3

Hits: 20207541 Misses: 0

CEF Translated packets: 18935688, CEF Punted packets: 2540075

Expired translations: 449027

Dynamic mappings:

-- Inside Source

[Id: 10] access-list 100 interface FastEthernet0/0.4 refcount 8

Appl doors: 0

Normal doors: 5

Queued Packets: 0


dodge#sh mem

Head Total(b) Used(b) Free(b) Lowest(b) Largest(b)

Processor 469A63C0 142974016 42030556 100943460 100193880 96017832

I/O 3F200000 14680064 6656656 8023408 8007104 8023228


CPU utilization for five seconds: 99%/95%; one minute: 22%; five minutes: 6%

PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process

229 2836 286 9916 0.08% 0.28% 0.42% 514 SSH Process

332 5664 317 17867 0.32% 0.58% 0.17% 515 SSH Process

2 23324 223291 104 2.84% 0.48% 0.15% 0 Load Meter

107 53392 34810386 1 0.00% 0.10% 0.13% 0 IP ARP Retry Age

5 1694716 134023 12644 0.00% 0.14% 0.12% 0 Check heaps

108 1891588 1872297 1010 0.16% 0.23% 0.10% 0 IP Input

103 51332 34810395 1 0.00% 0.06% 0.07% 0 ACCT Periodic Pr

181 23636 11155213 2 0.08% 0.05% 0.06% 0 RBSCP Background

144 21736 2237699 9 0.00% 0.03% 0.02% 0 DHCPD Receive

43 17848 1116445 15 0.00% 0.03% 0.02% 0 Per-Second Jobs

50 484224 18657 25954 0.73% 0.07% 0.01% 0 Per-minute Jobs

136 2684 7457 359 0.00% 0.02% 0.00% 0 TCP Timer

90 1684 1116313 1 0.08% 0.00% 0.00% 0 PI MATM Aging Pr

183 4504 2180056 2 0.08% 0.00% 0.00% 0 Inspect process

61 9136 4465465 2 0.00% 0.01% 0.00% 0 Netclock Backgro

84 24712 825136 29 0.00% 0.01% 0.00% 0 DSL State Machin

118 9600 4360919 2 0.00% 0.01% 0.00% 0 SSS Feature Time

139 9848 1809723 5 0.00% 0.01% 0.00% 0 CEF process

255 10852 5581827 1 0.00% 0.01% 0.00% 0 Atheros LED Ctro


c2800nm-advipservicesk9-mz.124-15.T5.bin


There one crash by bus error occurred once before, but its related to VOIP part of IOS.

Pravin Phadte Thu, 06/12/2008 - 05:22
User Badges:
  • Silver, 250 points or more

I feel this is a problem with the IOS. But i dont see any bugs on cisco website for the same.


Still i have one thing which would help for this ios. Well i can say may be.


If there is ACL applied for the NAT statement.


lets say the config is like below.


interface FastEthernet0/0

description Outside

ip address 192.168.0.1 255.255.255.0

ip nat outside

!

interface FastEthernet0/1

description Inside

ip address 10.0.0.1 255.255.255.0

ip nat inside

!

ip nat pool Outside 192.168.0.10 192.168.0.20 netmask 255.255.255.0


ip nat inside source list Inside pool Outside

!

ip access-list standard Inside

permit any


Change to as below.

ip nat pool Outside 192.168.0.10 192.168.0.20 netmask 255.255.255.0

ip nat inside source list Inside pool Outside

!

ip access-list standard Inside

permit 10.0.0.0 0.255.255.255


Regards,


Pravin

Sushil Kumar Katre Thu, 06/12/2008 - 08:06
User Badges:
  • Gold, 750 points or more

Hi Nikolay,


It would be helpful if you can share the configuration of the router so that we can check what else is configured along with NAT.


It might be a combination of NAT and other feature which is affecting the router. Since you are other members ahve already checked things related to NAT.


-> Sushil

Pravin Phadte Thu, 06/12/2008 - 09:48
User Badges:
  • Silver, 250 points or more

Hi,


Can you try to modiyf the NAT as below.


ip nat pool natpool Y.Y.Y.66 Y.Y.Y.66 netmask 255.255.255.240

ip nat inside source list 100 pool natpool overload


Regrads,


pravin

Pravin Phadte Fri, 06/13/2008 - 09:00
User Badges:
  • Silver, 250 points or more

Hi,

My last suggestion would be to upgrade the ios and check.


Nothing else is hitting my mind for troubleshooting this issue.


nikolay-shopik Fri, 06/13/2008 - 10:14
User Badges:

I'm running latest version for now, but thanks anyway trying to help me.

For first time I though maybe its because 1801 using FA0 and VLAN1 interfaces as outside and inside correspondingly, notice VLAN is not physical but virtual interface. So I give a try another test using cisco1812 using FA0 and FA1 interfaces and got same performance as 1801.

Probably I will open ticket for TAC, its seems problem in some kind hardware or software limitation, because friends of mine experience same problem using 2811.

Actions

This Discussion