Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements

Welcome to Cisco Support Community. We would love to have your feedback.

For an introduction to the new site, click here. And see here for current known issues.

New Member

High CPU Interrupt Rate on 7206vxr

Hello Everybody;

I had a very stange problem we suffer from serveral months, searching all internet sites disccusions with no luck to get a solution, here are the details:

I'm a service provider which provide ADSL service, we have 5 LNSs "BRAS" to terminat PPP users via LAC of Upper service provider.

Each LNS type is Cisco 7206vxr-NPE-G2 ( c7200p-adventerprisek9-mz.151-4.M1.bin ) handle 1800 PPP sessions and also each LNS connected to 13 LAC via vpdn over l2tp tunnels, and each LNS handle (400 - 500 Mbps) of traffic which make total of 2.5 Gbps to Internet Router.

LNS configuration not include any additional configuration else it runs OSPF to collect connected routes and send it to Internet Router.

The problem is we suffer from High cpu utlization of Interrupts. Each LNS have CPU rate of : 90/70.

Please could any one give me any advice to solve this or it's a normal thing?

Note: ip cef enabled

i can provide you any show results

Here some of show results:

LNS#show cef drop

% Command accepted but obsolete, see 'show (ip|ipv6) cef switching statistics [feature]'

IPv4 CEF Drop Statistics

Slot  Encap_fail  Unresolved Unsupported    No_route      No_adj  ChkSum_Err

RP             0           0  2562691697          90           0       12160

--------------------------------------------------------------------------------------------------------------------------------------------------------------------

LNS#show cef not-cef-switched

% Command accepted but obsolete, see 'show (ip|ipv6) cef switching statistics [feature]'

IPv4 CEF Packets passed on to next switching layer

Slot  No_adj No_encap Unsupp'ted Redirect  Receive  Options   Access     Frag

RP         0       0  2562929756        0 68566096        0   126945        0

--------------------------------------------------------------------------------------------------------------------------------------------------------------------

LNS#show interfaces switching

GigabitEthernet0/1 " Interface face upper service provider "

          Throttle count          0

                   Drops         RP        176         SP          0

             SPD Flushes       Fast     471823        SSE          0

             SPD Aggress       Fast          0

            SPD Priority     Inputs    1924470      Drops          0

    Protocol  IP                 

          Switching path    Pkts In   Chars In   Pkts Out  Chars Out

                 Process  718302296 4126662098 1112298943 1451433415

            Cache misses          0          -          -          -

                    Fast 3305818606 1149552345  661100592 1370347133

               Auton/SSE          0          0          0          0

    Protocol  DEC MOP            

          Switching path    Pkts In   Chars In   Pkts Out  Chars Out

                 Process          0          0       1738     133826

            Cache misses          0          -          -          -

                    Fast          0          0          0          0

               Auton/SSE          0          0          0          0

    Protocol  ARP                

          Switching path    Pkts In   Chars In   Pkts Out  Chars Out

                 Process       4780     286800       1526      91560

            Cache misses          0          -          -          -

                    Fast          0          0          0          0

               Auton/SSE          0          0          0          0

    Protocol  CDP                

          Switching path    Pkts In   Chars In   Pkts Out  Chars Out

                 Process      17399    8247126      19362    7163940

            Cache misses          0          -          -          -

                    Fast          0          0          0          0

               Auton/SSE          0          0          0          0

    Protocol  Other              

          Switching path    Pkts In   Chars In   Pkts Out  Chars Out

                 Process          0          0     104308    6258480

            Cache misses          0          -          -          -

                    Fast          0          0          0          0

               Auton/SSE          0          0          0          0

GigabitEthernet0/2 " Inerface connected to Internet Router "

          Throttle count          0

                   Drops         RP     959104         SP          0

             SPD Flushes       Fast   15007215        SSE          0

             SPD Aggress       Fast          0

            SPD Priority     Inputs    1364797      Drops          0

    Protocol  IP                 

          Switching path    Pkts In   Chars In   Pkts Out  Chars Out

                 Process  999401649 1612699701  712502786 2118640235

            Cache misses          0          -          -          -

                    Fast 1083143063 4057627019 3036792652 3002157150

               Auton/SSE          0          0          0          0

    Protocol  DEC MOP            

          Switching path    Pkts In   Chars In   Pkts Out  Chars Out

                 Process          0          0       1738     133826

            Cache misses          0          -          -          -

                    Fast          0          0          0          0

               Auton/SSE          0          0          0          0

    Protocol  ARP                

          Switching path    Pkts In   Chars In   Pkts Out  Chars Out

                 Process         75       4500       1254      75240

            Cache misses          0          -          -          -

                    Fast          0          0          0          0

               Auton/SSE          0          0          0          0

    Protocol  CDP                

          Switching path    Pkts In   Chars In   Pkts Out  Chars Out

                 Process      19355    8245225      19354    7160980

            Cache misses          0          -          -          -

                    Fast          0          0          0          0

               Auton/SSE          0          0          0          0

    Protocol  Other              

          Switching path    Pkts In   Chars In   Pkts Out  Chars Out

                 Process          0          0     104314    6258840

            Cache misses          0          -          -          -

                    Fast          0          0          0          0

               Auton/SSE          0          0          0          0

Some of Virtual-Access Users result:

Virtual-Access10

          Throttle count          0

                   Drops         RP          0         SP          0

             SPD Flushes       Fast          0        SSE          0

             SPD Aggress       Fast          0

            SPD Priority     Inputs      25451      Drops          0

    Protocol  IP                 

          Switching path    Pkts In   Chars In   Pkts Out  Chars Out

                 Process     385798   16849653     457847  635790618

            Cache misses          0          -          -          -

                    Fast     681084   63626366   26634169 1192416118

               Auton/SSE          0          0          0          0

    Protocol  Other              

          Switching path    Pkts In   Chars In   Pkts Out  Chars Out

                 Process          3         96      86610    1386434

            Cache misses          0          -          -          -

                    Fast   19576850 2329991433          0          0

               Auton/SSE          0          0          0          0

Virtual-Access12

          Throttle count          0

                   Drops         RP          0         SP          0

             SPD Flushes       Fast          0        SSE          0

             SPD Aggress       Fast          0

            SPD Priority     Inputs       8543      Drops          0

    Protocol  IP                 

          Switching path    Pkts In   Chars In   Pkts Out  Chars Out

                 Process      15848    8018299      21948    2215706

            Cache misses          0          -          -          -

                    Fast     310092   46529741   22224211 1403568072

               Auton/SSE          0          0          0          0

    Protocol  Other              

          Switching path    Pkts In   Chars In   Pkts Out  Chars Out

                 Process          4        128      85123    1362688

            Cache misses          0          -          -          -

                    Fast   17113093 2030440478          0          0

               Auton/SSE          0          0          0          0

Virtual-Access19

          Throttle count          0

                   Drops         RP          0         SP          0

             SPD Flushes       Fast          0        SSE          0

             SPD Aggress       Fast          0

            SPD Priority     Inputs       8123      Drops          0

    Protocol  IP                 

          Switching path    Pkts In   Chars In   Pkts Out  Chars Out

                 Process    1637093  139266284    2423448 3162258987

            Cache misses          0          -          -          -

                    Fast          0          0   28885932 2671057232

               Auton/SSE          0          0          0          0

    Protocol  Other              

          Switching path    Pkts In   Chars In   Pkts Out  Chars Out

                 Process          0          0      85303    1365572

            Cache misses          0          -          -          -

                    Fast   20771820 2246068842          0          0

               Auton/SSE          0          0          0          0

--------------------------------------------------------------------------------------------------------------------------------------------------------------------

LNS#show ip cef switching statistics

Path   Reason                          Drop       Punt  Punt2Host

RP LES Packet destined for us             0   35893991          0

RP LES TTL expired                        0          0        170

RP LES Fragmentation failed, DF         389          0          0

RP LES Features                     9615849          0          0

RP LES Unclassified reason                9  997284686          0

RP LES Neighbor resolution req            0          1          0

RP LES Fragmentation no pak               0          0       3441

RP LES Total                        9616247 1033178678       3611

RP PAS No route                          90          0          1

RP PAS Packet destined for us          2919   32696467        216

RP PAS Incomplete adjacency           21772          0          0

RP PAS Bad checksum                   12161          0          0

RP PAS TTL expired                        0          0   11118798

RP PAS Bad IP packet length              37          0          0

RP PAS Features                   964811261  557882479     131388

RP PAS Unclassified reason           294249  997284686          0

RP PAS Neighbor resolution req            1          0          0

RP PAS Total                      965142490 1587863632   11250403

All    Total                      974758737 2621042310   11254014

2 seconds after another show command

LNS#show ip cef switching statistics

Path   Reason                          Drop       Punt  Punt2Host

RP LES Packet destined for us             0   35894734          0

RP LES TTL expired                        0          0        170

RP LES Fragmentation failed, DF         389          0          0

RP LES Features                     9615935          0          0

RP LES Unclassified reason                9  997291274          0

RP LES Neighbor resolution req            0          1          0

RP LES Fragmentation no pak               0          0       3442

RP LES Total                        9616333 1033186009       3612

RP PAS No route                          90          0          1

RP PAS Packet destined for us          2919   32696508        216

RP PAS Incomplete adjacency           21772          0          0

RP PAS Bad checksum                   12161          0          0

RP PAS TTL expired                        0          0   11118890

RP PAS Bad IP packet length              37          0          0

RP PAS Features                   964819941  557889345     131389

RP PAS Unclassified reason           294250  997291274          0

RP PAS Neighbor resolution req            1          0          0

RP PAS Total                      965151171 1587877127   11250496

All    Total                      974767504 2621063136   11254108

--------------------------------------------------------------------------------------------------------------------------------------------------------------------

LNS#show ip cef switching statistics feature

IPv4 CEF input features:

Path   Feature                Drop    Consume       Punt  Punt2Host Gave route

RP LES uRPF                     17          0          0          0          0

RP LES CAR                   74919          0          0          0          0

RP PAS Virtual Fragment      54857    6440840          0       4470          0

RP PAS Access List          117919          0          0     126642          0

RP PAS uRPF                 247018          0          0          0          0

RP PAS CAR                11870139          0          0          0          0

Total                     12364869    6440840          0     131112          0

IPv4 CEF output features:

Path   Feature                Drop    Consume       Punt  Punt2Host    New i/f

RP LES CAR                 9541552          0          0          0          0

RP PAS Access List          415655          0          0        303          0

RP PAS TCP Adjust MSS            0          0  557939001          0          0

RP PAS CAR               952181909          0          0          0          0

Total                    962139116          0  557939001        303          0

IPv4 CEF post-encap features:

Path   Feature                Drop    Consume       Punt  Punt2Host    New i/f

Total                            0          0          0          0          0

IPv4 CEF for us features:

Path   Feature                Drop    Consume       Punt  Punt2Host    New i/f

Total                            0          0          0          0          0

IPv4 CEF punt features:

Path   Feature                Drop    Consume       Punt  Punt2Host    New i/f

Total                            0          0          0          0          0

IPv4 CEF local features:

Path   Feature                Drop    Consume       Punt  Punt2Host Gave route

Total                            0          0          0          0          0

--------------------------------------------------------------------------------------------------------------------------------------------------------------------

LNS#show ip traffic

IP statistics:

  Rcvd:  1731030398 total, 29323905 local destination

         7 format errors, 230 checksum errors, 9755726 bad hop count

         0 unknown protocol, 0 not a gateway

         0 security failures, 0 bad options, 9238659 with options

  Opts:  0 end, 0 nop, 0 basic security, 0 loose source route

         0 timestamp, 0 extended security, 0 record route

         0 stream ID, 0 strict source route, 9238659 alert, 0 cipso, 0 ump

         0 other

  Frags: 9717546 reassembled, 6040 timeouts, 0 couldn't reassemble

         443666552 fragmented, 887330011 fragments, 778 couldn't fragment

  Bcast: 90024 received, 0 sent

  Mcast: 1362936 received, 685736 sent

  Sent:  154123399 generated, 1231263328 forwarded

  Drop:  913 encapsulation failed, 0 unresolved, 0 no adjacency

         0 no route, 250302 unicast RPF, 0 forced drop

         0 options denied

  Drop:  0 packets with source IP address zero

  Drop:  0 packets with internal loop back IP address

         0 physical broadcast

  Reinj: 0 in input feature path, 0 in output feature path

ICMP statistics:

  Rcvd: 0 format errors, 147 checksum errors, 0 redirects, 22 unreachable

        104596 echo, 2 echo reply, 0 mask requests, 0 mask replies, 0 quench

        0 parameter, 0 timestamp, 0 timestamp replies, 0 info request, 0 other

        0 irdp solicitations, 0 irdp advertisements

        10662 time exceeded, 0 info replies

  Sent: 0 redirects, 132461 unreachable, 0 echo, 104596 echo reply

        0 mask requests, 0 mask replies, 0 quench, 0 timestamp, 0 timestamp replies

        0 info reply, 188205 time exceeded, 0 parameter problem

        0 irdp solicitations, 0 irdp advertisements

TCP statistics:

  Rcvd: 25409 total, 3 checksum errors, 59 no port

  Sent: 24035 total

BGP statistics:

  Rcvd: 0 total, 0 opens, 0 notifications, 0 updates

        0 keepalives, 0 route-refresh, 0 unrecognized

  Sent: 0 total, 0 opens, 0 notifications, 0 updates

        0 keepalives, 0 route-refresh

EIGRP-IPv4 statistics:

  Rcvd: 0 total

  Sent: 0 total

PIMv2 statistics: Sent/Received

  Total: 0/0, 0 checksum errors, 0 format errors

  Registers: 0/0 (0 non-rp, 0 non-sm-group), Register Stops: 0/0,  Hellos: 0/0

  Join/Prunes: 0/0, Asserts: 0/0, grafts: 0/0

  Bootstraps: 0/0, Candidate_RP_Advertisements: 0/0

  Queue drops: 0

  State-Refresh: 0/0

IGMP statistics: Sent/Received

  Total: 0/0, Format errors: 0/0, Checksum errors: 0/0

  Host Queries: 0/0, Host Reports: 0/0, Host Leaves: 0/0

  DVMRP: 0/0, PIM: 0/0

  Queue drops: 0

UDP statistics:

  Rcvd: 27819278 total, 0 checksum errors, 83282 no port

  Sent: 152988495 total, 0 forwarded broadcasts

OSPF statistics:

  Last clearing of OSPF traffic counters never

  Rcvd: 1363785 total, 0 checksum errors

        110059 hello, 172 database desc, 1 link state req

        1113254 link state updates, 140294 link state acks

  Sent: 685982 total

        110015 hello, 173 database desc, 52 link state req

        285043 link state updates, 290701 link state acks

ARP statistics:

  Rcvd: 30911 requests, 365 replies, 0 reverse, 0 other

  Sent: 370 requests, 34367 replies (0 proxy), 0 reverse

  Drop due to input queue full: 0

443666552 fragmented, 887330011 fragments and reassembled Increases Hevaily in IP Statistics

--------------------------------------------------------------------------------------------------------------------------------------------------------------------

interface Virtual-Template1

ip unnumbered GigabitEthernet0/1

ip access-group denysomething out

no ip redirects

no ip unreachables

no ip proxy-arp

ip mtu 1492

ip virtual-reassembly in

ip verify unicast reverse-path

ip tcp adjust-mss 1436

no logging event link-status

peer default ip address pool ADSL-POOL

no ipv6 redirects

ppp authentication pap vpdn

ppp authorization vpdn

ppp accounting vpdn

ppp ipcp dns xx.xx.xx.xx xx.xx.xx.xx

ppp ipcp wins reject

end

--------------------------------------------------------------------------------------------------------------------------------------------------------------------

Thanks !! waiting for your experience to solve with me this problem.

Regards

Everyone's tags (6)
16 REPLIES
New Member

High CPU Interrupt Rate on 7206vxr

Is there any one can help please ?

New Member

High CPU Interrupt Rate on 7206vxr

well ,

remove

ip access-group denysomething out


under ur virtual-template and tell me how much cpu will decrease ?

also ,

show config  under ur interfaces such as gi0/1 gi0/2 gi0/3

regards

New Member

High CPU Interrupt Rate on 7206vxr

This accesslist is just to deny anyone to access my customers CPE however i removed it and nothing happen to my CPU.

Here are the interfaces config

interface GigabitEthernet0/1

ip address yyy.yyy.yyy.yyy zzz.zzz.zzz.zzz

no ip redirects

no ip unreachables

ip flow ingress

ip virtual-reassembly in

ip route-cache same-interface

duplex auto

speed auto

media-type rj45

negotiation auto

hold-queue 4096 in

hold-queue 4096 out

end

interface GigabitEthernet0/2

description Internet

ip address 10.20.10.2 255.255.255.252

no ip redirects

no ip unreachables

ip route-cache same-interface

duplex auto

speed auto

media-type rj45

negotiation auto

hold-queue 4096 in

hold-queue 4096 out

end

interface GigabitEthernet0/3

ip address xxx.xxx.xxx.xxx 255.255.255.0

duplex auto

speed auto

media-type rj45

negotiation auto

end

First one connected to my service provider with their LACs.

Second one connected to my edge router to provide internet.

Third one to for AAA services.

New Member

High CPU Interrupt Rate on 7206vxr

hi ,

the acl on ur virtual templae must be processing more cpu , ur cpu must be decreased at least some percentage

well ,

u seem performed nat and removed it  ,

anyway

plz do the following :

interface gi0/1

no ip flow ingress

no ip virtual-reassembly in

no ip route-cache same-interface

no hold-queue 4096 in

no hold-queue 4096 out

interface gi0/2

no ip route-cache same-interface

no negotiation auto

no hold-queue 4096 in

no hold-queue 4096 ou

tell me with sh process cpu

and sh process cpu sorted

regards

New Member

High CPU Interrupt Rate on 7206vxr

I never used NAT is my LNS ever. all my customers have a vaild public IP.

all the above you mentioned is entered in my configuration early because of High CPU but still no changes, and know i removed them with the same result.

Just to be in the situation, my aaa send rate-limit attribut to users vi access interface to limit their download to a specific rate, these may be when the user reach his limit, the rate-limit will drop any packets exceeded the accepted rate which make all drops go to interrupt cpu level.

one more thing is i didn't see if my virtual-access interfaces are cef switching, because all traffic is cpu proccess, how can i make them to be cef switiched however it's globally enabled cef and under each interface enabled by default that's why they didn't appear in my config, however if i do ( show run all ) i see under all my interfaces cef enabled by default.

Any suggestions !!

New Member

did u fix the problem ??was

did u fix the problem ??

was it routing issue ?? IOS bug issue ??

 

looking forward to hear form you.

 

with my best regards

New Member

High CPU Interrupt Rate on 7206vxr

show proc cpu sort | ex 0.00

CPU utilization for five seconds: 95%/77%; one minute: 94%; five minutes: 93%

PID Runtime(ms)     Invoked      uSecs   5Sec   1Min   5Min TTY Process

109   260408828  1205848927        215 10.80%  9.85%  9.72%   0 IP Input        

314   111212364  1105392453        100  4.24%  3.83%  3.78%   0 L2X Data Daemon 

  66    48352752     1208092      40024  1.76%  1.73%  1.74%   0 Compute load avg

139     9484452     2106286       4502  0.39%  0.42%  0.42%   0 CEF: IPv4 proces

320     3267236    46016038         71  0.31%  0.32%  0.31%   0 PPP Events      

311     3783708    12534293        301  0.15%  0.16%  0.15%   0 RADIUS          

  94      782348     3398167        230  0.15%  0.16%  0.15%   0 ACCT Periodic Pr

319      306292    42998756          7  0.07%  0.07%  0.07%   0 PPP manager     

315     2621088     3284933        797  0.07%  0.04%  0.05%   0 L2TP mgmt daemon

316     1686088     2468047        683  0.07%  0.02%  0.01%   0 L2TUN Applicatio

334      726460     3644903        199  0.07%  0.03%  0.01%   0 OSPF-1 Router   

  93     1104480     3735866        295  0.07%  0.07%  0.07%   0 AAA ACCT Proc   

127     2138968     2226764        960  0.07%  0.04%  0.05%   0 SSS Manager     

201      877896    13560475         64  0.07%  0.15%  0.15%   0 IPHC Admin      

332     2837540      861892       3292  0.07%  0.08%  0.08%   0 VTEMPLATE Backgr

New Member

High CPU Interrupt Rate on 7206vxr

u have high differnee between hardware & software processing ,

95-77 is high value

how many users and how mush bw ur router loaded ?

also ,

give#  sh run  for ur routing table .

i mean sh run | i route

also ,

#sh ip arp

also

#show log

regards

New Member

High CPU Interrupt Rate on 7206vxr

NOW the concurrent session users is 1350.

The total cuonsumed traffic in the interface of gig 0/1 is 500 Mbps

and for interface gig 0/2 is 520 Mbps

in my show arp, it just appear the devices directly connected to my LNS is contains 14 address

show log result

Jan 22 10:33:13.729: %IP_VFR-4-FRAG_TABLE_OVERFLOW: Virtual-Access808: the fragment table has reached its maximum threshold 16

Jan 22 10:35:22.401: %IP_VFR-4-FRAG_TABLE_OVERFLOW: Virtual-Access808: the fragment table has reached its maximum threshold 16

Jan 22 10:37:38.401: %IP_VFR-4-FRAG_TABLE_OVERFLOW: Virtual-Access808: the fragment table has reached its maximum threshold 16

Jan 22 10:41:00.937: %IP_VFR-4-FRAG_TABLE_OVERFLOW: Virtual-Access808: the fragment table has reached its maximum threshold 16

Jan 22 10:47:48.289: %IP_VFR-4-FRAG_TABLE_OVERFLOW: Virtual-Access808: the fragment table has reached its maximum threshold 16

Jan 22 10:53:34.321: %IP_VFR-4-FRAG_TABLE_OVERFLOW: Virtual-Access808: the fragment table has reached its maximum threshold 16

Jan 22 12:05:39.841: %IP_VFR-4-FRAG_TABLE_OVERFLOW: Virtual-Access70: the fragment table has reached its maximum threshold 16

Jan 22 12:44:52.537: %IP_VFR-4-FRAG_TABLE_OVERFLOW: Virtual-Access950: the fragment table has reached its maximum threshold 16

Jan 22 16:07:35.016: %IP_VFR-3-OVERLAP_FRAGMENTS: Virtual-Access1429: from the host xxxxxxxx destined to 168.83.77.205

Jan 22 16:31:08.592: %IP_VFR-3-OVERLAP_FRAGMENTS: Virtual-Access713: from the host yyyyyyyyyyy destined to 2.89.62.26

Jan 22 17:06:47.504: %IP_VFR-4-FRAG_TABLE_OVERFLOW: Virtual-Access1350: the fragment table has reached its maximum threshold 1

LNS#show run | i route

no ip source-route

router ospf 1

router-id 127.1.1.1

ip route 0.0.0.0 0.0.0.0 10.20.10.1

ip route 2222222222222.0 255.255.255.0 next hop of giga0/1 interface

ip route 3333333333333.0 255.255.255.0 next hop of giga0/1 interface

New Member

High CPU Interrupt Rate on 7206vxr

hi  ,

go to virtual-temp 1

and

no ip virtual-reassembly in

and tell me

i faced this issue before ,

tell me what happen when u remove it

regards

New Member

High CPU Interrupt Rate on 7206vxr

the fragmnetation error stopped but still high cpu without any changes.

could you provide me your config of your LNS, my be i can match some differeneces

New Member

High CPU Interrupt Rate on 7206vxr

well ,

give me some topology , u may have routing loop

i mean that give me some info about topology

LNS----------------------------------???

regards

New Member

High CPU Interrupt Rate on 7206vxr

any new news ???

New Member

High CPU Interrupt Rate on 7206vxr

Still have the problem, i don't know what to do.

about topolgy

ISP<--------------->EdgeR <------------->LNS<--------------->LAC

I don't think that i have any routing issue because in non rush hours, the cpu slow down to 20%

Could you provide me your configuration please ? may be can help

New Member

Re: High CPU Interrupt Rate on 7206vxr

Plz post relative commands relate to routing on each of lns and 2nd router
The static routes & ospf
& sh ip protocols

Sent from Cisco Technical Support iPhone App

New Member

High CPU Interrupt Rate on 7206vxr

any news ?

plz post

sh run | i ip route

at the edgeR router

regards

1194
Views
0
Helpful
16
Replies
CreatePlease login to create content