Problems with 7206vxr-npe400

sergio.danelli · ‎08-10-2007

Hello,

The last month or so we have had some issues with our 7200. All the

protocols(LDP,OSPF,sometimes BGP) appear to bounce with dead timers

and the CPU goes up to 100%, weird thing is that it happens during off-

peak hours (Sunday, early morning weekdays).

We are mainly a voice provider with some internet. The router has 4

channelized DS-3s to the DS1 level fully populated and 2 fastethernet

uplinks. The fastethernet uplinks show a number of drops with

flushes.

the two fast ethernet interfaces (counters were cleared about a day

before this):

Input queue: 0/150/28287142/77 (size/max/drops/flushes); Total output

drops: 0

Input queue: 0/150/987356/1703 (size/max/drops/flushes); Total

output drops: 0

xxxx-xxx#show cef drop

CEF Drop Statistics

Slot Encap_fail Unresolved Unsupported No_route No_adj

ChkSum_Err

RP 320263 715965 0 88929869

0 0

xxxx-xxxxf#show cef not-cef-switched

CEF Packets passed on to next switching layer

Slot No_adj No_encap Unsupp'ted Redirect Receive Options

Access Frag

RP 81 0 0 20 1182639 0

0 0

At first we thought it was an OSPF issue and LDP, BGP were just

bouncing because they didn't have route resolution since the routes

were not there, but I am pretty sure the protocols bouncing is a bi-

product of something else.

We also have other 7200s with similar load and same version have no problems at all.

Any help will be appreciated.

thanks

wochanda · ‎08-10-2007

Is the ethernet interface with all the drops connected to a switched network? If so, there was probably a bridging loop.

Look in your switch logs for mac-move notifications, or unusually high numbers of input packets in any particular interfaces.

sergio.danelli · ‎08-10-2007

yeah it is, but so is a number of other 7200s with equal configuration that didn't show this problem.

I will take a look at the switch.

thanks

deveshkumar · ‎08-10-2007

Hi,

Sure for one thing ..i.e your protocol behaves wierdly bcos CPU % goes very high. Trush me, u loose ospf/bgp/eigrp neigbourship even if CPU utilisation reaches, 80% or high.

Now u need to find out reason for high CPU utilisation rather than reason for OSPF/BGP lossing its neighbours and routes.

Pls share show ver of that time / also if u can share .

One issue could be of bandwidth points.. refer to attachment for bandwidth points

sergio.danelli · ‎08-10-2007

ystem returned to ROM by reload at 22:07:18 PST Fri Mar 2 2007

System restarted at 22:09:26 PST Fri Mar 2 2007

System image file is "disk0:c7200-p-mz.123-22.bin"

cisco 7206VXR (NPE400) processor (revision A) with 229376K/32768K bytes of memory.

Processor board ID 20402899

R7000 CPU at 350MHz, Implementation 39, Rev 3.2, 256KB L2 Cache

6 slot VXR midplane, Version 2.1

Last reset from power-on

X.25 software, Version 3.0.0.

Bridging software.

Primary Rate ISDN software, Version 1.1.

PCI bus mb0_mb1 (Slots 0, 1, 3 and 5) has a capacity of 600 bandwidth points.

Current configuration on bus mb0_mb1 has a total of 580 bandwidth points.

This configuration is within the PCI bus capacity and is supported.

PCI bus mb2 (Slots 2, 4, 6) has a capacity of 600 bandwidth points.

Current configuration on bus mb2 has a total of 380 bandwidth points

This configuration is within the PCI bus capacity and is supported.

Please refer to the following document "Cisco 7200 Series Port Adaptor

Hardware Configuration Guidelines" on Cisco.com <http://www.cisco.com>

for c7200 bandwidth points oversubscription and usage guidelines.

3 FastEthernet/IEEE 802.3 interface(s)

112 Serial network interface(s)

8 Channelized T1/PRI port(s)

4 Channelized T3 port(s)

125K bytes of non-volatile configuration memory.

46976K bytes of ATA PCMCIA card at slot 0 (Sector size 512 bytes).

125440K bytes of ATA PCMCIA card at slot 1 (Sector size 512 bytes).

8192K bytes of Flash internal SIMM (Sector size 256K).

Configuration register is 0x102

sergio.danelli · ‎08-10-2007

thanks for the bandwidth points DOC, but it does not seem to be a problem with that. Both sides = less than 300 each.

deveshkumar · ‎08-12-2007

From ur show version ,

Ur Bandwidth points in consuption is 580 and are within the capacity of 600.

Can u share output of

sh proc cpu | ex 0.00

also i possible sh run, ..or sh tech

sergio.danelli · ‎08-13-2007

devesh,

I can mail you a scrubbed version of show tech directly if you give me an email address.

BTW router was upgraded to the last version of 12.3-22.

here is a show proc cpu | ex 0.00

7200-01sf#show proc cpu | ex 0.00

CPU utilization for five seconds: 54%/52%; one minute: 50%; five minutes: 47%

PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process

3 135840 3355093 40 0.15% 0.06% 0.06% 0 OSPF Hello

30 88396 92317 957 0.15% 0.03% 0.02% 0 Net Background

33 468496 215507 2173 0.15% 0.21% 0.22% 0 Per-Second Jobs

59 644464 9254719 69 0.23% 0.27% 0.27% 0 IP Input

61 133824 811511 164 0.07% 0.06% 0.06% 0 CDP Protocol

111 187528 4414912 42 0.07% 0.09% 0.08% 0 OSPF Hello

133 79756 2012974 39 0.07% 0.05% 0.04% 0 FR LMI

134 76296 2122448 35 0.07% 0.07% 0.05% 0 FR Broadcast Out

151 227172 1122442 202 0.23% 0.13% 0.08% 0 OSPF Router

valstan · ‎08-13-2007

Do you have any pools defined on this box and how do they perform?

sergio.danelli · ‎08-13-2007

pools as in NAT? we don't