cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
940
Views
7
Helpful
15
Replies

7206VXR NPE-300

palomoj
Level 1
Level 1

Anyone seen this hardware's CPU hit in the 90% range with only 50Mbps traffic, ~50-70Kpps? Router has 1 - POS interface and 1Gbps interface in use. All traffic is legitimate VoIP and GRE.

15 Replies 15

cisco_lad2004
Level 5
Level 5

you do "sh proc cpu" to see which process is actually hitting CPU.

alternatively, has there been any new config changes recently. maybe new policy-maps were added, new tunnels.

HTH

Sam

mohammedmahmoud
Level 11
Level 11

Hi Joe,

I've seen this only when CEF was disabled or not used by the router for any mean (the case that i've seen that the router was short in memory to built the FIB table - full internet routing table), but agree with Sam, lets see the show processes cpu sorted output.

HTH,

Mohammed Mahmoud.

paolo bevilacqua
Hall of Fame
Hall of Fame

Beside the shop proc cpu and the check on CEF as others suggested, there is also to see a show interfaces at peak hours.

If the traffic volumes you mentioned is bidirectional, you are approaching the point where an NPE upgrade is due.

Here's some more information. Thanks for the input everyone.

1. eBGP but only accepting default.

2. No ACL's, NAT, PBR, VPN, etc.

3. Traffic is around 25Mbps bidirectional, peaking around 30Mbps birectional.

4. CEF is enabled globally.

5. Show proc cpu doesn't indicate anything out of hand. I believe this is because it doesn't show interrupts in this summary.

6. 90% of traffic is VoIP (very small packets!) This is what I am thinking is causing the CPU to spike under a lot less PPS then this NPE should be able to handle but wanted to hear from others with similiar experiences with this router and NPE.

A couple more additional info items for this problem:

The only change recently was adding 2 additional VoIP gateways to the network increasing the VoIP traffic another 10Mbps bidirectional.

Netflow indicates legitimate traffic and no malicious, out of the ordinary flows.

Joseph W. Doherty
Hall of Fame
Hall of Fame

According to this listing: http://www.cisco.com/web/partners/downloads/765/tools/quickreference/routerperformance.pdf

a NPE-300 should be good for about 180 Mbps.

What I would wonder about is GRE, especially if GRE is fragmenting packets.

What's your interrupt % vs. CPU %? If there's a significant spread, what's the top software processes using? ("show proc cpu sorted")

Good point Joseph !

I have seen same issues on 7600 SUP720-3BXL, it was MTU mismatch on both end of a VLAN forcing defrag. CPU was over 90% but not listed in show proc cpu just as described in previous posts.

HTH

Sam

Sorry, I should have clarified. GRE is not terminated on the router it is simply routing the GRE traffic through the network.

Interrupt/CPU are just about the same.

Could excessive ARP traffic/incomplete entries and CEF Stalled adjacencies cause that much more CPU cycles?

I'm still waiting to get remote access to the client's switching environment to review their switch, VLAN, and port configurations.

Thanks again for the input.

Agree, GRE shouldn't be an issue if tunnel not terminated on router.

Have you run through all the performance tuning problem areas and guidelines?

e.g.

http://www.cisco.com/en/US/products/hw/routers/ps359/products_tech_note09186a00801c2af0.shtml

http://www.cisco.com/en/US/products/sw/iosswrel/ps1831/products_tech_note09186a00800a7306.shtml

http://www.cisco.com/en/US/products/sw/iosswrel/ps1828/products_tech_note09186a00800a65d1.shtml

In the above, especially "show cef drop" and "show interfaces switching"?

Have you looked at "show buffers" to confirm you're not making/dropping them?

You mentioned netflow. You're also using netflow caching? Defined on all interfaces?

What IOS are you using? (If I remember correctly, later versions complain when run on a NPE-300.)

"Could excessive ARP traffic/incomplete entries and CEF Stalled adjacencies cause that much more CPU cycles?"

Unsure, but interrupt to CEF might certainly slow things down. If you haven't already, you might look at: http://www.cisco.com/en/US/tech/tk827/tk831/technologies_tech_note09186a0080094303.shtml

BTW, that's the same PDF I looked at while I was scratching my head trying to figure out what could cause the CPU to be so high with this amount of traffic. I did some research with others in puck.nether.net and found a few others complaining about poor PPS and throughput with high CPU.

Could the large amount of small packets be the culprit. It makes sense but wanted to get some feedback from others in a large VoIP traffic environment.

yes small packets will kill your router, no doubts!

it also correlates with the addition of your Voice gateways.

Sam

Joe

I would like to go back to your question about:

Could excessive ARP traffic/incomplete entries and CEF Stalled adjacencies cause that much more CPU cycles?

Is it possible that you have a static route (especially a static default route) that points to an outbound interface rather than pointing to a next hop address? I have seen static routes to an interface (especially a LAN interface) cause excessive ARP traffic and lots of incomplete entries (and though I have not seen it, I suspect that stalled CEF adjacencies might go along). If there is a static route to an outbound interface couls you change it to specify a next hop address and let us know if the behavior changes any?

HTH

Rick

HTH

Rick

all static routes point to the next hop ip address, no interfaces

i'm trying to get the client to order NPE-G1 to replace the NPE-300 but client is not convinced its the increased traffic + small packet issue.

:( i can't win this one

Try showing them the date NPE-300 went EOS(2002) and the traffic volumes in use at that time. He probably has replaced already PCs and servers twice since then, why not the poor old router ?

Another thing that I like to do in these cases, is saying, fine you take the responsibility of leaving things like that, sign here, and I will pass the note up the chain.

On these terms none has signed anything else than a PO :)

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: