Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements

Welcome to Cisco Support Community. We would love to have your feedback.

For an introduction to the new site, click here. And see here for current known issues.

Purple

6509 -sup720 problem

We are having a problem where the CPU is being driven to over 90% at times and we can't even get at this box , high input on the CPU is "ip input" . This box has all DFC3 cards in it . Under what circumstances does ip traffic get forwarded to the CPU when you have DFC cards installed ? Anyone have any ideas on how to track something like this down? Have checked all links for errors , everything clean . Looked at spanning tree , don't see an issue there . Hard to get any info off the box when this is happening . You look at any of the interfaces and the traffic is not that high , these are gig links down to access layer boxes that are trunked . Frankly I am running out of ideas on what to do with this , any ideas appreciated .

7 REPLIES
Cisco Employee

Re: 6509 -sup720 problem

There are quite a bit of reasons traffic may be punted to the MSFC for process switching. Some include unsupported features in hardware, IP options, ttl = 1, and ICMP unreachables/redirects. Here is a link with a more complete list:

http://www.cisco.com/en/US/products/hw/switches/ps708/products_tech_note09186a00804916e0.shtml

The best way to determine what is causing the process switching is to dump the packet buffers on the interface(s) that is seeing the large volumes of process level traffic. You can do this one of two ways:

1. "show interface switching" and look for the interfaces with increasing IP Process counter.

or

2. "show interfaces" this is probably easier. You can look at the Input Queue for drops or packets actually in the queue.

Vlan10 is up, line protocol is up

Hardware is EtherSVI, address is 00d0.0061.040a (bia 00d0.0061.040a)

Description: VLAN10: Uplink

Internet address is xxx.xxx.xxx.xxx/xx

MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,

reliability 255/255, txload 1/255, rxload 1/255

Encapsulation ARPA, loopback not set

ARP type: ARPA, ARP Timeout 04:00:00

Last input 00:00:00, output 00:00:00, output hang never

Last clearing of "show interface" counters never

Input queue: 33/75/3385/3367 (size/max/drops/flushes); Total output drops: 0

In this case we have 33 packets in the queue and we are seeing drops and flushes due to the amount of process level traffic. So once I have this information I can dump those packets by entering "show buffer input-interface vlan10 dump". This will dump those packets from the queue so you can take a look at whats in there. Should look like this:

Router# show buffers input-interface vlan10 dump

Buffer information for Small buffer at 0x437874D4

data_area 0x8060F04, refcount 1, next 0x5006D400, flags 0x280

linktype 7 (IP), enctype 1 (ARPA), encsize 14, rxtype 1

if_input 0x505BC20C (GigabitEthernet4/1), if_output 0x0 (None)

inputtime 00:00:00.000 (elapsed never)

outputtime 00:00:00.000 (elapsed never), oqnumber 65535

datagramstart 0x8060F7A, datagramsize 60, maximum size 308

mac_start 0x8060F7A, addr_start 0x8060F7A, info_start 0x0

network_start 0x8060F88, transport_start 0x8060F9C, caller_pc 0x403519B4

source: xxx.xxx.xxx.xxx, destination: xxx.xxx.xxx.xxx, id: 0x0000, ttl: 63,

TOS: 0 prot: 17, source port 63, destination port 63

08060F70: 000A 42D17580 ..BQu.

08060F80: 00000000 11110800 4500002E 00000000 ........E.......

08060F90: 3F11EAF3 64646401 64646402 003F003F ?.jsddd.ddd..?.?

08060FA0: 001A261F 00010203 04050607 08090A0B ..&.............

08060FB0: 0C0D0E0F 101164 ......d

I would suggest sending this output to me as well so I can take a look at it as well. Other useful commands are:

1. show ip traffic

This command is useful to see if things like bad hop count, fragments, unreachables, redirects, and other various common traffic that can cause IP Input.

2. show cef not

3. show cef drop.

Please email me with the buffer output when you get a chance. Sup720 has some good hw rate-limiters that we can use if we can not correct the situation.

Warm Regards

Anthony

answanso@cisco.com

Purple

Re: 6509 -sup720 problem

Hi Anthony great info on the buffer dumping stuff . What I need is a way to be able to get to the box when the cpu gets above 90% , it is all but unusuable via telnet when it happens . I need to be able to dump those buffers when it gets busy . When the problem is not occuring the 720 runs less than 5% all the time . The code level is 12.2.18SXD4 . I did do some of the cef command you described and didn't see too much there . If there is way to allocate more cycles to the vty sessions let me know . Thanks alot...

Purple

Re: 6509 -sup720 problem

In a native IOS 6509 Sup 720 what is the command to be able to log into the L2 SP ? I used to know this but can't seem to remember and find it on CCO .

Cisco Employee

Re: 6509 -sup720 problem

You can use "remote command" or "remote login switch" to get onto the supervisor. Is the switch remote from your current location? I would recommend login through the console port to capture the information instead of telnet. However you can try the "scheduler allocate" command. Try "sheduler allocate 3000 1000", where 3000 is interrupt time and 1000 is process time.

Purple

Re: 6509 -sup720 problem

We have a dialup modem on the console but won't that be tied up as well when the cpu gets real busy ?

New Member

Re: 6509 -sup720 problem

I'm seeing the exact same symptom on our two new Cat6513 with Sup720 running 12.2(17d)SXB10. Both 6513's are trunked to a 6509 with Sup2/MSFC2. CPU utilization on both 6513's are at 99%, 6509 at 1%. Network utilization across the GB trunks are ~50%. The interesting part is after I disabled all ports except for the trunk ports on all three switches the CPU and network utilization remain the same and when I span/sniff the trunk ports on 6509 I see 'phantom' WINS host accouncement broadcast traffic from devices no longer connected to the switch. Fortunately, this issue reared its head in testing phase before going to production. I have a case opened with our reseller but it's not too assuring that you're seeing the same issue with a later IOS release.

Cisco Employee

Re: 6509 -sup720 problem

Just because you are seeing high CPU does not make this an IOS issue. Was your issue in interrupt or Process level? In this case, its not IOS, its the fact that there is a type of traffic being sent to the device that is not supported in hardware. There a lot of reasons for this, could be a ttl with a value of 0 or 1 and we need to send an IP Unreachable in response, could be a feature that the Sup720 doesn't support in hardware or other traffic related issues. The fact is without seeing the show proc cpu from your case, the type of traffic being punted, and all the information you are blindly comparing all issues to the one you are seeing and assuming its due to a defect. I would be happy to look into your case if you are still seeing it. Shoot me an email and I will request some information from you and we will try and narrow it down to why the packet is being punted.

Anthony

answanso@cisco.com

405
Views
0
Helpful
7
Replies
CreatePlease login to create content