Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements
Webcast-Catalyst9k
New Member

Catalyst 3750 high CPU utilization

Hi,

I have a 3750 switch stack (2x3750G-24 (one is master) and 4x3750-48 10/100 PoE) with several VLANs configured and IP Routing enabled. When I carry out a large file copy from a server in one VLAN to a server in another VLAN the CPU utilization goes to 70 - 90%. However a "sho processes cpu sorted" shows the highest process utilizing only 2-3% and the sum for all processes reaches less than 10%.

What else can I use to see what is consuming the CPU?

Also the file copy runs MUCH (10 - 100 times)faster if I copy beween 2 servers on the same VLAN. Is the 3750 limited in terms of Layer 3 routing throughput? When does a switch use switching to route between the 2 VLANs and when does it use the CPU?

Regards,

Nigel Lyons

8 REPLIES

Re: Catalyst 3750 high CPU utilization

hello Nigel,

can u send us the copy of the entire show proc cpu output ?? need to analyze which process is actually eating up the bandwidth .....

3750 has a 32 Gbps backplane throughput.... when packets are transmitted across vlans, the switch definitely uses the cpu and the backplane to transmit it to the destination. if there are too many broadcasts/unnecessary traffic/virus, the cpu can go up !!!!

hope this helps.. all the best..

Raj

New Member

Re: Catalyst 3750 high CPU utilization

Hi Raj,

I have attached a wordpad doc with the output of a sho process and the cpu history. The history shows the CPU use climbing to 65% as I start a copy between 2 servers, one on VLAN1/192.168.1.0/Gi2/0/10 and the other on VLAN50/192.168.50.10/gi2/0/19. The Cpu use before the copy started was 6%. The sorted process list shows nothing that sums to 65% so far as I can see.

This info was gathered at night when the network is quiet just before the backups start.

Regards,

Nigel

Re: Catalyst 3750 high CPU utilization

Hi Nige,

with addition to sachinz post, make sure that you have not turned on the debugging as that might process more CPU.

hope this helps,

rate this post if satisfied

New Member

Re: Catalyst 3750 high CPU utilization

Hi,

I do not have any debugging turned on. See below

Regards,

Nigel

Airport_Rd_3750_Stack#sho debug

Airport_Rd_3750_Stack#

Purple

Re: Catalyst 3750 high CPU utilization

Make sure they aren't using multicasting to copy the servers , if not setup correctly this can bury a cpu . I have seen 1 multicast transfer bury a Sup 720 which has a lot more horsepower than a 3750 . Something is not right a single transfer should have very little effect on the cpu as most traffic should be hardware switched and not hit the cpu . We have the entire web setup for a major company runing thru 3750's and I bet they aren't running 10% cpu..

Silver

Re: Catalyst 3750 high CPU utilization

The output of show proc cpu shows 2 numbers, such as 20%/10%. The first (20%) is the total amount of CPU in use. The second (10%) is the amount used specifically by hardware interrupts. The second number plus all numbers listed below should equal the first number.

In your output, most of the CPU is being used by interrupts. The following link talks about troubleshooting high interrupt utilization. My first guess would be a problem with the cef configuration, so could you check the output of "show ip cef summary"?

Another likely candidate is if you have ACLs applied to the interfaces in question without enabling netflow. Depending on how they are configured, they could drastically impact performance. This is also covered in the troubleshooting page below.

http://www.cisco.com/en/US/products/hw/routers/ps359/products_tech_note09186a00801c2af0.shtml

New Member

Re: Catalyst 3750 high CPU utilization

Hi,

Output of sho cef and sho access-list below. I have no access-lists configured. Can you comments on these?

Regards,

Nigel

Airport_Rd_3750_Stack#show ip cef summary

IPv4 CEF is enabled for distributed and running

VRF Default:

225 prefixes (225/0 fwd/non-fwd)

Default network 0.0.0.0/0

Table id 0, 0 resets

Database epoch: 2 (225 entries at this epoch)

Airport_Rd_3750_Stack#sho access-l

Airport_Rd_3750_Stack#sho access-lists

Airport_Rd_3750_Stack#

New Member

Re: Catalyst 3750 high CPU utilization

Hi Grant,

Below is a "sho int" output for the 2 interfaces the servers are connected to. I cleared counters and rand a 2GB copy between teh 2 and CPU jumped from 6% to 72%. the "sho int" was taken immediately after.

Does this confirm that there is no broadcast/multicast issue?

Regards,

Nigel

Airport_Rd_3750_Stack#sho int gi2/0/10

GigabitEthernet2/0/10 is up, line protocol is up (connected)

Hardware is Gigabit Ethernet, address is 0013.1a79.688a (bia 0013.1a79.688a)

MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,

reliability 255/255, txload 10/255, rxload 1/255

Encapsulation ARPA, loopback not set

Keepalive set (10 sec)

Full-duplex, 1000Mb/s, media type is 10/100/1000BaseTX

Media-type configured as connector

input flow-control is off, output flow-control is unsupported

ARP type: ARPA, ARP Timeout 04:00:00

Last input never, output 00:00:04, output hang never

Last clearing of "show interface" counters 00:04:03

Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: fifo

Output queue: 0/40 (size/max)

5 minute input rate 870000 bits/sec, 1553 packets/sec

5 minute output rate 41619000 bits/sec, 3506 packets/sec

655707 packets input, 46033416 bytes, 0 no buffer

Received 7 broadcasts (0 multicast)

0 runts, 0 giants, 0 throttles

0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

0 watchdog, 0 multicast, 0 pause input

0 input packets with dribble condition detected

1478235 packets output, 2189196995 bytes, 0 underruns

0 output errors, 0 collisions, 0 interface resets

0 babbles, 0 late collision, 0 deferred

0 lost carrier, 0 no carrier, 0 PAUSE output

0 output buffer failures, 0 output buffers swapped out

Airport_Rd_3750_Stack#sho int gi2/0/19

GigabitEthernet2/0/19 is up, line protocol is up (connected)

Hardware is Gigabit Ethernet, address is 0013.1a79.6893 (bia 0013.1a79.6893)

Description: ### SAP1 NIC2

MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,

reliability 255/255, txload 1/255, rxload 10/255

Encapsulation ARPA, loopback not set

Keepalive set (10 sec)

Full-duplex, 1000Mb/s, media type is 10/100/1000BaseTX

Media-type configured as connector

input flow-control is off, output flow-control is unsupported

ARP type: ARPA, ARP Timeout 04:00:00

Last input never, output 00:00:09, output hang never

Last clearing of "show interface" counters 00:04:03

Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: fifo

Output queue: 0/40 (size/max)

5 minute input rate 40916000 bits/sec, 3443 packets/sec

5 minute output rate 854000 bits/sec, 1526 packets/sec

1476665 packets input, 2189057616 bytes, 0 no buffer

Received 1 broadcasts (0 multicast)

0 runts, 0 giants, 0 throttles

0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

0 watchdog, 0 multicast, 0 pause input

0 input packets with dribble condition detected

655217 packets output, 45921769 bytes, 0 underruns

0 output errors, 0 collisions, 0 interface resets

0 babbles, 0 late collision, 0 deferred

0 lost carrier, 0 no carrier, 0 PAUSE output

0 output buffer failures, 0 output buffers swapped out

Airport_Rd_3750_Stack#

732
Views
14
Helpful
8
Replies
CreatePlease to create content