Very high CPU utilisation on Cat6509 layer three module.

Unanswered Question
Feb 19th, 2004
User Badges:

I have three MSM modules in Cat6509s which occasionally reach 99% utilisation. One of these modules only sits on VLAN1 as a host (does not participate in routing - ie a spare unit) The only thing that I can think of which would cause the high utilisation is broadcast/flooded network traffic.


Although the router is only configured to process VLAN1 packets the switch has all vlans enabled on the internal trunk link to the MSM - This allows multiple packets to enter the router, even though they are not to be processed.


Our company utilises the Microsoft WLBS method of HA between application servers on one of the VLANs on the switch - this can cause up to 3,000 packets per second/25Mbps to flood into all the MSM modules (WBLS uses a nonexistant virtual destination MAC address for the HA server cluser which does not exist in the CAM tables of the switches causing all traffic to flood).


The question I have is - is this flooded traffic enough to cause problems for the router, even though the router should discard them or would there be another reason for the high CPU values. An expample of the show proc cpu is given below:


Many thanks

R hughes


Europa# sh proc cpu

CPU utilization for five seconds: 97%/76%; one minute: 86%; five minutes: 47%

PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process

1 12972 2151011 6 0.00% 0.00% 0.00% 0 Load Meter

2 176 140 1257 0.24% 0.06% 0.02% 2 Virtual Exec

3 27371260 2200973 12436 0.32% 0.28% 0.33% 0 Check heaps

4 0 1 0 0.00% 0.00% 0.00% 0 Pool Manager

5 0 2 0 0.00% 0.00% 0.00% 0 Timers

6 26658632 54577884 488 0.32% 0.27% 0.23% 0 ARP Input

7 0 2 0 0.00% 0.00% 0.00% 0 Serial Backgroun

8 0 2 0 0.00% 0.00% 0.00% 0 DDR Timers

9 0 2 0 0.00% 0.00% 0.00% 0 Entity MIB API

10 0 1 0 0.00% 0.00% 0.00% 0 OIR Handler

11 0 2 0 0.00% 0.00% 0.00% 0 EPIF reset proce

12 0 1 0 0.00% 0.00% 0.00% 0 IPC Zone Manager

13 15160 10751670 1 0.00% 0.00% 0.00% 0 IPC Periodic Tim

14 0 1 0 0.00% 0.00% 0.00% 0 IPC Seat Manager

15 46696 2264921 20 0.00% 0.00% 0.00% 0 HC Counter Timer

16 0 1 0 0.00% 0.00% 0.00% 0 SERIAL A'detect

17 504 20 25200 0.00% 0.00% 0.00% 0 EPAM Card Manage

18 8 16 500 0.00% 0.00% 0.00% 0 EPIF Enable If p

19 0 1 0 0.00% 0.00% 0.00% 0 Critical Bkgnd

20 16804 1131178 14 0.00% 0.00% 0.00% 0 Net Background

21 4364 1184 3685 0.00% 0.01% 0.00% 0 Logger

22 25116 10751949 2 0.00% 0.00% 0.00% 0 TTY Background

PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process

23 18872 10751670 1 0.00% 0.00% 0.00% 0 Per-Second Jobs

24 99144 10751897 9 0.00% 0.02% 0.00% 0 LINE Monitor Dea

25 1166064 10751896 108 0.16% 0.05% 0.01% 0 Per VC Counter

26 0 2 0 0.00% 0.00% 0.00% 0 ATM OAM Input

27 0 2 0 0.00% 0.00% 0.00% 0 ATM OAM TIMER

28 2092 2161438 0 0.00% 0.00% 0.00% 0 ATM Periodic

29 0 1 0 0.00% 0.00% 0.00% 0 ATM ARP INPUT

30 21488 649549 33 0.00% 0.00% 0.00% 0 PINNACLE GMAC Pr

31 5848904 358518 16314 0.00% 0.03% 0.04% 0 THERMOSTAT monit

32 10684 10775428 0 0.00% 0.00% 0.00% 0 LSIPC Periodic T

33 20870020 239618666 87 0.57% 0.18% 0.10% 0 LSIPC RECV PROCE

34 0 2 0 0.00% 0.00% 0.00% 0 LSIPC BCAST PROC

35 10716 10751625 0 0.00% 0.00% 0.00% 0 LSS port status

36 0 1 0 0.00% 0.00% 0.00% 0 CES Line Conditi

37 73896 1896107 38 0.00% 0.00% 0.00% 0 Net Input

38 15460 2151012 7 0.00% 0.00% 0.00% 0 Compute load avg

39 2925888 179210 16326 0.00% 0.01% 0.00% 0 Per-minute Jobs

40 56304 1773591 31 0.00% 0.00% 0.00% 0 CDP Protocol

41 149317272 395768095 377 3.83% 4.79% 2.97% 0 IP Input

42 139980 1144936 122 0.00% 0.00% 0.00% 0 SCP Process

43 0 13 0 0.00% 0.00% 0.00% 0 LSS VLAN Reload

44 23362052 2682517 8709 0.24% 0.19% 0.13% 0 LSS stat L3 proc

45 193310144 26402039 7321 0.89% 1.32% 1.03% 0 LSS stat L2 proc

PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process

46 387068 185200 2090 0.00% 0.00% 0.00% 0 IP Background

47 64 4903 13 0.00% 0.00% 0.00% 0 TCP Timer

48 152 265 573 0.00% 0.00% 0.00% 0 TCP Protocols

49 0 1 0 0.00% 0.00% 0.00% 0 Probe Input

50 3660 92485 39 0.00% 0.00% 0.00% 0 RARP Input

51 1561184 1373573 1136 0.00% 0.00% 0.00% 0 BOOTP Server

52 2836 179210 15 0.00% 0.00% 0.00% 0 IP Cache Ager

53 8 168 47 0.00% 0.00% 0.00% 0 AUTOSTATE proces

54 8008 187700 42 0.00% 0.00% 0.00% 0 Adj Manager

55 0 1 0 0.00% 0.00% 0.00% 0 Resource Mgmt ba

56 0 2 0 0.00% 0.00% 0.00% 0 ATMCORE OAM Proc

57 0 2 0 0.00% 0.00% 0.00% 0 ATMCORE OAM Ping

58 0 2 0 0.00% 0.00% 0.00% 0 SSCOP Input

59 0 2 0 0.00% 0.00% 0.00% 0 SSCOP Output

60 2396 179211 13 0.00% 0.00% 0.00% 0 SSCOP Timer

61 29912 10751656 2 0.00% 0.00% 0.00% 0 ATMSIG Timer

62 0 2 0 0.00% 0.00% 0.00% 0 ATMSIG Input

63 0 2 0 0.00% 0.00% 0.00% 0 ATMSIG Output

64 0 2 0 0.00% 0.00% 0.00% 0 ATM Soft VC Time

65 0 2 0 0.00% 0.00% 0.00% 0 ATM Rerouting Ti

66 127672 1075256 118 0.00% 0.03% 0.00% 0 LSS IPX Reload P

67 0 1 0 0.00% 0.00% 0.00% 0 LSS IPX BVI Proc

68 11924324 369822234 32 0.65% 0.37% 0.21% 0 LSS TB RECV PROC

PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process

69 18838440 15607440 1207 0.32% 0.33% 0.30% 0 CEF process

70 2332608 25574943 91 1.22% 0.22% 0.05% 0 LSS IP Backgroun

71 0 1 0 0.00% 0.00% 0.00% 0 SNMP ConfCopyPro

72 103532 11806850 8 0.00% 0.00% 0.00% 0 LSS IP Multicast

73 12360 10751953 1 0.00% 0.00% 0.00% 0 TTY Background

74 13400 10751672 1 0.00% 0.00% 0.00% 0 Per-Second Jobs

75 21751160 68848631 315 0.32% 0.36% 0.21% 0 Standby

76 4168 179210 23 0.00% 0.00% 0.00% 0 CEF Scanner

77 512020 62698817 8 0.49% 0.23% 0.11% 0 OSPF Hello

78 39288 179278 219 0.00% 0.00% 0.00% 0 OSPF Scanner

79 0 1 0 0.00% 0.00% 0.00% 0 SNMP Timers

80 35800 435824 82 0.00% 0.04% 0.02% 0 IP SNMP

81 0 2 0 0.00% 0.00% 0.00% 0 SNMP Traps

82 1546040 13244394 116 1.88% 1.41% 0.38% 0 OSPF Router

83 0 1 0 0.00% 0.00% 0.00% 0 DSS Process

88 385432 184851 2085 0.16% 0.01% 0.00% 0 IP-RT Background


  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
glen.grant Thu, 02/19/2004 - 18:51
User Badges:
  • Purple, 4500 points or more

You might want to turn netflow on temporarily to see what is generating all the traffic , possible you may have infected hosts on network . Doubt The servers would cause any major problems we have those on our our network and they don't bother the 6509's at all though we run MSFC2 instead of the old MSM's .

Actions

This Discussion