×

Warning message

  • Cisco Support Forums is in Read Only mode while the site is being migrated.
  • Cisco Support Forums is in Read Only mode while the site is being migrated.

%SYS-3-CPUHOG because of Inspect process

Unanswered Question
Oct 28th, 2008
User Badges:

Hi


We are having 7206 VXR with NPE G2 getting %SYS-3-CPUHOG: Task is running for (48004)msecs, more than (2000)msecs (1464/11),process = Inspect process.

And links are flapping beacuse of this

Couldnt find much info on CPUHOG because of Inspect process

Pls refer to the attached logs and suggest



Attachment: 
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
Amit Singh Tue, 10/28/2008 - 22:13
User Badges:
  • Cisco Employee,

Madan,


I see some tracebacks also generated by the router. I am unable to decode the tracebacks and didnt find much info on the CPUhog error that you are getting. I would suggest the following :


- Try upgrading the router software to a latest one and sometimes it can take care of any cosmetic problem with the software.


- Open a TAC and report the problem to the TAC.


-amit singh

madan.kumar Tue, 10/28/2008 - 22:30
User Badges:

Hi


We had already upgraded the sw frm 124-15.T5 to 124-22.T.I coudnt find any Bug mathes with 124-22.T.


The blow are the mathes i found related to CPU HOG

CSCsu01812

CSCsm72482

CSCso62511

We are using IP INSPECT for SIP calls the below is the related config

interface Multilink7

no ip redirects

no ip unreachables

no ip proxy-arp

ip accounting output-packets

ip accounting precedence input

ip load-sharing per-packet

ip nat outside

ip inspect voip in

ip virtual-reassembly

load-interval 30

no keepalive

ppp multilink

ppp multilink group 7

ppp multilink fragment disable

interface Serial3/0

ip accounting output-packets

ip accounting precedence input

ip nat outside

ip inspect voip in

ip virtual-reassembly

dsu bandwidth 44210

framing c-bit

cablelength 25

serial restart-delay 0


Pls let me know is anything we can chk before considering TAC


Thanks


Madan

nehakulsum Thu, 10/30/2008 - 09:40
User Badges:

Hello Madan,

%SYS-3-CPUHOG: Task is running for ([dec])msecs, more than ([dec])msecs ([dec]/[dec]),process = [chars].[chars]


The indicated process ran too long without relinquishing the processor. Each process should run for a small amount of time and then relinquish the CPU so other processes can run. This problem can be caused by many things like traffic or system load, hardware or operational configuration, a configuration change, initialization (of many interfaces, cards or ports for example), a high momentary error rate, a sustained abnormal condition or many other conditions.


And when i decode i got for following information:-


%SYS-3-CPUHOG: Task ran for [dec] msec ([dec]/[dec]), process = [chars], PC

= [hex].

Explanation: Every process that runs must relinquish the processor after a certain

period so that other processes can run. 'CPUHOG' is an informational message that

occurs when a certain process holds the CPU for 2000 msec or longer. This problem

can occur due to one or more of these conditions:

Heavy Traffic

System load

Faulty hardware

Improper operational configuration

Configuration change

Initialization of many interfaces

High momentary error rate

Sustained abnormal condition

Software bug in the IOS during normal operation.


Recommended Action: If communication is fine, and no other messages are displayed

before this message, you can ignore the CPUHOG message. If another error message

appears along with this message, decode the other error message using the Error

Message Decoder tool. If the CPU usage is high because of IP input process, then

the router is process-switching a lot of IP traffic. Enable CEF to lower the CPU

usage by this process. Review the Bug Toolkit for potential issues related to

this issue.



STACK DECODE NOTIFICATIONS (if any)


Note: Understanding the 'Fixed In Version' field - Sometimes the fixed-in version

specified may not be available for download from the IOS Upgrade Planner. This

is because, bug fixes are incorporated into software versions which have not been

regression tested. These builds are called interim images and will have a "."

in the build number found between the parenthesis, for example 12.3(8.1). The bug

fix will be in the next available image, and all later images. So if a bug is fixed

in 12.3(8.1), the bug fix is present in 12.3(9), 12.3(10), and so on.

REFERENCE: For more information on how IOS images are named, see Cisco IOS Reference

Guide.


The failure was caused by a software defect.


Also you see there is huge peak on the cpu status. anyways As suggested by amit you could probably think of upgrading IOS on the device.


If you still required you can contact cisco tac.


Thanks

Neha

madan.kumar Thu, 10/30/2008 - 20:44
User Badges:

Hi Neha


Thanks for that, I was just wondering why exactly the INSPECT process causes high cpu?


JUST ATTACHING THE SH IP INSPECT ALL output


sh ip inspect all

Session audit trail is disabled

Session alert is enabled

one-minute (sampling period) thresholds are [unlimited : unlimited] connections

max-incomplete sessions thresholds are [unlimited : unlimited]

max-incomplete tcp connections per host is unlimited. Block-time 0 minute.

tcp synwait-time is 30 sec -- tcp finwait-time is 5 sec

tcp idle-time is 3600 sec -- udp idle-time is 30 sec

tcp reassembly queue length 16; timeout 5 sec; memory-limit 1024 kilo bytes

dns-timeout is 5 sec

Inspection Rule Configuration

Inspection name voip

sip alert is on audit-trail is off timeout 30

udp alert is on audit-trail is off timeout 30


Interface Configuration

Interface Serial3/0

Inbound inspection rule is voip

sip alert is on audit-trail is off timeout 30

udp alert is on audit-trail is off timeout 30

Outgoing inspection rule is not set

Inbound access list is not set

Outgoing access list is not set

Interface Multilink7

--More-- Inbound inspection rule is voip

sip alert is on audit-trail is off timeout 30

udp alert is on audit-trail is off timeout 30

Outgoing inspection rule is not set

Inbound access list is not set

Outgoing access list is not set


Established Sessions

Session 7351A74 (202.69.195.115:9298)=>(192.168.35.67:56576) udp SIS_OPEN

Session AF39E84 (202.69.195.115:13580)=>(192.168.35.67:32440) udp SIS_OPEN

Session 7347BEC (202.69.195.115:9299)=>(192.168.35.67:56577) udp SIS_OPEN

Session 7354164 (202.69.195.115:5743)=>(192.168.35.67:46049) udp SIS_OPEN

Session 734A5A4 (202.69.195.115:13581)=>(192.168.35.67:32441) udp SIS_OPEN

Session 734F64C (192.168.35.67:52240)=>(202.69.195.115:9682) sip-RTP-data SIS_OPEN

Session 734A2DC (202.69.195.115:14070)=>(192.168.35.67:49404) udp SIS_OPEN

Session 734BEAC (202.69.195.47:5060)=>(192.168.20.5:5060) sip SIS_OPEN

Session 7345D54 (202.69.195.115:14503)=>(192.168.35.67:25973) udp SIS_OPEN

Session 734EDF4 (202.69.195.115:9683)=>(192.168.35.67:52241) sip-RTCP-data SIS_OPEN

Session 734A86C (202.69.195.115:11947)=>(192.168.35.67:58373) udp SIS_OPEN

Session 735390C (202.69.195.115:5742)=>(192.168.35.67:46048) udp SIS_OPEN

Session 7353BD4 (202.69.195.115:5701)=>(192.168.35.67:22537) sip-RTCP-data SIS_OPEN

--More-- Session 734C9CC (58.27.96.229:5060)=>(192.168.10.7:5060) sip SIS_OPEN

Session 734601C (192.168.35.67:25856)=>(202.69.195.115:14592) sip-RTP-data SIS_OPEN

Session 7352B24 (202.69.195.115:5700)=>(192.168.35.67:22536) udp SIS_OPEN

Session 734FEA4 (202.69.195.115:14502)=>(192.168.35.67:25972) udp SIS_OPEN

Session 734D4EC (202.69.195.115:11946)=>(192.168.35.67:58372) udp SIS_OPEN

Session 734BBE4 (192.168.35.67:55345)=>(202.69.195.115:5323) sip-RTCP-data SIS_OPEN

Session 7347EB4 (202.69.195.115:14593)=>(192.168.35.67:25857) sip-RTCP-data SIS_OPEN

Session 734AB34 (202.69.195.115:14071)=>(192.168.35.67:49405) udp SIS_OPEN

Session 73514E4 (192.168.35.67:55344)=>(202.69.195.115:5322) sip-RTP-data SIS_OPEN

sip-RTCP-data SIS_OPENING

Session 734B0C4 (192.168.10.7:39344)=>(202.69.195.47:5060) sip SIS_OPENING

Pre-generated Sessions

Pre-gen session 734DD44 192.168.10.7[1024:65535]=>202.69.195.47[5060:5060] sip

Pre-gen session 734F914 202.69.195.47[1024:65535]=>192.168.10.7[5060:5060] sip

Pre-gen session 7346B3C 202.69.195.47[1024:65535]=>121.243.16.135[5060:5060] sip

--More-- Pre-gen session 7352004 192.168.10.7[1024:65535]=>58.27.96.229[5060:5060] sip

Pre-gen session 734922C 58.27.96.229[ Pre-gen session 734CC94 58.27.96.235[1024:65535]=>


Planning to go for 12.4 15 T7 since the Bug tools are not yet updaed for the present 12.4 (22)


Regards


Madan

Actions

This Discussion