Solved: EEM Script to use in an triple play network

lap · ‎08-30-2010

Hi,

I have a customer which have a tripe play network. I attach the following drawing so you can have a better idea how the network looks like.

IPTV, Internet and IP telephony is getting in in our customer network at both PE some you can see in the drawing.
Basicaly our customer deliver the network infrastructure while ISP1, ISP2 and ISP3 deliver the content (IPTV, Internet and IP telephony).

The two PEs which are getting the content in the network from the 3 ISPs are running the following hardware and IOS:

cisco CISCO7606 with WS-SUP720 MSFC3 Daughterboard Rev. 3.2.
IOS version: c7600s72033-advipservicesk9-mz.122-33.SRC5.bin

BGP is used between the PEs and the ISPs router.

The PEs are redundant for each other.

Our issue is when one of the ISP is having some maintenance on their routers one the PEs goes really high on CPU until reaching 100%.

I don't know why that is happening that is why I want to use EEM to dig deeper into it.

So What I want to do is an EEM script that could do the following:

1) The script should be triggered when CPU Threshold is reached (lets say 85 %).
3) Run a show tech and save the output in flash.
2) Then the script should run do the following commando and save the outputs to flash:
        - sh ip bgp vpnv4 vrf VRF10102 neighbors
        - sh bgp vpnv4 unicast vrf VRF10102 summary
        - sh ip bgp vpnv4 vrf VRF10102 neighbors x.x.x.x advertised-routes
        - sh ip bgp vpnv4 vrf VRF10102 neighbors x.x.x.x received-routes
        - sh isis neighbors detail
3) Run "sh log" and save the output into flash.

Anyone has an idea how I could made this script with EEM?

Thanks in advance for your help.

Best regards,

Laurent

Joe Clarke · ‎08-30-2010

I don't think you need a script for this. An applet should work just fine. This kind of thing is commonly done by TAC to aid in data collection for transient problems. This policy should work provided the CPU instance is correct. A 7600 will have at least two CPUs. This applet assumes CPU instance 1 is the desired CPU, but it may be CPU instance 2 you need to monitor.

event manager applet watch-cpu

event snmp oid 1.3.6.1.4.1.9.9.109.1.1.1.1.11.1 get-type exact entry-op ge entry-val 90 exit-op lt exit-val 90 poll-interval 60 maxrun 300

action 1.0 cli command "enable"

action 1.1 cli command "term exec promp time"

action 2.0 cli command "show tech | append disk0:/high_cpu.txt"

action 3.0 cli command "sh ip bgp vpnv4 vrf VRF10102 neighbors | append disk0:/high_cpu.txt"

action 4.0 cli command "sh bgp vpnv4 unicast vrf VRF10102 summary | append disk0:/high_cpu.txt"

action 5.0 cli command "sh ip bgp vpnv4 vrf VRF10102 neighbors x.x.x.x advertised-routes | append disk0:/high_cpu.txt"

action 6.0 cli command "sh ip bgp vpnv4 vrf VRF10102 neighbors x.x.x.x received-routes | append disk0:/high_cpu.txt"

action 7.0 cli command "sh isis neighbors detail | append disk0:/high_cpu.txt"

action 8.0 cli command "show logg | append disk0:/high_cpu.txt"

View solution in original post

Joe Clarke · ‎08-30-2010

I don't think you need a script for this. An applet should work just fine. This kind of thing is commonly done by TAC to aid in data collection for transient problems. This policy should work provided the CPU instance is correct. A 7600 will have at least two CPUs. This applet assumes CPU instance 1 is the desired CPU, but it may be CPU instance 2 you need to monitor.

event manager applet watch-cpu

event snmp oid 1.3.6.1.4.1.9.9.109.1.1.1.1.11.1 get-type exact entry-op ge entry-val 90 exit-op lt exit-val 90 poll-interval 60 maxrun 300

action 1.0 cli command "enable"

action 1.1 cli command "term exec promp time"

action 2.0 cli command "show tech | append disk0:/high_cpu.txt"

action 3.0 cli command "sh ip bgp vpnv4 vrf VRF10102 neighbors | append disk0:/high_cpu.txt"

action 4.0 cli command "sh bgp vpnv4 unicast vrf VRF10102 summary | append disk0:/high_cpu.txt"

action 5.0 cli command "sh ip bgp vpnv4 vrf VRF10102 neighbors x.x.x.x advertised-routes | append disk0:/high_cpu.txt"

action 6.0 cli command "sh ip bgp vpnv4 vrf VRF10102 neighbors x.x.x.x received-routes | append disk0:/high_cpu.txt"

action 7.0 cli command "sh isis neighbors detail | append disk0:/high_cpu.txt"

action 8.0 cli command "show logg | append disk0:/high_cpu.txt"

lap · ‎09-01-2010

Hi Joseph,

Thank you very much for your reply. It was really fast. I will configure the applet you gave me tell you if it works. Actually my customer runs only with one CPU on their C7600.

But I was thinking is their a way to find out which oid does what for a given platform? Is there any doc around regarding this?

Best Regards,

Laurent

Joe Clarke · ‎09-01-2010

I have some material I have presented internally, but I haven't published it externally. CPU information can be tricky. Essentially, you need to map the cpmCPUTotalPhysicalIndex to the entPhysicalDescr. That will tell you name of the CPU. Then, take the instance of the cpmCPUTotalPhysicalIndex, and that will be the index you use on cpmCPUInterruptMonIntervalValue to get that CPUs five second CPU time.

lap · ‎09-08-2010

That sounds a bit complicated. I will let you know if it is the right CPU.

Regards,

Laurent

lap · ‎10-05-2010

Hi Joseph,

So I have finally had time to test your script.

It looks like it is working as I got the output file high_cpu.txt.

I have a some questions:

regarding this line: event snmp oid 1.3.6.1.4.1.9.9.109.1.1.1.1.11.1 get-type exact entry-op ge entry-val 90 exit-op lt exit-val 90 poll-interval 60 maxrun 300

So you are saying EEM that if CPU is equal or greater than 90 % then you will execute the applet, right?

How do you make the percent lower? for example when you want to configure the applet to be launched at 80 %

Best Regards,

Laurent

Joe Clarke · ‎10-05-2010

Yes, if the five-second CPU utilization is greater than or equal to 90% then the policy will execute. If you want to trigger the policy at 80%, change the entry-val. The entry-val controls when to trigger the policy. The exit-val controls when to re-arm the policy. That is, the policy checks the CPU utilization every 60 seconds. If it sees the utilization at 90%, it runs the actions. The next time it checks, if the CPU utilization is still at or above 90% it will not run the actions. The actions will not be run again until CPU utilization goes below 90% then goes back to or above 90%.

lap · ‎10-07-2010

Hi Joseph,

Thanks a lot for your help. I will rate you at max as we were really helpful :-)

Best Regards,

Laurent