ASK THE EXPERT - IOS EMBEDDED MANAGEMENT TECHNOLOGIES

Unanswered Question
Oct 9th, 2009
User Badges:
  • Gold, 750 points or more

Welcome to the Cisco Networking Professionals Ask the Expert conversation. This is an opportunity to get information on design tips and scripting help for embedded management technologies with Cisco expert Joe Clarke. Joe Marcus Clarke has been with Cisco since 1998, working on the network management Technical Assistance Center (TAC) team in North Carolina. As technical lead, he handles world-wide network management escalations particularly those pertaining to CiscoWorks, Tcl scripting, and embedded management technologies. He is CCIE certified (#5384), a certified Java programmer, Solaris system admin, Solaris network admin, and Solaris security admin.


Remember to use the rating system to let Joe know if you have received an adequate response.


Joe might not be able to answer each question due to the volume expected during this event. Our moderators will post many of the unanswered questions in other discussion forums shortly after the event. This event lasts through October 23, 2009. Visit this forum often to view responses to your questions and the questions of other community members.

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 4.9 (9 ratings)
Loading.
Joe Clarke Fri, 10/09/2009 - 12:33
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

This discussion will focus on Tcl scripting in IOS, Embedded Event Manager, Embedded Syslog Manager, Embedded Menu Manager, and Embedded Resource Manager. I will attempt to provide code examples, and help people design and troubleshoot their scripts and policies.

illusion_rox Sun, 10/11/2009 - 02:16
User Badges:

Dear Sir, its an honour to be part of this conversation with you. Thanks alot in advance.


Sir, is there anyway that i have lets say 2 tcl scripts. I want to set one variable in one script and be able to call that variable later at some time from second script ? does IOS provide this feature. For example, i want to calculate how many times a user lets say user_A logs in the router on oct 12. This script will increment a variable everytime user_A logs in. ( i will use SSH logging event to trigger the script). For this purpose i need to retain the value of the variable so that when the next time this script runs, IT REMEMBERS the previous modified value.


Sir, if this conversation is not for this type of queries then pls sorry.

Joe Clarke Sun, 10/11/2009 - 10:26
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

If you are referring to EEM Tcl scripts, then yes. This feature is called contexts. You can save a variable from one EEM policy in a context, then retrieve that context in another policy (or in another execution of the same policy). For example:


Script A:

---------


set count 0

# Do some calculations...

if { $found } {

incr count

# $count is now 1

}


if { [catch {context_save ACTXT count} result] } {

error "Failed to save context: '$result'" $errorInfo

}


Script B:

---------


if { [catch {context_retrieve ACTXT count} result] } {

error "Failed to retrieve context: '$result'" $errorInfo

}


set count $result


puts $count

# Output will be 1


This is the elegant way of passing information between most policies in EEM. Other alternatives are to use files on flash (especially if you need to share large amounts of data), or make the other policies application event or none policies so that you can directly pass variables to them as arguments.

illusion_rox Sun, 10/11/2009 - 19:47
User Badges:

Dear Sir, its quite possible to build network based tools in tcl. If i want to do something advance like lets say, i want to write a packet catching script that will look for any packet of size 600 bytes and will perform the action i define. So can i do it ? does cisco provide a respositery about how much of actual tcl is embedded in IOS ?. I m asking the above question because lets say i want to catch a particular application packet but i dont know from which IP will it be coming. If i use access-list then my source ip would be "any", if i use nbar, i cant still determine from which IP this packet came from ? so i want to write a packet catching program that can tell me from which IP this packet came from..


Also sir, can i ask more questions ? i dont want to miss this opportunity with you.


Thanks in advance



Joe Clarke Sun, 10/11/2009 - 20:09
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

IOS now actually has a built-in packet capture tool called Embedded Packet Capture. It's available in 12.4(20)T and higher. With it, you can capture actual data on the wire, then inspect the packet buffer either from the command line or using Wireshark. See http://www.cisco.com/en/US/docs/ios/netmgmt/configuration/guide/nm_packet_capture_ps6441_TSD_Products_Configuration_Guide_Chapter.html for more details.


And yes, you can use Tcl to process the packets in the capture buffer. You can use the "show monitor capture buffer BUFNAME dump" command within a Tcl script, then take the output, and process it. In order to do so, you'll need to most likely use the Tcl "binary" command to convert the ASCII hex data into raw binary data (which can then be further converted).


However, to the more general question, the answer is no. While Tcl in IOS is quite powerful, you do not have full access to all IOS functions. You cannot control many of the low-level features of IOS. You have to rely on what IOS provides you in terms of show commands, configs, etc. That is, if the EPC feature did not already exist, you wouldn't be able to write your own packet capture app in Tcl.

illusion_rox Tue, 10/13/2009 - 22:51
User Badges:

Dear Sir,


I have one more question, sorry if its not relevant. When we access the router, then to view logs we need terminal monitor command. Is there any way using tcl/EEM to automatically execute this command whenever a user successfully logs in.


If possible can you also shed some light about the null user account that EEM uses to run its applet. Why it needs a vty line to execute scripts/applets ?


Thanks alot in advance

Joe Clarke Wed, 10/14/2009 - 07:42
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

No, this is not possible. As you state, EEM uses a separate VTY to run its commands. So, if you execute "term mon" in an EEM policy, that command will be executed in a separate VTY line, and not for the current user. However, as an alternative, you could add "monitor" to the all of the VTY lines. You would first need to configure:


login on-success log


Then, create an EEM policy to watch for SEC_LOGIN-5-LOGIN_SUCCESS messages:


event manager applet term-mon

event syslog pattern SEC_LOGIN-5-LOGIN_SUCCESS

action 1.0 cli command "enable"

action 2.0 cli command "config t"

action 3.0 cli command "line vty 0 15"

action 4.0 cli command "monitor"

action 5.0 cli command "end"


That would enable term mon for all active VTY lines. You may need to adjust the max VTY line depending on the platform.


In order to run commands asynchronously, EEM must use its own VTY to run CLI commands. Think of the scenario where you're just watching for a syslog message, and you want to make a config change. You wouldn't necessarily want to EEM to require a user to be logged in to do this. So, a separate VTY is needed. And if the device is configured to use AAA command authorization, EEM may also need to use a real username to run commands. For that, you will need to configure:


event manager session cli username USERNAME

illusion_rox Wed, 10/14/2009 - 18:48
User Badges:

Dear Sir, thanks again for detail answer. It surely cleared my doubts. One thing if you can pls tell (i know its irrelevant here), is there any other way of enabling terminal monitor on all lines by default except using EEM ?


Sir, can you also pls shed some light on the username concept used in EEM ? what its importance ?

Joe Clarke Wed, 10/14/2009 - 20:30
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

I don't know of any other way to enable terminal monitoring by default. The monitor config command only applies to currently active VTY lines.


Normally, EEM doesn't need to use a real username to execute CLI commands. However, if AAA command authorization is configured, then it does. If a username is not specified, all commands will fail with a "Command not authorized" error. To get around this, you must use the event manager session cli username command to tel EEM what username to use when it executes CLI commands. This username will be sent to the AAA serve prior to execution. If the commands are authorized according to the AAA server, then EEM will be allowed to execute them.

Jonatan Jonasson Tue, 10/20/2009 - 15:46
User Badges:

Hi,


How can I get a tcl script to parse a syslog message?


For example; Lets say I use EEM applet or EEM policy with event_register_syslog.

And the syslog pattern that triggers the eem is SYS-5-CONFIG_I.

When the eem is triggered, I want it to run a tcl script.

So far, no problem.


But what I want is for the syslog message that triggered the eem to be 'sent' to the tcl script (as a variable) so I can parse the syslog message in the tcl script and work with it.

Joe Clarke Tue, 10/20/2009 - 16:01
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

Every event detector has its own set of variables. If you have a device running EEM 2.4 or higher, you can see all of these variables by running the "show event manager detector DETECTOR detailed" command. For example:


show event manager detector syslog detailed

Tcl event_reqinfo Array Names:

event_id

event_type

event_type_string

event_pub_time

event_pub_sec

event_pub_msec

event_severity

msg

priority


This says that within the event_reqinfo array (used within a Tcl EEM policy), you will have these list of elements. One of them, msg, happens to hold the syslog message which triggered the event.


So, from within your EEM Tcl policy, you could have code like:


array set arr_einfo [event_reqinfo]

set msg $arr_einfo(msg)


Then, the $msg variable would hold the syslog message. You could then do anything you want with it.

Joe Clarke Sun, 10/11/2009 - 10:20
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

This thread is covering Tcl scripting and the embedded management subsystems in IOS. CCA is not one of the topics. Please start a new thread for your CCA question.

Few questions:

1. Is it possible to write the syslog message on a file using EEM/TCL.. also only the syslog messages are supposed to be copied in the file..

2. RTR-->sdh box-1-->sdh box-2-->RTR

In case the link of sdh-box2 disconnects wid RTR or sdh-box-1, only the layer2 link goes down, but the ospf process on both router goes down. we have catered this using CDP check, but we want to block cdp now. anything that could be done using EEM?

Joe Clarke Mon, 10/12/2009 - 06:23
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

1. I'm not sure I follow. One can send syslog messages from within EEM using the syslog action. If you wanted to save syslog messages to a file on flash, you could parse the output of "show logg", and write the desired messages out to a file. Is this what you want to do?


2. Sure. If layer 2 on the local router is going down, you could react to that using EEM and Object Tracking (or an EEM policy which does periodic polling). What version of IOS are these routers running? Exactly what do you want to do when you detect the link has gone down?

Eduardo Aliaga Tue, 10/13/2009 - 08:01
User Badges:
  • Silver, 250 points or more

Do you know any time frame for TCL to be included in non IOS devices, like, for instance, ASA firewall ?

Joe Clarke Tue, 10/13/2009 - 08:13
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

There is an enhancement request for EEM on ASA, but I do not know an ETA on when it will be implemented. However, Tcl, and EEM are already available in IOS-XR, IOS-XE, and EEM is available in NX-OS.


The enhancement bug for ASA support is CSCsh14024. If you talk to your account team, they can bolster this bug with a Product Enhancement Request which adds a business case for implementing this feature.

Jon Marshall Tue, 10/13/2009 - 10:05
User Badges:
  • Super Blue, 32500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

Joe


Thanks for hosting this Ask the Expert.


I'll probably have quite a few more questions as i think of them :-) and happy to be pointed off to links but a couple of questions -


1) I have used TCL with TK quite a lot in the past. How much of the base TCL language is included in the IOS implementation. Are there any major bits that aren't there ?


2) How much overhead does running an EEM script add to router. I appreciate this is a bit of an open-ended question because a lot depends on the script itself and how much it is doing but is the TCL implementation optimised for IOS ?


Can the script run all the time in the background waiting on an event or is it more advisable to simply schedule the script to run at certain times ?


When the script is running is there a process you can view to see just how much memory/CPU it is using ?


Jon

Joe Clarke Tue, 10/13/2009 - 10:32
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

1. IOS is a nearly full implementation of Tcl 8.3.5. there are some things which don't work in IOS, though. Most notably one cannot load binary Tcl packages. The full list of differences can be found at http://www.cisco.com/en/US/docs/ios/netmgmt/configuration/guide/nm_script_tcl.html#wp1055689 . One thing you won't see here is that IOS Tcl lacks most floating point math functions (e.g. pow, sin, tan, etc.). IOS doesn't really handle FP math too well.


2. The overhead is kept minimal. The most important thing to note is that EEM processes run at medium priority, and can preempt low priority processes. However, EEM has quite a few niceties included to prevent run aways. First, each policy has a maxrun timer (default 20 seconds). If a policy tries to run longer than this, it is killed. Second, newer versions of EEM give the user a lot of control over the policy scheduler. You can stop running policies with the event manager scheduler clear command, control how many threads are allocated per policy queue, and even suspend execution of all policies.


If you want to get a good baseline of how much impact your use of EEM will have, you can look at the show proc cpu and mem for the various "EEM*" processes. Each event detector will have its own process, then there will be the server and policy director. Typically, none of these processes ever take much CPU or memory.


EEM scripts are generally run in the background. The notable exceptions are those that use a "none" event detector. These are executed synchronously as requested. Additionally, EDs such as cli and snmp-object can run synchronously to facilitate dropping CLI commands and SNMP object requests respectively.


You MUST NEVER write a tclsh script which runs continuously. Okay, this may be a little strong, but tclsh will run in the foreground synchronously, and will wedge the current VTY while it runs. If you need event-based scripting, move to EEM, and use its Tcl interface.


The EEM processes to look at while a script is running are the EEM Policy Director, EEM Server, and the EEM Callback process. It may also be useful to watch the associated ED process as well, but that should be very short-lived in the whole lifecycle.

Jon Marshall Tue, 10/13/2009 - 11:02
User Badges:
  • Super Blue, 32500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

Joe


Thanks for the prompt response.


Apologies if this is a little basic but i'm still trying to appreciate the difference between EEM and just TCL scripting.


1) EEM allows you to script an action(s) in response to a certain event ie. loss of a route, interface going down etc. Are these events defined within the IOS ie. you can't code them yourself, or you could but then it wouldn't be EEM it would be a simple TCL script ?


If the above is correct is there a list of all the current events that IOS "recognises" in terms of EEM and is there a future roadmap for additional events.


2) Assuming 1 is correct a TCL script rather than EEM would be used for example to perhaps change an acl based on the time of day and remove any existing connections (i appreciate there are time based acls but i'm just using this as an example).


If so how does this script run ie. there is no event as such to initiate it. Do you have to write it so that it runs every half an hour or so to check the time etc..


3) Are these scripts stored in flash ? Also an EEM script - is this also stored in flash ?


Jon

Joe Clarke Tue, 10/13/2009 - 11:14
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

1. EEM allows for event-based scripting based on event detectors which are built in to IOS. You cannot add new EDs. However, you can use the timer ED to run a script periodically, and check for a custom event (e.g. run a show command and parse the output to see whether or not you need to take some kind of action).


EEM policies come in two forms: applet and Tcl script. Applets are relatively simple EEM policies which are configured in the running config of the device (as such they are kept in NVRAM). While EEM 3.0 offers some programmatic hooks for applets, the applet syntax is not a complete programming language.


That's where Tcl comes in. Tcl policies are complete Tcl scripts with some EEM-specific proc calls which must live in the device's local flash. Once registered, these scripts are copied to a private location in memory (for security reasons). Tcl policies allow for a much more powerful interface into scripting IOS since they give you a complete programming language. Everything you can do in an applet can be done in a Tcl script, but the inverse is not true.


Depending on your version of IOS, you can run the command "show event manager detector all" to get a list of all EDs supported by the given device. The EEM homepage at http://www.cisco.com/en/US/products/ps6815/products_ios_protocol_group_home.html is also a good starting point for documentation.


2. It depends. If I am designing a custom command, I will tend to use a tclsh script. However, if anything will require some event-based trigger, I will use EEM. If I can get away with building an applet, I will. Else, I will use EEM's Tcl. In your example of configuring an ACL at a certain time, I would use EEM, and most likely an applet. For example:


event manager applet mod-acl

event timer cron cron-entry "0 8 * * *"

action 1.0 cli command "enable"

action 2.0 cli command "config t"

action 3.0 cli command "no access-list 101"

action 4.0 cli command "access-list 101 permit tcp any 10.1.1.1 eq 80"

action 5.0 cli command "access-list 101 permit tcp 10.1.1.0 0.0.0.255 any"

action 6.0 cli command "access-list 101 permit tcp any any established"

action 7.0 cli command "access-list 101 deny ip any any log"


This policy will be triggered based on time. It will run every day at 08:00 am. The timer ED is very powerful. It allows you to run scripts periodically, in a certain amount of time, or at a certain time.


3. EEM Tcl policies must be kept in local flash. Tclsh scripts can be loaded from any URI which IOS supports (e.g. flash, tftp, http, scp, etc.).

Joe Clarke Fri, 10/16/2009 - 09:47
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

This depends on whether or not your device supports EEM 2.2 with enhanced object tracking or not. If you have a router running 12.4(2)T or higher, doing this becomes very easy. First, you will need to configure your IP SLA collector to test network connectivity. For example, a simple IP Echo collector will work:


ip sla 1

icmp-echo 10.1.1.1 source-interface Serial0


ip sla schedule 1 life forever start now


Then, use EOT to track the state of this collector:


track 1 ip sla 1 reachability


Finally, build an EEM applet policy to watch the tracked object. When the tracked object goes down, you can bring up a failover interface. When the tracked object comes back up, you can shutdown that failover interface. For example:


event manager applet track-down

event track state down

action 1.0 cli command "enable"

action 2.0 cli command "config t"

action 3.0 cli command "int serial1"

action 4.0 cli command "no shut"

action 5.0 cli command "end"


event manager applet track-up

event track state up

action 1.0 cli command "enable"

action 2.0 cli command "config t"

action 3.0 cli command "int serial1"

action 4.0 cli command "shut"

action 5.0 cli command "end"


Of course, you can do much more based on the tracked object state, but this is a simple example which should provide you some direction.


Without EEM 2.2 or without EOT support (e.g. if you have a Cat 6500 running code less than 12.2(33)SXI), you can still use EEM and EOT, but you will not be able to tie to the two together as easily. Instead, you will need an EEM Tcl policy using the timer event detector to periodically run the "show track" command, and parse the output. This isn't as real-time as the applet example above, but should work well.


I've built such a script for a program we're currently rolling out internally within Cisco called EASy (Embedded Automation Systems). I've attached the policy here as a guide. You can use the same IP SLA collector and EOT definition with this policy.



villi1977 Sun, 10/18/2009 - 06:15
User Badges:

Hello.


My cisco is WS-C3560-24TS-S

My ios c3560-ipservicesk9-mz.122-52.SE

My aunomus is 196714

Please hemk my ios dont support bgp after 65635.

Avaliable ios for cisco WS-C3560-24TS-S with suppot bgp after 65635.

Please help my.

Please answer my.

Joe Clarke Sun, 10/18/2009 - 08:05
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

This thread is to discuss embedded management features in IOS. If you have questions about BGP, you should start a thread on the LAN Switching and Routing forum.

villi1977 Sun, 10/18/2009 - 08:39
User Badges:

Problem with ios.


Ios 12.2.2 dont support new bgp.

New bgp is 4 bytes.


Device WS-C3560-24TS-S support only 12.2 ios.


New bgp reliase on 12.4 ios.

Please specify new bgp will be release on 12.2 ios?

Joe Clarke Sun, 10/18/2009 - 08:40
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

As I said before, this is not the appropriate forum or thread for your question. Please start a new thread on LAN, Switching and Routing forum.

pnicolette Mon, 10/19/2009 - 13:44
User Badges:

Joe,


Thank you much for hosting this discussion.


1. How might you script monitoring of "output drops" on an interface, so that when the counter increases by more than n in m seconds, you can email a netflow top report to an administrator?


2. Do you know of any scripts to auto-fix duplex mismatches?

Joe Clarke Mon, 10/19/2009 - 15:39
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

1. You could do this with the Embedded Event Manager interface event detector. For example, to monitor output drops on interface FastEthernet0/0, and send the output of "show ip cache flow" when the drops are greater than or equal to 3 in a period of 10 seconds:


event manager applet monitor-output-drops

event interface parameter output_packet_dropped name FastEthernet0/0 entry-op ge entry-val 3 entry-type increment exit-op lt exit-val 1 exit-type increment poll-interval 10

action 1.0 cli command "enable"

action 2.0 cli command "show ip cache flow"

action 3.0 mail to [email protected] from [email protected] subject "Output drops are $_interface_value on interface $_interface_name" server 10.1.1.1 body $_cli_result


More complex policies could be created using Tcl.


2. I do not, and I'm not sure how easy this would be in the general case. You could use CDP to determine the duplex of the neighbor on an interface WITH a CDP neighbor, but I'm not sure how you would find the duplex at the other end of the link without it.

ex-engineer Fri, 10/23/2009 - 08:57
User Badges:

Hi, Joe:


Please don't take my question the wrong way because I have a lot of respect for all of Cisco's accomplishments and contributions to the industry for the last decade. But I do have something to ask....


I have been working with Juniper routers (J2350, M series, SRX 3400) for the last 6 months and I must say I am pretty impressed with some of their features. Features that one may argue are pretty basic, which Juniper has offered for the last 11 years, but Cisco still does not.


The ones I will address have to do with CLI flexibility and router management.


For example, with Juniper, the command configuration lines you enter do not take effect until you execute a "commit". When you do, a "commit instance" is created and numbered. So, if I enter 50 command configuration lines and I need to roll them all back, all I have to do is enter "rollback 1" and every command is immediately removed! In Cisco, you have to negate each and every line using the "no" keyword. Very slow and clumsy.


Moreover, you can check the sanity of your configurations before you "commit" them by entering "commit check". And if, for example, you applied an ACL to an interface, but never actually created and defined the ACL in the first place, the router will tell you when you do a "commit check."


Lastly, how many times have we entered a configuration line and didnt think something through, or the router reacted unexpectedly, and we got locked out? It's happened to all of us at one point. With Juniper, when you commit, you can do a "commit confirm", and if you get locked out, its OK because the router will automatically rollback within 10 minutes if it never receives the "confirm" from you. With Cisco, you have to do a "reload in and have the entire box reboot. Not a good thing when youre talking about a 7600 processing massive amounts of traffic!


Oh yes -- and this is really amazing -- the Juniper router will keep a history of the last 50 "commits", the name of the person who did the commiting, the date and time the commands were committed, and exactly what commands were entered and committed! Its like TACACS+ in a box. Awesome.


Such features are extremely valuable and make life a lot easier for the engineer. Best yet, they are all "on" by default; no configuration is necessary to enable these services. It's native to JUNOS.


Does Cisco have any plan on implementing such remarkably engineer-friendly mechanisms to their CLI?


I know some of these features exist is IOS XR and the CRS, but not in the Enterprise product line.


Thanks


Joe Clarke Fri, 10/23/2009 - 10:29
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

Many of these features exist today in IOS. The ability to do configuration versioning and rollback is already present (though not on by default). It is very easy to activate and use. See http://www.cisco.com/en/US/docs/ios/12_3t/12_3t7/feature/guide/gtrollbk.html for more details.


Using config archive, you can store up to 14 previous configs, and rollback to them as needed. You can even view context-sensitive diffs between different configs in the archive as well as between startup and running. With config replace, you can create a "safety net" of a risky configuration change. You can say, "revert to the last known good config after two minutes if I do not confirm the change." This is much better than "reload in" as it quickly reverts the problematic config change with very little (or no) downtime.


You can also enable config logging as part of this config archive feature. With config logging, each change is logged. For example:


Router#show archive log config all

idx sess [email protected] Logged command

1 1 [email protected] | logging enable

2 0 [email protected] |!exec: enable

3 0 [email protected] |!exec: enable

4 2 [email protected] |snmp-server community *****

5 4 [email protected] |logging 172.18.123.166

6 0 [email protected] |!exec: enable

7 5 [email protected] |archive

8 5 [email protected] | log config

9 5 [email protected] | logging size 1000

10 0 [email protected] |!exec: enable

11 0 [email protected] |!exec: enable

12 0 [email protected] |!exec: enable


Up to 1000 commands can be logged.

Actions

This Discussion