Working with the Embedded Event Manager (EEM)

Cisco IOS has plenty of gems contained within, but few are as fun, and as endlessly useful as the Embedded Event Manager, or EEM. To define it simply, EEM is a technology that allows you to run a script or a set of commands upon an event. Basically, it’s an IOS scripting language that allows you to insert additional functionality to the IOS.

I think an example is in order. Lets say, you want to make sure loopback0 never gets shut down accidentally. We can create a EEM applet to watch for the syslog message that loopback0 has been shutdown, and automagicly bring it back up!

event manager applet WatchLo0
 event syslog pattern "Interface Loopback0.* down" period 1
 action 2.0 cli command "enable"
 action 2.1 cli command "config t"
 action 2.2 cli command "interface lo0"
 action 2.3 cli command "no shutdown"
 action 3.0 syslog msg "Interface Loopback0 was brought up via EEM"
!

When the lo0 interface is shutdown manually it should trigger the EEM applet and perform the actions that we have defined. Lets go ahead and test this out.

R1(config)#int lo0
R1(config-if)#shut
R1(config-if)#end
R1#
*Jun  6 11:04:52.006: %SYS-5-CONFIG_I: Configured from console by console
*Jun  6 11:04:53.052: %LINK-5-CHANGED: Interface Loopback0, changed state to administratively down
*Jun  6 11:04:55.536: %LINK-3-UPDOWN: Interface Loopback0, changed state to up
*Jun  6 11:04:56.586: %HA_EM-6-LOG: WatchLo0: Interface Loopback0 was brought up via EEM

*Jun  6 11:04:56.594: %SYS-5-CONFIG_I: Configured from console by vty0

Before we dive into the details, lets take a look at another quick example. This time we want to essentially disable reload on the router. Although this would work, using AAA would be the preferred method.

event manager applet DisableReload
 event cli pattern "reload" sync no skip yes occurs 1
 action 1.0 syslog msg "$_cli_msg has been disabled."

Instead of looking for a pattern in syslog, this time we’re waiting for a pattern entered onto the CLI. Let’s break down the options on that event real quick.

• event cli pattern: Defines the event criteria to initialize the EEM applet.
• sync:Specifies if the policy should be executed synchronously before the CLI commands executes
• skip: Indicates if the CLI commands should be executed
• occurs: Indicates the number of occurrences before the EEM applet is triggers.

When we attempt to reload the router, the results are as expected:

R1#reload
R1#
*Jun  6 12:19:55.579: %HA_EM-6-LOG: DisableReload: reload has been disabled.

Building EEM applets

The first step in creating an applet is obviously to define the applet itself. This is done through the configuration command event manager applet name, where name is the actual name of your applet. Next we need to define the event that we’re looking for. This event could be any number of things from syslog pattern matching, a CLI event, or even a timed entry. The full list of event types are listed here.

• cli – watches CLI input for regex pattern
• counter – watch a named counter
• interface – generic interface counters / thresholds (absolute or incremental)
• ioswdsysmon – watchdog – CPU / memory values/ thresholds
• ipsla – watch for IP SLA events
• nf – watch for NetFlow events
• none – allows an event to be run manually using the event manager applet name run command.
• oir – hardware – online insertion removal events
• resource – event from embedded resource manager
• rf – dual RP – redundancy framework
• routing – changes to RIB
• rpc – invoke from off-box using SSH/SOAP encoding to exchange XML messages
• snmp – monitor a MIB object for values / thresholds
• snmp-notification – match info in trap/inform messages
• syslog – screen for regex match
• timer – absolute time of day, countdown, watchdog, CRON
• track – Tracking object event

Lets take a look at two of the event detectors a bit closer.

CLI Event

Within the CLI event detector we have the option of Synchronous/Asynchronous. With synchronous turned on the CLI command matching the event pattern is not executed until the policy exits. Whether the command runs or not depends on the exit status of the applet. With Asynchronous the EEM events are allowed to run first, then the CLI command is executed, unless the command was suppressed using the skip yes directive.

Timer Event

Within the timer event we have various options we can use to time events. These options are listed here.

• absolute – execute an event at an exact time and date. Since July 15 2010 only comes once, this event will only ever be executed once.
• countdown – counts down to 0
• watchdog – counts down to 0 and then resets to initial value
• cron – uses the cron timer (see below)

Cron Timer

For the non *nix geeks out there, lets take a look at how cron is configured. A cron timer is composed of 5 numeric ranges in the following order.

1. minute………….. 0 – 59
2. hour……………… 0 – 23
3. day of month….. 1 – 31
4. month of year…. 1 – 12
5. day of week……. 0 – 6 (where Sunday == 0)

Any individual field can use an asterisk which represents the wildcard symbol, meaning match any value. You can also use hypens to specify a range, or commas in a list of values. Lets take a look at a couple example cron event timers.

event timer cron cron_entry "45 18 15 * *"      --- 18:45 or 6:45 PM on the 15th of every month 
event timer cron cron_entry “15 1 15 1 *”       --- 1:15 AM, January 15
event timer cron cron_entry “0 9 * * 1-5"       --- 9:00 AM, Monday-Friday
event timer cron cron_entry “5 8,16 * * 0,6"    --- 8:05 AM or 4:05 PM, Saturday or Sunday

EEM Actions

When compared to EEM events, actions are fairly straightforward. When using multiple actions lines they will be sorted using their label in alphabetical order. While most examples use numeric labels such as 1.0, 1.1, 1.2, 2.0 – using alphanumeric values is supported. Just remember number, before letters, 1x.0 before 2.0, and capital letters before lower case.

OK, so let’s look at some of the individual actions, grouped by function.

• cli – execute a CLI command
• gets/puts – used to send to or pull from tty.
• mail/syslog/snmp-trap/cns-event – used to send messages
• increment/decrement/append – changing variables
• if/else/elseif/while/end/break/continue/foreach – conditional operators
• wait – pause for a period of time
• track – read or set a tracking object
• regexp – match
• reload/force-switchover – system actions
• add/subtract/multiply/divide/set – perform actions on variables (stored in $_result / $_remainder if applicable)

Simple Examples

This example will hide applets from showing up on the running config by intercepting the show run command and replacing it with filtered output. Very sneaky sneaky.

event manager applet NoAppletsHere
 event cli pattern "show run" sync yes
 action 111 cli command "enable"
 action 112 cli command "show run | excl applet|event|action"
 action 113 puts "$_cli_result"
 action 114 set _exit_status "0"

A user executed script that prompts you for a number of loopbacks to create.

event manager applet CreateLoopbacks
 event none
 action 1.0 puts "How many Loopback interfaces do you wish to create?"
 action 1.1 puts nonewline "> "
 action 1.2 gets num 
 action 2.0 cli command "enable"
 action 2.1 cli command "conf t"
 action 3.0 set i "1"
 action 3.1 while $i le $num
 action 3.2  cli command "interface Loopback $i"
 action 3.3  cli command "ip address $i.$i.$i.$i 255.255.255.255"
 action 3.4  increment i 1
 action 3.5 end
 action 4.0 cli command "end"

Several of these scripts require EEM 3.0 or later. Please ensure you’re running IOS version 12.4(22)T/12.2(33)SRE or later.

A simple, no action script that prevents write erase from being run.

event manager applet NOWRE
event cli pattern "write erase" sync no skip yes

Advanced Example

This example uses an IP SLA to watch an ip address and email us troubleshooting information when it goes down. This could be very helpful for network issues that are hard to catch in the act. I found this example on the Cisco Learning Network and felt the need to share it here. The first thing we need to configure is the IP SLA to watch 198.19.8.4 for packet loss. We also need to configure advanced object tracking to ensure that the host is down, and not just flapping. Here we’re saying that the server is down if it is unreachable for 8 seconds, and it is up if it is reachable for a total of 10 seconds.

R1(config)#ip sla 11
R1(config)#icmp-echo 198.19.8.4
R1(config)#timeout 500
R1(config)#frequency 3
R1(config)#ip sla schedule 11 life forever start-time now
R1(config)#track 1 ip sla 11 reachability
R1(config)#delay down 8 up 10

Hopefully you have an internal email server that this router can reach. Obviously we’re not doing SMTP authentication here…

R1(config)#event manager environment _email_server 10.1.1.4
R1(config)#event manager environment _email_to [email protected]
R1(config)#event manager environment _email_from [email protected]

Now we just have to define the EEM applet that uses the advanced object tracking event!

event manager applet email_server_unreachable
event track 1 state down
action 1.0 syslog msg "Houston we have a problem. Ping failed, server unreachable!"
action 1.1 cli command "enable"
action 1.2 cli command "del /force flash:server_unreachable"
action 1.3 cli command "show clock | append server_unreachable"
action 1.4 cli command "show ip arp 198.19.8.4 | append server_unreachable"
action 1.5 cli command "show ip route 198.19.8.4 | append server_unreachable"
action 1.6 cli command "show interface FastEthernet0/1/1 | append server_unreachable"
action 1.7 cli command "more flash:server_unreachable"
action 1.8 mail server "$_email_server" to "$_email_to" from "$_email_from" subject "Server Unreachable: ICMP-Echos Failed" body "$_cli_result"
action 1.9 syslog msg "Server unreachable alert has been sent to email server!"

Conclusion

Obviously EEM is a very powerful tool and could provide some essential functionality for any network engineer. If this has sparked your imagination or curiosity, I highly recommend you visit the Cisco EEM Community page . As usual, if you have any questions, comments, or concerns please leave me a comment below and I’ll get back with you as soon as I can.