This document provides the steps to perform an ELAM on the Nexus 7000 M-Series modules, explains the most relevant outputs, and how to interpret their results. Please refer to the following document for an overview on ELAM:
In this example a host on Vlan2500 (10.0.5.101) port Eth4/1 sends an ICMP request to a host on Vlan55 (10.0.3.101) off port Eth3/5. We will use ELAM to capture this single packet from 10.0.5.101 to 10.0.3.101. It's important to remember that ELAM allows us to capture a single frame.
To perform an ELAM on the Nexus7000, you need to first attach to the appropriate module. This requires the network-admin privilege.
N7K# attach module 4
Attaching to module 4 ...
To exit type 'exit', to abort type '$.'
Determine the ingress FE
We expect the traffic to ingress the switch on port Eth4/1. Checking the modules in the system we can see that module 4 is an M-series module. Remember, the Nexus 7000 is fully distributed and the modules, not the supervisor, are responsible for making the forwarding decision for dataplane traffic.
4 48 10/100/1000 Mbps Ethernet Module N7K-M148GT-11 ok
5 0 Supervisor module-1X N7K-SUP1 active *
6 0 Supervisor module-1X N7K-SUP1 ha-standby
For M-Series modules, we want to perform the ELAM on the L2 forwarding engine (FE) with internal codename Eureka. Note that the L2 FE data bus (DBUS) contains original header information before the L2 and L3 lookup and the result bus (RBUS) contains the results after both L3 and L2 lookups.
Nexus 7000 modules can have multiple FEs per module. We need to determine which Eureka ASIC is the FE for port Eth4/1. We can use the following command to verify:
From the output above, we can see that Eth4/1 is on Eureka (L2LKP) instance 0. Note for M-Series modules, the ELAM syntax utilizes 1-based values so instance 0 becomes instance 1 when setting up the ELAM. This is not the case for F-Series modules.
module-4# elam asic eureka instance 1
Configure the Trigger
The Eureka ASIC supports ELAM triggers for IPv4, IPv6, and Other. The ELAM trigger must align to the frame type. If the frame is an IPv4 frame then the trigger must also be IPv4. An IPv4 frame will not be captured with an "other" trigger. The same logic applies to IPv6.
For NX-OS you can utilize the question mark to help parse out the ELAM trigger
module-4(eureka-elam)# trigger dbus dbi ingress ipv4 if ?
(some output omitted)
destination-flood Destination Flood
destination-index Destination Index
destination-ipv4-address Destination IP Address
destination-mac-address Destination MAC Address
ip-tos IP TOS
ip-total-len IP Total Length
ip-ttl IP TTL
source-mac-address Source MAC Address
vlan-id Vlan ID Number
For this example we want to capture the frame based off source and destination IPv4 address so we will only specify those values.
Eureka requires a trigger to be set for the DBUS and the RBUS. There are two different packet buffers (PB) in which the RBUS data may reside. Determining the correct PB instance is dependent on the exact module type and ingress port. As a rule of thumb, the recommendation is to configure PB1 and if the RBUS does not trigger repeat with PB2.
We can display the results via the show dbus and show rbus command. For scenarios where there is a high volume of traffic matching the same trigger, it is possible that the DBUS can trigger on a different frame than the RBUS. Therefore, it is important to check the internal sequence numbers on the DBUS and RBUS data to ensure they match.
module-4(eureka-elam)# show dbus | i seq
seq = 0x05
module-4(eureka-elam)# show rbus | i seq
seq = 0x05
Below is an excerpt of the ELAM data that is most relevant in this example.
From the DBUS data above can we validate the frame was received on Vlan2500 with a source MAC of d0d0.fdb7.3dc2 and a destination MAC of 0000.0c07.ac65. We can also see that this is an IPv4 frame sourced from 10.0.5.101 destined to 10.0.3.101. There are several other fields not included in this output such as TOS value, IP flags, IP length, L2 frame length, etc... that are also often useful to check.
We can also validate what port the frame was received on via the source_index (the source LTL). For Nexus 7000, we can map an LTL to a port or group of ports via the following command:
N7K# show system internal pixm info ltl 0xa21
The above output shows that source_index of 0xa21 maps to port Eth4/1. This confirms that the frame was received on Eth4/1.
From the RBUS data we can validate that the frame was routed to Vlan 55 and the TTL was decremented from 0xff in the DBUS data to 0xfe. We can also see that the source and destination MAC addresses were rewritten to 8478.ac0e.4741 and 0005.73a9.5541, respectively. Finally, we can confirm the egress port from the dest_index (destination LTL):
N7K# show system internal pixm info ltl 0x9ed
The above output shows that the dest_index of 0x9ed maps to port Eth3/5. This confirms that the frame was sent out Eth3/5.
Another command to remember is show system internal pixm info ltl-region, which will show how the switch has allocated the pool of LTL's. This is useful to understand the purpose of an LTL if it does not match to a physical port. A good example is a drop LTL: