On the Nexus 7000 you may see ICMP packet loss when pinging from the CPU to another device depending on the speed in which this traffic is responded and how much ICMP traffic is being sent to the switch at that moment. This is due to the default CoPP (Control Plane Policing) service policy that is enabled by default on the N7k.
The following will be seen when attempting to ping another switch from the N7k:
nexus#ping 18.104.22.168 c 1000 PING 22.214.171.124 (126.96.36.199): 56 data bytes 64 bytes from 188.8.131.52: icmp_seq=0 ttl=254 time=0.852 ms 64 bytes from 184.108.40.206: icmp_seq=1 ttl=254 time=0.977 ms 64 bytes from 220.127.116.11: icmp_seq=2 ttl=254 time=4.101 ms <snip> 64 bytes from 18.104.22.168: icmp_seq=22 ttl=254 time=3.002 ms Request 23 timed out Request 24 timed out Request 25 timed out <snip> --- 22.214.171.124 ping statistics --- 1000 packets transmitted, 987 packets received, 0.98% packet loss round-trip min/avg/max = 0.847/1.112/11.473 ms
The reason for these drops is that ICMP is policed on the N7k very aggressively. Below is the portion of the CoPP policy which handles ICMP:
IP access list copp-system-acl-icmp 10 permit icmp any any echo 20 permit icmp any any echo-reply
class-map type control-plane match-any copp-system-class-monitoring match access-group name copp-system-acl-icmp match access-group name copp-system-acl-icmp6 match access-group name copp-system-acl-traceroute
class copp-system-class-monitoring set cos 1 police cir 130 kbps bc 1000 ms conform transmit violate drop
We can see that this policy polices ICMP down to 130kbps and that we will drop this traffic if it goes over this amount. Also keep in mind that these policers are performed a per-module basis. That means that each module is limited to 130kbps independently.
We can look to see if we are dropping any traffic due to the policy-map with the following command:
7000-1# show policy-map interface control-plane
Below, I am going to pipe for only the monitoring class:
7000-1# sh policy-map interface Control-plane | b monitoring class-map copp-system-class-monitoring (match-any) match access-grp name copp-system-acl-icmp match access-grp name copp-system-acl-icmp6 match access-grp name copp-system-acl-traceroute set cos 1 police cir 130 kbps , bc 1000 ms module 2 : conformed 397890 bytes; action: transmit violated 12386 bytes; action: drop
As we can see we have violated this policer on module 2 due to this is where 126.96.36.199 is attached:
7000-1# sh ip arp 188.8.131.52
Flags: * - Adjacencies learnt on non-active FHRP router # - Adjacencies Throttled for Glean D - Static Adjacencies attached to down interface
IP ARP Table Total number of entries: 1 Address Age MAC Address Interface 184.108.40.206 00:03:45 0000.0c07.acca Ethernet 2/48
However, we have not sent any ICMP traffic on any other module, since these are independently policed.
We can remove this order to prove that this is the cause of the drops, we can remove this policy-map for a short period of time and perform the ping again.
However, first lets look at the name of the policy-map used for COPP. This can be done using the "show run copp" command.
This command will show the entire CoPP policy. At the bottom of this output will show the same of the service-policy attached to the control-plane:
7000-1# show run copp <snip> control-plane service-policy input copp-system-policy
Now that we have the name lets remove this policy and attempt the ping again:
7000-1(config)# control-plane 7000-1(config-cp)# no service-policy input copp-system-policy 7000-1(config-cp)#ping 220.127.116.11 c 1000 PING 18.104.22.168 (22.214.171.124): 56 data bytes 64 bytes from 126.96.36.199: icmp_seq=0 ttl=254 time=1.16 ms 64 bytes from 188.8.131.52: icmp_seq=1 ttl=254 time=0.976 ms 64 bytes from 184.108.40.206: icmp_seq=2 ttl=254 time=0.841 ms <snip>
--- 220.127.116.11 ping statistics --- 1000 packets transmitted, 1000 packets received, 0.00% packet loss round-trip min/avg/max = 0.84/1.131/19.714 ms
As you can see wiwth the CoPP policy removed we are not longer dropping traffic. At this ping we can put back on the CoPP policy to further protect the CPU.
Also, if we want to leave on the CoPP policy and be able to ping wihtout loss we can adjust the policer to a higher value. This can be done in the following manner:
7000-1(config)# policy-map type control-plane copp-system-policy 7000-1(config-pmap)# class copp-system-class-monitoring 7000-1(config-pmap-c)#police cir 500 kbps bc 1000 ms conform transmit violate drop
This will increase the amount of ICMP allowed to the CPU to 500kbps per module. This can be adjusted to your need.
We can see this change with the following command:
F340.11.03-7000-1# sh run copp | b 'control-plane copp-system-policy' policy-map type control-plane copp-system-policy class copp-system-class-exception set cos 1 police cir 360 kbps bc 250 ms conform transmit violate drop class copp-system-class-critical set cos 7 police cir 39600 kbps bc 250 ms conform transmit violate drop class copp-system-class-important set cos 6 police cir 1060 kbps bc 1000 ms conform transmit violate drop class copp-system-class-management set cos 2 police cir 10000 kbps bc 250 ms conform transmit violate drop class copp-system-class-normal set cos 1 police cir 680 kbps bc 250 ms conform transmit violate drop class copp-system-class-redirect set cos 1 police cir 280 kbps bc 250 ms conform transmit violate drop class copp-system-class-monitoring set cos 1 police cir 500 kbps bc 1000 ms conform transmit violate drop <snip> control-plane service-policy input copp-system-policy