Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 

Packet Loss When Pinging From/To A Nexus 7000

 

 

 

Introduction

 

Cisco Nexus 7000 Series Switches offer one of the most comprehensive data center network feature sets in a single platform. As ping is a common tool used to test connectivity in networks, it  is not uncommon for users to try to ping a Nexus 7000 as a test.  However, when pinging a Nexus 7000, it is very common to see packet loss  due to the default behavior in which the Nexus 7000 uses Control Plane  Policing (CoPP) to rate-limit certain types of traffic to the CPU.

 

The main asset of the Nexus 7000 (as well as other Cisco switches) is  the ability to switch packets in hardware using high speed ASICs. The  CPU however, is not designed for extensive packet forwarding and is  typically reserved for the processing and creation of control plane  packets such as routing protocol hellos, BPDUs, LACP, CDP, etc. Loss of  these control plane packets can create network instability, so it is  very important to protect the CPU from traffic that could potentially  cause high utilization. This is the main motivation behind CoPP on this  platform.

 

NOTE - CoPP only applies to traffic destined to the switch  itself (control plane).  It has no impact on hardware switched traffic  through the box.

 

CoPP

CoPP is a hardware-based feature that protects the Supervisor from DoS attacks. It controls the rate at which packets are allowed to reach the Supervisor. The CoPP feature is modeled like an input QoS policy attached to the special interface called the control-plane. However, CoPP is a security feature and not part of QoS. In order to protect the Supervisor, the CoPP separates data plane packets from the control plane packets. After a packet is classified, the packet can also be marked and used to assign different priorities based on the type of packets. Conform, exceed, and violate actions (transmit, drop, mark-down) can be set. If no policer is attached to a class, then a default policer is added whose conform action is drop. Glean packets are policed with default-class. One rate, two color, and two rate, three color policing are supported.

 

Example

 

Here is an example of the type of packet loss one can expect when  trying to ping a Nexus 7000. The ip address 10.48.1.50 is an ip address  of an SVI configured on a Nexus 7000 :

 

StaticVSS#ping 10.48.1.50 repeat 1000

 

Type escape sequence to abort.

Sending 1000, 100-byte ICMP Echos to 10.48.1.50, timeout is 2 seconds:

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!

Success rate is 99 percent (994/1000), round-trip min/avg/max = 1/1/4 ms

StaticVSS#

 

A quick Ethanalyrzer on the N7K shows these ICMP requests/replies  at the CPU level of the N7K, therefore making it prone to rate-limiting by CoPP.

 

Nexus_7000# ethanalyzer local interface inband capture-filter "host 10.48.1.50 and icmp" limit-captured-frames 100 | no-more

 

<snippet>

 

Capturing on inband

 

2012-07-04 16:28:45.723631   10.48.1.49 -> 10.48.1.50   ICMP Echo (ping) request

2012-07-04 16:28:45.723649   10.48.1.50 -> 10.48.1.49   ICMP Echo (ping) reply

2012-07-04 16:28:45.724063   10.48.1.49 -> 10.48.1.50   ICMP Echo (ping) request

2012-07-04 16:28:45.724200   10.48.1.50 -> 10.48.1.49   ICMP Echo (ping) reply

2012-07-04 16:28:45.724562   10.48.1.49 -> 10.48.1.50   ICMP Echo (ping) request

2012-07-04 16:28:45.724694   10.48.1.50 -> 10.48.1.49   ICMP Echo (ping) reply

2012-07-04 16:28:45.725062   10.48.1.49 -> 10.48.1.50   ICMP Echo (ping) request

2012-07-04 16:28:45.725192   10.48.1.50 -> 10.48.1.49   ICMP Echo (ping) reply

2012-07-04 16:28:45.725561   10.48.1.49 -> 10.48.1.50   ICMP Echo (ping) request

2012-07-04 16:28:45.725692   10.48.1.50 -> 10.48.1.49   ICMP Echo (ping) reply

 

First we check to make sure  there is a CoPP policy applied with the command "Show copp status". To view the policy, you can use  “show policy-map interface control-plane”.

 

Nexus_7000# show copp status
Last Config Operation: None
Last Config Operation Timestamp: None
Last Config Operation Status: None
Policy-map attached to the control-plane: copp-system-p-policy-strict

 

Here is an example of the global config showing a policy appled under the control-plane:

 

control-plane

 

service-policy input copp-system-policy

 

Looking further at the N7K, we can see that there is a class for  ICMP that matches an ACL with ACEs referencing echo request/replies..

 

Nexus_7000# show access-lists copp-system-acl-icmp

 

IP access list copp-system-acl-icmp

        10 permit icmp any any echo

        20 permit icmp any any echo-reply

 

 

Nexus_7000# show class-map type control-plane copp-system-class-monitoring

 

    class-map type control-plane match-any copp-system-class-monitoring

      match access-group name copp-system-acl-icmp

      match access-group name copp-system-acl-icmp6

      match access-group name copp-system-acl-traceroute

 

Here we confrim what we suspected above --- ICMP is being  rate-limited and dropped at the CPU. By default, ICMP traffic will be  policed with a cir of 130 kbp per slot.  NOTE - In the below example  module 3 is the module where the traffic is ingressing/egressing

 

Nexus_7000# show policy-map interface control-plane class copp-system-class-monitoring

 

Control Plane

 

  service-policy  input: copp-system-policy

    class-map copp-system-class-monitoring (match-any)

      match access-group name copp-system-acl-icmp

      match access-group name copp-system-acl-icmp6

      match access-group name copp-system-acl-traceroute

      set cos 1

      police cir 130 kbps , bc 1000 ms

 

      module 1 :

        conformed 0 bytes; action: transmit

        violated 0 bytes; action: drop

 

      module 3 :

        conformed 543980 bytes; action: transmit

        violated 3540 bytes; action: drop

 

      module 4 :

        conformed 0 bytes; action: transmit

        violated 0 bytes; action: drop

 


FAQ

 

Q – How did the CoPP configuration get added? I don’t recall ever explicitly adding it.

 

A- CoPP is typically added during the initial setup configuration  of a vDC, so this is likely when the configuration was generated.

 

 

http://www.cisco.com/en/US/docs/switches/datacenter/sw/4_1/nx-os/security/configuration/guide/sec_cp...

 

 

Q – I am considering removing CoPP. Is there any downside in doing so?

 

A – Cisco recommends keeping CoPP in place in order to provide a  level of protection at the control-plane level. However, it may be  necessary to tweak the CoPP config in some cases depending on your  specific traffic requirements.

 

Q –  I have other Cisco switches in my network. Why is CoPP not enabled on those platforms?

 

A - Although other Cisco switches like the 6500 and 4500 support  CoPP, only NX-OS based switches have it enabled by default. More  information about CoPP on a particular platform can be found in the  supporting configuration guides.

 

 

Related Information

 

Cisco NX-OS and Virtual Device Contexts (VDCs)
VRF Configuration and Verification on Nexus 7000

Version history
Revision #:
2 of 2
Last update:
‎08-27-2017 10:40 PM
Updated by:
 
Labels (1)
Contributors