Buy or Renew
Buy or Renew
Cisco + Splunk: It’s a new day for your data. Learn more.
 
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
cancel
Start a conversation
18556
Views
15
Helpful
9
Replies

arp-inspection error & err-disable state

Mohammad Tbaili
Level 1
Level 1

‎09-23-2013 08:28 PM - edited ‎03-04-2019 09:07 PM

                   Hi ,

We have a serious issue in our environment with arp inspection feature , we are enabling this feature on all our sites switches and we are even increasing the limit rate to 100 but still the ports are going to error disable state !! Sometimes we are put the command (ip arp inspection trust) but still some ports are going to err-disable state . We are getting  some logs like :

%SW_DAI-4-PACKET_RATE_EXCEEDED: 128 packets received in 139 milliseconds on Fa2/9.

%PM-4-ERR_DISABLE: arp-inspection error detected on Fa2/9, putting Fa2/9 in err-disable state

So what is the causes of increasing this limit rate to 200 or 300 for example

Note : The issue is happening on different switches models and different IOS versions .

2 people had this problem
Labels:
0 Helpful
9 Replies 9

devils_advocate
Level 7
Level 7

‎09-24-2013 01:09 AM

128 ARP packets in 139 ms is a lot in my experience, we had our limit at 20 per second and that was ample.

Are these on end user switches or data centre switches?

Can you post the config of an example switch?

Personally, I would be SPANning a port which has the issue to see whats going on.

0 Helpful

Mohammad Tbaili
Level 1
Level 1
In response to devils_advocate

‎09-24-2013 08:52 PM

Hi devils ,

Actually these are normal end user switches & we are connecting IP phones , PCs and printers . Please find the configuration of one of our switches and the logs we are getting .

15367991-NALSGCSWI450301.log.zip

Tom Vanhout
Level 1
Level 1

‎09-30-2013 03:24 AM

We run a university campus, and we see a big increase in arp-traffic the last year.

A few years ago we implemented dynamic arp inspection on our student-networks.

The default rate-limit of 15pps was initially enough.

Last year we had a lot of cases where the rate-limit was triggered, and by default we set it now to 100.

This year the students are starting to come in again, and we notice quite a few connections where the limit of 100 is triggered,

even 300 often does not seem to be enough.

The "problem" seems to be client side by things like bonjour-service or network-discovery services that cause massive amounts of arp's.

The amount of devices that cause high rate's of arp-packets increases, as well as the amount they send.

So either we go for a lot of work for us to find out which all of these programs are, then a lot of work for our helpdesk to help students to disable them all or we increase the limit rate to 2000 or something, perhaps even unlimited.

If anyone else is experiencing this, i would be interested to know how you decided to handle this.

0 Helpful

Rolf Fischer
Level 9
Level 9
In response to Tom Vanhout

‎09-30-2013 06:20 AM

This is interesting. We saw similar increases after the rollout of new windows clients in our /22 subnets a couple of years ago.

We had to set the rate-limit value to 500 pps in order to achieve an acceptable amount of "false positives".

I couldn't spend much time on it, but it looked like DAI took send and also received ARP traffic into account. In any case I saw ports beeing err-disabled that didn't send not nearly as much as the port's rate-limit was configured to.

Meanwhile the security officer decided not longer to use DAI.

0 Helpful

jamie.grive
Level 1
Level 1

‎09-30-2013 03:38 AM

That does seem like an unusual amount for end user devices.

Are you able to run a sniffer on the devices to see which service is causing all the ARP requests? e.g. what it is ARPing for would be a clue.

Could potentially possibly be a malicious application.

0 Helpful

Kacejohns
Level 1
Level 1

‎09-22-2022 07:43 AM

Same issues here. Switchports continues to err-disable. The MAC a8a1.5991.9fd0 and have Infoblox provide DHCP IP 10.48.11.96

Should device be statically assigned the IP address 10.48.11.96 to resolve issue? 

.Sep 22 10:25:23.998: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.53/10:25:23 EDT Thu Sep 22 2022])
.Sep 22 10:25:23.998: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.47/10:25:23 EDT Thu Sep 22 2022])
.Sep 22 10:25:23.998: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.52/10:25:23 EDT Thu Sep 22 2022])
.Sep 22 10:25:23.998: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.31/10:25:23 EDT Thu Sep 22 2022])
.Sep 22 10:25:24.904: %CTS-3-PAC_PROVI_FAIL: PAC Provisioning failed for 10.72.20.10
.Sep 22 10:25:24.998: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.32/10:25:23 EDT Thu Sep 22 2022])
.Sep 22 10:25:24.998: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.34/10:25:23 EDT Thu Sep 22 2022])
.Sep 22 10:25:24.998: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.35/10:25:23 EDT Thu Sep 22 2022])
.Sep 22 10:25:24.998: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.36/10:25:23 EDT Thu Sep 22 2022])
.Sep 22 10:25:24.998: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.37/10:25:23 EDT Thu Sep 22 2022])
.Sep 22 10:25:26.008: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.46/10:25:23 EDT Thu Sep 22 2022])
.Sep 22 10:25:26.008: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.50/10:25:23 EDT Thu Sep 22 2022])
.Sep 22 10:25:26.008: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.48/10:25:23 EDT Thu Sep 22 2022])
.Sep 22 10:25:26.008: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.38/10:25:23 EDT Thu Sep 22 2022])
.Sep 22 10:25:26.008: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.39/10:25:23 EDT Thu Sep 22 2022])
.Sep 22 10:25:45.626: %CTS-3-PAC_PROVI_FAIL: PAC Provisioning failed for 10.74.20.10
.Sep 22 10:27:09.901: %CTS-3-PAC_PROVI_FAIL: PAC Provisioning failed for 10.72.20.10
.Sep 22 10:27:30.626: %CTS-3-PAC_PROVI_FAIL: PAC Provisioning failed for 10.74.20.10
.Sep 22 10:28:54.909: %CTS-3-PAC_PROVI_FAIL: PAC Provisioning failed for 10.72.20.10
.Sep 22 10:29:15.626: %CTS-3-PAC_PROVI_FAIL: PAC Provisioning failed for 10.74.20.10
.Sep 22 10:30:23.885: %PM-4-ERR_RECOVER: Attempting to recover from arp-inspection err-disable state on Gi1/24
.Sep 22 10:30:39.909: %CTS-3-PAC_PROVI_FAIL: PAC Provisioning failed for 10.72.20.10
.Sep 22 10:31:00.625: %CTS-3-PAC_PROVI_FAIL: PAC Provisioning failed for 10.74.20.10
.Sep 22 10:31:04.908: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Res) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/5057.a87d.2d7f/10.48.11.1/10:31:04 EDT Thu Sep 22 2022])
.Sep 22 10:31:05.387: %SEC_LOGIN-5-LOGIN_SUCCESS: Login Success [user: stkmj1] [Source: 10.71.80.64] [localport: 22] at 10:31:05 EDT Thu Sep 22 2022
.Sep 22 10:31:06.910: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.96/10:31:06 EDT Thu Sep 22 2022])
.Sep 22 10:31:06.910: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Res) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/5057.a87d.2d7f/10.48.11.1/10:31:06 EDT Thu Sep 22 2022])
.Sep 22 10:31:07.924: %SW_DAI-4-DHCP_SNOOPING_DENY: 2 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.1/10:31:07 EDT Thu Sep 22 2022])
.Sep 22 10:31:08.922: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.1/10:31:08 EDT Thu Sep 22 2022])
.Sep 22 10:31:09.922: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.1/10:31:09 EDT Thu Sep 22 2022])
.Sep 22 10:31:09.922: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Res) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/5057.a87d.2d7f/10.48.11.1/10:31:09 EDT Thu Sep 22 2022])
.Sep 22 10:31:10.922: %SW_DAI-4-DHCP_SNOOPING_DENY: 2 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.53/10:31:10 EDT Thu Sep 22 2022])
.Sep 22 10:31:10.922: %SW_DAI-4-DHCP_SNOOPING_DENY: 2 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.47/10:31:10 EDT Thu Sep 22 2022])
.Sep 22 10:31:11.928: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Res) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/5057.a87d.2d7f/10.48.11.1/10:31:11 EDT Thu Sep 22 2022])
.Sep 22 10:31:11.928: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Res) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/accc.8e7b.79e9/10.48.11.34/10:31:11 EDT Thu Sep 22 2022])
.Sep 22 10:31:11.928: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.1/10:31:11 EDT Thu Sep 22 2022])
.Sep 22 10:31:11.928: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.53/10:31:11 EDT Thu Sep 22 2022])
.Sep 22 10:31:11.928: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.47/10:31:11 EDT Thu Sep 22 2022])
.Sep 22 10:31:11.958: %SW_DAI-4-PACKET_RATE_EXCEEDED: 20 packets received in 322 milliseconds on Gi1/24.
.Sep 22 10:31:11.958: %PM-4-ERR_DISABLE: arp-inspection error detected on Gi1/24, putting Gi1/24 in err-disable state
.Sep 22 10:31:12.940: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.52/10:31:11 EDT Thu Sep 22 2022])
.Sep 22 10:31:12.941: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.31/10:31:11 EDT Thu Sep 22 2022])
.Sep 22 10:31:12.941: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.32/10:31:11 EDT Thu Sep 22 2022])
.Sep 22 10:31:12.941: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.33/10:31:11 EDT Thu Sep 22 2022])
.Sep 22 10:31:12.941: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.35/10:31:11 EDT Thu Sep 22 2022])
.Sep 22 10:31:13.943: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.36/10:31:11 EDT Thu Sep 22 2022])
.Sep 22 10:31:13.943: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.37/10:31:11 EDT Thu Sep 22 2022])
.Sep 22 10:31:13.944: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.46/10:31:11 EDT Thu Sep 22 2022])
.Sep 22 10:31:13.944: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.50/10:31:11 EDT Thu Sep 22 2022])
.Sep 22 10:31:13.944: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.48/10:31:11 EDT Thu Sep 22 2022])
.Sep 22 10:31:14.944: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.38/10:31:11 EDT Thu Sep 22 2022])
.Sep 22 10:31:14.944: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.39/10:31:11 EDT Thu Sep 22 2022])
.Sep 22 10:31:14.944: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Req) on Gi1/24, vlan 40.([a8a1.5991.9fd0/10.48.11.96/0000.0000.0000/10.48.11.60/10:31:11 EDT Thu Sep 22 2022])
PC-OPS-CORE-SW# 
PC-OPS-CORE-SW#
PC-OPS-CORE-SW#
PC-OPS-CORE-SW#
PC-OPS-CORE-SW#
PC-OPS-CORE-SW#show run int gig1/24 
Building configuration...

Current configuration : 720 bytes
!
interface GigabitEthernet1/24
description NVR-ASRock_9fd0
switchport access vlan 40
switchport mode access
authentication event fail action next-method
authentication event server dead action reinitialize vlan 40
authentication event server dead action authorize voice
authentication event server alive action reinitialize 
authentication host-mode multi-auth
authentication open
authentication order dot1x mab
authentication priority dot1x mab
authentication port-control auto
authentication periodic
authentication timer reauthenticate server
authentication violation restrict
mab
dot1x pae authenticator
dot1x timeout tx-period 10
spanning-tree portfast edge
spanning-tree bpduguard enable
end

PC-OPS-CORE-SW 

 

MHM Cisco World
VIP
VIP

‎09-22-2022 08:40 AM

ARP can send by IP device tracking, which will send via trunk and hence you get err-disable, are you run IPDT??

0 Helpful

Hyperion0000
Level 1
Level 1
In response to MHM Cisco World

‎09-29-2022 12:22 PM - edited ‎09-29-2022 12:50 PM

MHM - what would you recommend in regards to IPDT? I'm running into a similar issue.  I'm running DHCP snooping + DAI and I'm having random Cisco phones go err-disabled (running default rates).

%SW_DAI-4-PACKET_RATE_EXCEEDED: 18 packets received in 546 milliseconds on Gi1/0/3

So far I'm looking an increasing the Rate to 30 pps + adding a burst of 15: "ip arp inspection limit rate 30 burst interval 15".  Also - I'm adding err-disable recovery for ARP "errdisable recovery cause arp-inspection".  I'm just curious if something with IPDT could help

MHM Cisco World
VIP
VIP
In response to Hyperion0000

‎10-01-2022 05:46 AM - edited ‎10-01-2022 06:18 AM

https://bst.cisco.com/bugsearch/bug/CSCvd51480
please check this bug and how cisco deal with trunk in case of IPTD and DAI is enable. 
hope this help you

0 Helpful
Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Customers Also Viewed These Support Documents
Review Cisco Networking products for a $25 gift card