Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements

Welcome to Cisco Support Community. We would love to have your feedback.

For an introduction to the new site, click here. And see here for current known issues.

New Member

ARP table not populating mac address for previously reachable IP address

Router has been online and working fine with one BGP neighbor for almost 2 years and no downtime.  2 weeks ago, added a 2nd BGP peer.  Everything worked fine for 2 weeks, then all of a sudden yesterday the 2nd BGP peer is disconnected and does not come back.  ISP checks and sees everything looks fine on their end.  We cannot even ping each other now.

 

Upon investigation, the ARP table is not even populating the MAC address for the BGP peer IP anymore (same local subnet).  Stays "incomplete" in the table no matter what we do, including clearing arp table, changing IP address, etc.

 

Plug a laptop directly into the 2nd BGP peer FE port and replicate the IP addressing.  Laptop cannot ping Router, but Router CAN ping laptop.  Check ARP table, but STILL no mac address assigned and now not even the ARP table showing "incomplete".

 

Thinking it could be the FE interface, switch to the 2nd FE interface and perform same laptop test, this time with arbitrary IP addressing.  Now cannot ping each other, no MAC in ARP table.

 

End up rebooting the router and lo-and-behold, everything is working normally again.  2nd BGP peer peers up instantly.

 

I should also mention that the 1st BGP peer worked flawlessly throughout, taking all the Internet load and having no issues throughout.


Also, the FE ports for the 2nd BGP peer are on an HWIC FE card plugged into the router.  The 1st BGP peer is plugged into the built-in GE interface.  2901 running: c2900-universalk9-mz.SPA.151-4.M4.bin

 

Lastly, no router resource issues, no error messages, no logs.  Just the BGP peer disconnecting.

I have never, in 20 years working with Cisco routers seen something like this before.  This is the most fundamental aspect of IP and Ethernet that was not working.

 

Has anyone ever seen this behavior before??

 

Here is the router config (IP's changed):

 


version 15.1
service timestamps debug datetime msec localtime show-timezone
service timestamps log datetime msec localtime show-timezone
service password-encryption
service internal
service sequence-numbers
!
boot-start-marker
boot-end-marker
!
!
logging buffered 150000
!
aaa new-model
!
!
aaa authentication login LAUTHEN local
aaa authentication login TAUTHEN local group tacacs+ enable
aaa authorization console
aaa authorization exec LAUTHOR local if-authenticated
aaa authorization exec TAUTHOR local group tacacs+ if-authenticated
!
!
!
!
!
aaa session-id common
clock timezone PST -8 0
clock summer-time PDT recurring
!
no ipv6 cef
no ip source-route
ip cef
!
!
!
!
!
no ip domain lookup
multilink bundle-name authenticated
!
!

!


!
!
username ubiadmin privilege 15 secret 4 .JbeuWXuZvchrG0OL.5BftFtqrrEyxcnVHn5rIuCnTk
username umitsnoc01 privilege 15 secret 4 cUmoRUjey9O1x.wk9S.kleX.iAAhCwihupr6Z98p6OA
!
redundancy
!
!
ip ssh version 2
!
track 1 interface GigabitEthernet0/0 line-protocol
!
class-map match-any AutoQoS-VoIP-RTP-Trust
 match access-group name SIP-Media-INBOUND
class-map match-any AutoQoS-VoIP-Control-Trust
 match ip dscp cs3
 match ip dscp af31
class-map match-any Customer-Voice
 match access-group name Customer-VPNs
class-map match-any media
 match access-group name SIP-Media
class-map match-any signaling
 match access-group name SIP-Signaling
!
!
policy-map AutoQoS-Policy-Trust
 class AutoQoS-VoIP-RTP-Trust
  priority percent 70
 class AutoQoS-VoIP-Control-Trust
  bandwidth percent 5
 class class-default
  fair-queue
policy-map queue
 class signaling
  bandwidth percent 5
 class media
  priority percent 50
 class Customer-Voice
  priority percent 40
 class class-default
  fair-queue
policy-map shape
 class class-default
  shape average 10000000
  service-policy queue
!
!
!
!
!
interface Embedded-Service-Engine0/0
 no ip address
 shutdown
!
interface GigabitEthernet0/0
 description BGP Peer 1
 ip address 2.2.2.2 255.255.255.252
 no ip redirects
 ip flow ingress
 ip flow egress
 duplex auto
 speed auto
 service-policy output shape
!
interface GigabitEthernet0/1
 description LAN
 ip address 1.2.3.4 255.255.255.0
 no ip redirects
 ip flow ingress
 ip flow egress
 standby 255 ip 1.2.3.1
 standby 255 priority 105
 standby 255 preempt
 standby 255 mac-address 1a2b.3c4d.5e6f
 standby 255 track 1 decrement 10
 duplex auto
 speed auto
 service-policy output AutoQoS-Policy-Trust
!
interface FastEthernet0/0/0
 description BGP Peer 2
 ip address 1.1.1.1 255.255.255.252
 ip flow ingress
 ip flow egress
 duplex full
 speed 100
 service-policy output shape
!
interface FastEthernet0/0/1
 no ip address
 shutdown
 duplex auto
 speed auto
!
!
router bgp 7777
 bgp router-id 2.2.2.2
 bgp log-neighbor-changes
 network 1.2.3.0 mask 255.255.255.0
 neighbor 1.1.1.2 remote-as 5555
 neighbor 1.1.1.2 update-source FastEthernet0/0/0
 neighbor 1.1.1.2 prefix-list L3-DEFGW in
 neighbor 1.1.1.2 route-map L3-LPREF-IN in
 neighbor 2.2.2.1 remote-as 6666
 neighbor 2.2.2.1 ebgp-multihop 2
 neighbor 2.2.2.1 update-source GigabitEthernet0/0
 neighbor 2.2.2.1 send-community
 neighbor 2.2.2.1 prefix-list COLO-DEFGW in
 neighbor 2.2.2.1 route-map COLO-LPREF-IN in
 neighbor 2.2.2.1 route-map COLO-OUT out
!
ip forward-protocol nd
!
ip bgp-community new-format
ip as-path access-list 5 permit _5555_
ip as-path access-list 5 deny .*
ip as-path access-list 10 permit ^6666$
no ip http server
no ip http secure-server
ip flow-top-talkers
 top 50
 sort-by bytes
!
ip route 0.0.0.0 0.0.0.0 1.1.1.2 254 name L3
ip route 0.0.0.0 0.0.0.0 2.2.2.1 255 name COLO1
ip route 10.0.0.0 255.0.0.0 10.10.10.10 name FW_OUTSIDE
ip tacacs source-interface GigabitEthernet0/1
!
ip access-list standard SNMP_SOURCES
 permit 12.12.12.0 0.0.0.255
 deny   any log

!       

!
!
ip prefix-list L3-DEFGW seq 5 permit 0.0.0.0/0
!
ip prefix-list COLO-DEFGW seq 5 permit 0.0.0.0/0
!
ip prefix-list COLO-LPREF-OUT seq 5 permit 1.2.3.0/24

!
!
!
!
route-map COLO-LPREF-IN permit 5
 match as-path 5
 set local-preference 250
!
route-map COLO-LPREF-IN permit 10
 set local-preference 150
!
route-map COLO-LPREF-IN permit 20
!
route-map COLO-OUT permit 10
 match ip address prefix-list COLO-LPREF-OUT
 set as-path prepend 7777 7777 7777
 set community 29795:1004
!
route-map COLO-OUT permit 20
!
route-map L3-LPREF-IN permit 10
 match as-path 10
 set local-preference 200
!
route-map L3-LPREF-IN permit 20
 set local-preference 150
!
!
snmp-server enable traps snmp authentication linkdown linkup coldstart warmstart
snmp-server enable traps vrrp
snmp-server enable traps flowmon
snmp-server enable traps transceiver all
snmp-server enable traps ds1
snmp-server enable traps call-home message-send-fail server-fail
snmp-server enable traps tty
snmp-server enable traps license
snmp-server enable traps envmon
snmp-server enable traps ethernet cfm cc mep-up mep-down cross-connect loop config
snmp-server enable traps ethernet cfm crosscheck mep-missing mep-unknown service-up
snmp-server enable traps flash insertion removal
snmp-server enable traps mac-notification
snmp-server enable traps aaa_server
snmp-server enable traps cef resource-failure peer-state-change peer-fib-state-change inconsistency
snmp-server enable traps memory bufferpeak
snmp-server enable traps config-copy
snmp-server enable traps config
snmp-server enable traps config-ctid
snmp-server enable traps event-manager
snmp-server enable traps hsrp
snmp-server enable traps cpu threshold
snmp-server enable traps rsvp
snmp-server enable traps syslog
snmp-server enable traps vtp
snmp-server enable traps ipsla

!

1 REPLY
Hall of Fame Super Silver

When you were checking the

When you were checking the ARP table was there an entry for Fast0/0/0?

 

HTH

 

Rick

479
Views
0
Helpful
1
Replies
CreatePlease login to create content