EIGRP neighbors flapping through GRE tunnel

Answered Question
Jan 23rd, 2008
User Badges:

I have 7 remote locations connect to our main site through IPSec VPN. I also have GRE tunnel established between remote locations and our main site. EIGRP neighbor is established through GRE tunnel. Lately, EIGRP neighbors have been flapping more frequent. Internet service did not go down at the remote sites when EIGRP flapped. Any thoughts of what might be causing this problem?


Jan 22 13:59:30.489: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 200: Neighbor 192.168.62.81 (Tunnel7) is down: holding time expired

Jan 22 13:59:31.081: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 200: Neighbor 192.168.62.73 (Tunnel6) is down: holding time expired

Jan 22 13:59:31.369: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 200: Neighbor 192.168.62.57 (Tunnel2) is down: holding time expired

Jan 22 13:59:32.565: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 200: Neighbor 192.168.62.65 (Tunnel4) is down: Interface Goodbye received

Jan 22 13:59:33.721: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 200: Neighbor 192.168.62.25 (Tunnel1) is down: holding time expired

Jan 22 13:59:33.986: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 200: Neighbor 192.168.62.85 (Tunnel8) is down: Interface Goodbye received

Jan 22 13:59:34.142: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 200: Neighbor 192.168.62.45 (Tunnel3) is down: holding time


Thank you.

Correct Answer by Richard Burts about 9 years 6 months ago

Santi


This has been an interesting discussion and I am glad that my responses have been helpful.


The forum is an excellent place to learn about Cisco networking. I encourage you to continue your participation in the forum.


HTH


Rick

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 4.5 (4 ratings)
Loading.
mounir.mohamed Thu, 01/24/2008 - 01:17
User Badges:
  • Gold, 750 points or more

As my friend mentioned this is one of the reasons behind such flaps, but for clarity please share your configurations.


Best Regards,

Mounir Mohamed



smitty6504 Thu, 01/24/2008 - 05:30
User Badges:

Have you tried adjusting your hello timers? You might want to debug EIGRP on both sides and post them.

Richard Burts Thu, 01/24/2008 - 05:38
User Badges:
  • Super Silver, 17500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

The recursive routing theory is a good suggestion given the lack of detail in the original post, but almost certainly not the explanation. I am running a bunch of IPSec/GRE tunnels with EIGRP as the routing protocol and I see the same type of neighbor flapping - and I know in my case that it is not a recursive routing problem.


We did find that adjusting the EIGRP timers (make them longer) reduces the incidence but does not solve the problem.


I have not gotten a much better theory of the problem beyond the possibility that packet loss may be dropping enough EIGRP hellos to provoke the timeouts. I would be very curious to know if anyone else has experienced this and has found an explanation for it.


HTH


Rick

santipongv Thu, 01/24/2008 - 09:19
User Badges:

I am not quite conviced that recursive routing is the culprit since I am not getting "%TUN-5-RECURDOWN" error messages. I will provide the config shortly.

santipongv Thu, 01/24/2008 - 10:23
User Badges:

R1-VPN#

!

interface Tunnel2

description IPSEC Tunnel endpoint to Chan-PRI-Tunnel2

bandwidth 768

ip address 192.168.62.57 255.255.255.252

ip authentication mode eigrp 200 md5

ip authentication key-chain eigrp 200 secure-chain

ip tcp adjust-mss 1300

keepalive 10 3

cdp enable

tunnel source Loopback2

tunnel destination 192.168.62.33

crypto map Shaftervpn

end



ip route 0.0.0.0 0.0.0.0 192.168.62.58




Hub#

!

interface Tunnel2

description R1-VPN-Tunnel

bandwidth 1544

ip address 192.168.62.58 255.255.255.252

ip authentication mode eigrp 200 md5

ip authentication key-chain eigrp 200 secure-chain

ip tcp adjust-mss 1300

keepalive 10 3

cdp enable

tunnel source Loopback2

tunnel destination 192.168.62.32



ip route 192.168.162.254 255.255.255.255 192.168.62.57

Richard Burts Thu, 01/24/2008 - 10:44
User Badges:
  • Super Silver, 17500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

Santipong


Thank you for posting the tunnel configuration. There are a couple of things that may be worth commenting about (though I am not sure that they are related to the issue of EIGRP neighbor flapping):

- I am surprised at the static default route in R1-VPN (ip route 0.0.0.0 0.0.0.0 192.168.62.58) which uses the IP address of the other end of the tunnel as the next hop. Since the tunnel destination is tunnel destination 192.168.62.33 it would appear that the tunnel destination is reached through the tunnel. This is the classic definition of the recursive routing problem with GRE tunnels. Is there perhaps another static route for the tunnel end point that you did not post?

- I am surprised to see the crypto map assigned to the tunnel interface rather than the physical interface. What version of code is this router running? In older versions of code it was necessary to configure the crypto map on both the physical and the tunnel interface. But that changed and it now needs to be only on the physical interface. Perhaps you are running a version that needs both. But if not, then I have seen some strange behaviors that seems to relate to having the map on the tunnel when it did not need to be.

- I am a bit puzzled by the static route on the Hub router. It obviously is some host address reached via the GRE tunnel. But what is it? And are there other static routes defined on this router?


HTH


Rick

santipongv Thu, 01/24/2008 - 11:31
User Badges:

Rick,


1.

ip route 0.0.0.0 0.0.0.0 192.168.62.58

ip route 0.0.0.0 0.0.0.0 192.168.182.1 240 <== ISDN Backup

ip route 192.168.62.33 255.255.255.255 10.255.255.5 <== Firewall Interface (Physical path)

2.

12.4(13d)

3.

It is the Loopback0 IP address of the R1-VPN

Richard Burts Thu, 01/24/2008 - 18:40
User Badges:
  • Super Silver, 17500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

Santipong


1) Having the primary static default route point to the next hop over the tunnel seems to invite the recursive routing problem since any unknown destination will be routed through the tunnel - I worry that the router will attempt to get to the tunnel destination through the tunnel.


Having a floating static through ISDN backup does not make it much better.


2) 12.4(13d) is certainly recent. Is there any reason to have the crypto map on the tunnel?


Is the crypto map also on the physical interface (which you have not included in what you posted)?


I highly recommend that you remove the crypto map from the tunnel interface.


3) Did you really mean loopback2 instead of loopback0? If so pointing the tunnel destination as being reachable via the next hop of the remote tunnel interface is another invitation to the recursive routing problem.


If not what is loopback0 and how does it relate to this question?


HTH


Rick

santipongv Fri, 01/25/2008 - 05:37
User Badges:

Rick,


I have updated configurations on both ends as shown below. Please advise.


R1-VPN#

!

interface Tunnel2

description IPSEC Tunnel endpoint to Hub-Tunnel2

bandwidth 768

ip address 192.168.62.57 255.255.255.252

ip authentication mode eigrp 200 md5

ip authentication key-chain eigrp 200 secure-chain

ip tcp adjust-mss 1300

keepalive 10 3

cdp enable

tunnel source Loopback2

tunnel destination 192.168.62.33

crypto map Shaftervpn <== I will remove this statement


interface Loopback2

description Loopback for VPN IPSEC Tunnel termination from Hub

ip address 192.168.62.32 255.255.255.255

no ip redirects

no ip unreachables

no ip proxy-arp

ip route-cache flow


interface Loopback0

ip address 192.168.162.254 255.255.255.252

no ip redirects

no ip proxy-arp


ip route 0.0.0.0 0.0.0.0 192.168.62.58

ip route 0.0.0.0 0.0.0.0 192.168.182.1 240 <== ISDN Backup

ip route 192.168.62.33 255.255.255.255 10.255.255.5 <== Firewall Interface (Physical path)


Hub#

!

interface Tunnel2

description R1-VPN-Tunnel

bandwidth 1544

ip address 192.168.62.58 255.255.255.252

ip authentication mode eigrp 200 md5

ip authentication key-chain eigrp 200 secure-chain

ip tcp adjust-mss 1300

keepalive 10 3

cdp enable

tunnel source Loopback2

tunnel destination 192.168.62.32


!

interface Loopback2

description R1-VPN

ip address 192.168.62.33 255.255.255.255

no ip redirects

no ip unreachables

no ip proxy-arp


ip route 192.168.162.254 255.255.255.255 192.168.62.57


Regards,

Santi

Richard Burts Fri, 01/25/2008 - 08:25
User Badges:
  • Super Silver, 17500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

Santi


I believe that this is better. I still would like to see the crypto map statement removed from the tunnel on the R1-VPN. And the Hub static route that you show is to loopback 0 but the tunnel is using loopback 2. There should be a route to loopback 2 that does not involve traffic going through the tunnel. Is there other routing information for loopback 2 on R1-VPN that we are not seeing?


HTH


Rick

santipongv Fri, 01/25/2008 - 08:34
User Badges:

Rick,


I will remote crypto map statement from the tunnel interface on the R1-VPN. As for Hub, route to loopback 2 is learned via eigrp. There is no other static route statement. With these configuration statements, do you still believe that I am experiencing "recusive routing"?


Regards,

Santi

Richard Burts Fri, 01/25/2008 - 09:29
User Badges:
  • Super Silver, 17500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

Santi


I am slightly concerned that you are learning the loopback 2 address (the tunnel destination) via EIGRP since this is one of the steps to the recursion problem. And I believe that it would be somewhat better if there were a static route on the Hub for the loopback 2 on R1-VPN. But I do not believe that you are experiencing recursion on the tunnel interface. If you were experiencing that there would be error messages about it in the log. And the tunnel interface would go down and then come back up instead of just having the EIGRP neighbor relationship down and up.


As I said in my first post in this thread, I believe that the symptoms that you describe are not due to recursion on the tunnel. I believe that they are due to something else. But I am not clear what the something else is. Having cleaned up some parts of your configuration, I would be curious to know if your symptoms change any?


HTH


Rick

santipongv Fri, 01/25/2008 - 09:33
User Badges:

Rick,


We have similar set up at other locations, I removed crypto map statement from Tunnel interface. I am still experiencing similar problem but less frequent.


Regards,

Santi

santipongv Fri, 01/25/2008 - 09:40
User Badges:

Rick,


Would you like me to try adding the following statement on Hub router?

ip route 192.168.62.32 255.255.255.255 192.168.62.57


192.168.62.32 - Loopback2 R1-VPN

192.168.62.57 - Tunnel2 R1-VPN


Regards,

Santi

Richard Burts Fri, 01/25/2008 - 09:49
User Badges:
  • Super Silver, 17500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

Santi


NO. I would not like to have a static route which indicates that the tunnel destination is reached by going through the tunnel. If you do a static route it should indicate the next hop as the physical interface next hop that it will take on its way out to the other router.


HTH


Rick

santipongv Fri, 01/25/2008 - 09:52
User Badges:

Rick,


You mean a static route to Loopback2 interface of R1-VPN via a physical next hop IP address, correct? I will try and will observe the result.


Regards,

Santi

Richard Burts Fri, 01/25/2008 - 10:28
User Badges:
  • Super Silver, 17500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

Santi


Yes that is what I mean. If you do a static route on the Hub, then it should be for the loopback 2 (tunnel destination as specified on Hub) and should have its next hop as some physical interface from Hub.


HTH


Rick

santipongv Fri, 01/25/2008 - 10:30
User Badges:

Rick,


I will let you know the result. Thank you.


Regards,

Santi

santipongv Mon, 01/28/2008 - 04:54
User Badges:

Rick,


I looked at the configurations again. There is in fact a static route statement for Loopback 2 (Tunnel destinatin as specified on Hub) on the IPSed Hub router sitting in front of this GRE Hub router.


Internet router <=> Firewall <=> IPSec Hub router <=> Firewall <=> GRE Hub router <=> Internal network


Will this make a difference?


Regards,

Santi

Richard Burts Mon, 01/28/2008 - 05:51
User Badges:
  • Super Silver, 17500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

Santi


In terms of what we have been talking about I do not think that a static route on another router would make a difference. The fundamental question is about the router that terminates the GRE tunnel and what route it would use to reach the tunnel destination. If the router is learning the tunnel destination address via EIGRP and if it selects the EIGRP route through the tunnel as the best way to reach the tunnel destination then it leads to the recursion problem. The router needs some route that it will prefer to the EIGRP route. A static route on the next router does not help this issue.


HTH


Rick

santipongv Mon, 01/28/2008 - 05:57
User Badges:

Rick,


Why is it that some of the eigrp neighbors through GRE tunnels are more stable than others (If there are no static routes to Loopback for those GRE tunnel destinations from the GRE Hub router)?


Regards,

Santi

santipongv Tue, 01/29/2008 - 04:43
User Badges:

Rick,


I added the suggested static route on the GRE Hub router, EIGRP appears to be more stable. Thank you for your advice.


Regards,

Santi

Correct Answer
Richard Burts Tue, 01/29/2008 - 05:13
User Badges:
  • Super Silver, 17500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

Santi


This has been an interesting discussion and I am glad that my responses have been helpful.


The forum is an excellent place to learn about Cisco networking. I encourage you to continue your participation in the forum.


HTH


Rick

santipongv Tue, 01/29/2008 - 05:23
User Badges:

Rick,


Yes, it has indeed. I couldn't agree with you more aobut the discussion forum. I learn a lot from this forum. I will continue to participate in the forum. Thank you!


Regards,

Santi

FrankCisco049 Thu, 06/02/2016 - 08:52
User Badges:

I resolve this issue using the command neighbor in bought locations.


I dont use IP sec, but Im sure that is no a security problem. 


EIGRP by default uses multicast for neighbor discovery but it also allows you to configure EIGRP neighbors statically. Once you do this, EIGRP will only use unicast and disables EIGRP multicast on the selected interface.

This is my config.

Current configuration : 295 bytes
!
interface Tunnel1
bandwidth 100000
ip address 172.16.1.1 255.255.255.0
ip mtu 1400
ip hold-time eigrp 500 35
no ip next-hop-self eigrp 500
ip tcp adjust-mss 1360
no ip split-horizon eigrp 500
tunnel source FastEthernet0/0
tunnel destination 172.16.4.2
tunnel path-mtu-discovery
end

R3#show run | sec router eig
R3#show run | sec router eigrp
router eigrp 500
network 172.16.1.0 0.0.0.255
network 192.168.1.0
no auto-summary
neighbor 172.16.1.2 Tunnel1
R3#

Building configuration...

Current configuration : 295 bytes
!
interface Tunnel1
bandwidth 100000
ip address 172.16.1.2 255.255.255.0
ip mtu 1400
ip hold-time eigrp 500 35
no ip next-hop-self eigrp 500
ip tcp adjust-mss 1360
no ip split-horizon eigrp 500
tunnel source FastEthernet0/0
tunnel destination 172.16.3.2
tunnel path-mtu-discovery
end

R4#show run | sec router eigrp
router eigrp 500
network 172.16.1.0 0.0.0.255
network 192.168.2.0
no auto-summary
neighbor 172.16.1.1 Tunnel1
R4#

Attachment: 

Actions

This Discussion