I have am using DMVPN which is working perfectly for domestic sites but as soon as I start using routers overseas I am getting the following error:
%ADJ-5-PARENT: Midchain parent maintenance for IP midchain out of Tunnel1, addr 10.48.12.2 854E1580 - looped chain attempting to stack
These routers are the exact same routers that we use in the United States and we are using the same config. I have no idea what's going on here.
Right now I have only one tunnel connecting to the routers and it's still complaining about the loop.
I've attached the config..
crypto isakmp policy 1
encr aes 256
crypto isakmp key <removed> address 0.0.0.0 0.0.0.0 no-xauth
crypto ipsec transform-set AES esp-aes 256 esp-md5-hmac
crypto ipsec profile AAA_VPN_2.0
set transform-set AES
ip address 10.100.1.52 255.255.255.255
ip address 10.48.12.22 255.255.254.0
no ip redirects
ip mtu 1200
ip nhrp authentication VPN20_T1
ip nhrp map 10.48.12.2 184.108.40.206
ip nhrp map 10.48.12.1 220.127.116.11
ip nhrp map multicast 18.104.22.168
ip nhrp map multicast 22.214.171.124
ip nhrp network-id 1
ip nhrp nhs 10.48.12.2
ip nhrp nhs 10.48.12.1
ip nhrp registration no-unique
tunnel source FastEthernet4
tunnel mode gre multipoint
tunnel protection ipsec profile AAA_VPN_2.0
ip address dhcp
ip nat outside
no ip address
speed basic-1.0 basic-2.0 basic-5.5 6.0 9.0 basic-11.0 12.0 18.0 24.0 36.0 48.0 54.0
bridge-group 1 subscriber-loop-control
bridge-group 1 spanning-disabled
bridge-group 1 block-unknown-source
no bridge-group 1 source-learning
no bridge-group 1 unicast-flooding
no ip address
bridge-group 1 spanning-disabled
ip address 10.32.50.1 255.255.255.0
ip nat inside
router eigrp 10
no passive-interface Tunnel1
network 10.48.12.22 0.0.0.0
network 10.100.1.52 0.0.0.0
tacacs-server host 172.27.10.212 key 7 <removed>
tacacs-server host 10.2.100.208 key 7 <removed>
bridge 1 route ip
line con 0
no modem enable
line aux 0
line vty 0 4
transport input ssh
scheduler max-task-time 5000
Usually, this message is related to routing loop where you are learning tunnel destination addresses inside the tunnel.
Could you check which routes the router is using to join 126.96.36.199 and 188.8.131.52 ?
Can this problame be the cause of 881 router crashing?
Using IOS 12.4(20).T3 and this log is last before crashing:
*May 19 08:17:56.909: %ADJ-5-PARENT: Midchain parent maintenance for IP midchain out of Tunnel1, addr 192.168.188.65 (incomplete) - looped chain attempting to stack
I would not think that this error would cause a crash of a router. I have seen that error many times when a GRE tunnel is configured and the physical interface that terminates the tunnel changes state to down. When you see the error message %ADJ-5-PARENT is there perhaps a message just before it about a physical interface changing state to down?
If the router crashes does it produce a crashinfo file? If so there would perhaps be information in the crashinfo file that might help understand the cause of the crash.
I took this log exectly from the crashinfo file. It is very big for posting here, and this log string was last recorded before crashing.
Crashinfo files are frequently pretty big. And without seeing what is in the crashinfo file it may be difficult to know for sure what caused the crash. Sometimes the output of show version may have some information about what caused the last restart. Perhaps you could post the output of show version for your 881?
Is this router under a maintenance contract? If so then opening a case with Cisco TAC is a possibility and they may be able to determine the cause of the crash.
I can only say that I have seen this log message quite a bit and I have yet to see it cause a crash on the routers that I was working on.
Thanks for posting the crashinfo file. I have taken a look at the file. Unfortunately I do not have the proper tools to fully interpret the crashinfo. A full interpretation would require the resources available through Cisco TAC. The crashinfo does allow us to make some observations and suggestions. My primary observation is that the crashinfo file has multiple references to traceback such as this one
-Traceback= 0x8039D95C 0x8039D8B8 0x800A4320 0x800A52C4 0x800A5B80 0x8039F9AC 0x803A273C
I do not have access to tools that would interpret this particular traceback but I can observe that traceback is almost always an indication of a software problem. And the usual solution to a problem involving traceback is to change the version of code that the router is running.
Hello Richard, I'm also using a DMVPN configuration and just had this issue in my network. Before I saw:
%ADJ-5-PARENT: Midchain parent maintenance for IP midchain out of Tunnel0 - looped chain attempting to stack
I saw the physical interface protocol go down. This actually happened to 2 sites that were within 10 miles of each other. Do you think it was an ISP issue that made the physical interfaces go down or a problem related to OSPF? I just implemented OSPF network wide this past Monday so it has me concerned. Thanks a lot for your advice! -Mark
I am fairly certain that this reflects a physical/ISP problem and not an issue with OSPF. Your observation that you saw a message about the physical interface protocol go down pretty much confirms this. I do not see how OSPF could make a physical interface protocol go down.
Keep an eye on it and let us know what you find. But I am confident that it is not an OSPF issue.
Hello and good morning! I just wanted to add to what Richard is suggesting (that it is a physical/ISP issue and not an OSPF/recursion issue) with respect to what the probable cause could be. We are running a point-to-point SVTI between Maryland and Alaska and everything has been working great for about 2 months now. Then, all of sudden, yesterday I get a SolarWinds alert indicating that my tunnel interfaces on either end had gone down. Here is the log output from both routers (full disclosure, I changed the tunnel/IP addresses from the originals):
000236: *Jan 31 2016 01:11:35.793 UTC: %ADJ-5-PARENT: Midchain parent maintenance for IP midchain out of Tunnel64620 - looped chain attempting to stack
000237: *Jan 31 2016 01:11:42.365 UTC: %TUN-5-RECURDOWN: Tunnel64620 temporarily disabled due to recursive routing
000238: *Jan 31 2016 01:11:42.365 UTC: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel64620, changed state to down
000239: *Jan 31 2016 01:11:42.365 UTC: %OSPF-5-ADJCHG: Process 64620, Nbr 184.108.40.206 on Tunnel64620 from FULL to DOWN, Neighbor Down: Interface down or detached
000240: *Jan 31 2016 01:16:32.825 UTC: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel64620, changed state to up
000241: *Jan 31 2016 01:16:41.285 UTC: %OSPF-5-ADJCHG: Process 64620, Nbr 220.127.116.11 on Tunnel64620 from LOADING to FULL, Loading Done
...and here is the Alaska router:
00282: Jan 31 2016 01:12:09.930 UTC: %ADJ-5-PARENT: Midchain parent maintenance for IP midchain out of Tunnel64621 - looped chain attempting to stack
000283: Jan 31 2016 01:12:13.418 UTC: %TUN-5-RECURDOWN: Tunnel64621 temporarily disabled due to recursive routing
000284: Jan 31 2016 01:12:13.418 UTC: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel64621, changed state to down
000285: Jan 31 2016 01:12:13.418 UTC: %OSPF-5-ADJCHG: Process 64621, Nbr 18.104.22.168 on Tunnel64621 from FULL to DOWN, Neighbor Down: Interface down or detached
000286: Jan 31 2016 01:16:53.891 UTC: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel64621, changed state to up
000287: Jan 31 2016 01:16:54.335 UTC: %OSPF-5-ADJCHG: Process 64621, Nbr 22.214.171.124 on Tunnel64621 from LOADING to FULL, Loading Done
When I first saw these log messages, my immediate thoughts were that we couldn't have "recursive" routing just all-of-a-sudden pop up out of nowhere. If there was recursive routing we should have/would have seen it long before this incident happened on the 31st. A quick search of the support forums (always my "go to" when things like this pop up!) turned up this thread and since I haven't seen this error before I was thrilled to find Richard's feedback that it is either a physical problem or something with the ISP.
Ironically, about 3 months ago we transitioned the connectivity between these two routers to a 3rd party provider's MPLS L3 VPN network. I fired off a copy of our logs to see if they might have seen anything during this time frame and "BINGO"! They responded by letting me know that they had performed a JUNOS upgrade on their router which links from Seattle to Alaska...nailed it! :-) Now we just need to figure out why they never told us they were doing that! ;-)))
So, to Richard's point, it definitely appears that you can end up with this error if there is something going on in the ISPs network that causes a loss in connectivity.
Also wanted to say, "THANK YOU" to Richard (whose posts I always seem to come across when looking for information!) for always sharing what seems to be "spot on" information!!!
Thank you for the kind words about my contributions. I am glad that you have found them helpful and hope that you will continue to be active and to post to the forum.
I remember the case that I opened with Cisco TAC when I encountered this issue and am surprised that I did not include an explanation of this symptom for this forum. So let me address that now.
We believe that this is what is happening:
- the router has a route for the tunnel endpoint that points to a next hop out some physical interface (this is pretty necessary to avoid the recursive routing problem with the tunnel).
- the router runs a dynamic routing protocol over the tunnel and learns routes (probably including a default route) through the tunnel.
- some event impacts the physical interface or something causes it to go to the protocol down state.
- the router withdraws the route to the tunnel end point (since the next hop is not available or the outbound interface is in the protocol down state).
- for a brief period the dynamic learned route is still present in the routing table and the router believes that it can reach the tunnel end point by going through the tunnel (which is the classic definition of recursive routing). So the router generates the midchain parent maintenance error message.
- the dynamic routing protocol converges and removes the route from the routing table.
- now things can return to normal, wait for the physical interface to recover, and re-establish the tunnel.
So the midchain parent maintenance error is essentially an issue of timing of how quickly the route based on the physical interface is removed vs how quickly the dynamic learned route is removed. And in my experience the midchain parent maintenance message is always related to an issue with the connection going out the outbound interface (or perhaps an issue in the provider network).