I have a problem with an IPv6 tunnel (ipv6ip) on a Cisco 1841 runnining 15.1(4)M4 or 15.1(4)M5.
It appears that a bug was introduced into 15.1(4)M4 and it is related to IPv6 tunnels and IP SLA.
description IPv6 Tunnel to x.x.x.x
ipv6 address 2001:XXXX:XXXX:XXXX::2/64
tunnel source ATM0/1/0.1
tunnel mode ipv6ip
tunnel destination x.x.x.x
After reloading the router, I can see the size of the input queue slowly increasing "Input queue: 30/75/0/0". It appears that specific packets are getting stuck in the input queue while still processing the majority of IPv6 packets. After a short period of time the input queue gets wedged "Input queue: 76/75/0/0" and it stops working for IPv6 unless I reload the router.
Tunnel64 is up, line protocol is up
Hardware is Tunnel
Description: IPv6 Tunnel to x.x.x.x
MTU 17920 bytes, BW 100 Kbit/sec, DLY 50000 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation TUNNEL, loopback not set
Keepalive not set
Tunnel source x.x.x.x (ATM0/1/0.1), destination x.x.x.x
Tunnel64 source tracking subblock associated with ATM0/1/0.1
Set of tunnels with source ATM0/1/0.1, 1 member (includes iterators), on interface <OK>
Tunnel protocol/transport IPv6/IP
Tunnel TTL 255
Tunnel transport MTU 1480 bytes
Tunnel transmit bandwidth 8000 (kbps)
Tunnel receive bandwidth 8000 (kbps)
Last input 00:00:15, output 00:00:15, output hang never
Last clearing of "show interface" counters never
Input queue: 76/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/0 (size/max)
30 second input rate 0 bits/sec, 0 packets/sec
30 second output rate 0 bits/sec, 0 packets/sec
2253 packets input, 1691254 bytes, 0 no buffer
Received 0 broadcasts (0 IP multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
1844 packets output, 730645 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 unknown protocol drops
0 output buffer failures, 0 output buffers swapped out
I also have an IP SLA probe on the router to verify if connectivity is working over the IPv6 tunnel:
ip sla 10
ip sla schedule 10 life forever start-time now
It appears that IP SLA return packets are getting stuck in the input queue as the input queue increments every time I receive a response to my IP SLA probe (every 60 seconds). I have tried to change the values in the probe (packet size, tos, etc) without any luck. I am able to ping the same IPv6 address normally from the command line without seeing this behaviour.
Can I deduce that this is a potential buffer leak - I can't find anything on Bug Toolkit relating to this.
Has anyone come across this issue before and know any workarounds?
Thanks in advance,
Hi Chris and Andrea,
Thanks for testing the various versions.
I was able to reproduce the issue in order to investigate further. Here's what I have been able to figure out:
The input queue leak was introduced by the fix for
CSCtn36227 Alignment correction at ipv6_checksum with IPv6 ping sweep
it is fixed via
CSCto56317 Backward compatibility regarding pak release strategy in ipv6_ping_send
CSCto56317 was committed into 15.2(1)T, but was never committed into the 15.1(4)M throttle.
I have put in a request to get CSCto56317 fixed in 15.1(4)M throttle. The next potential release that can get the fix is 15.1(4)M7 which is due out in October.
Please note that CSCto56317 is currently an internal defect. I will be making it external, but it may take a day or two for that change to propogate.
Unfortunately, I don't see an easy workaround to prevent the input leak in the meantime. Given that you decided to move back to 15.1(4)M3, I would assume ip sla is a required feature for you both. Another option would be to modify the frequency of the sla icmps and/or then increase the size of the input hold queue (hold-queue 24000 in) to allow more time before the input queue filled up. Changing the frequency from 1 min to 5 minutes and increasing the hold queue to 24000 should allow the device to go ~83 days before needing a reload to clear the input queue.