Cisco Support Community
Showing results for 
Search instead for 
Did you mean: 
Community Member

ACKed lost segment


I troubleshoot performance problems on our XEN Servers and I see some weird problems. Attached you can find a drawing with the network layout.

The Xenserver is connected with 4 x 10Gbit/s to 2 Nexus 5k switches with FEXes. The 6500's are a VSS cluster.
I've created an ERSPAN session with all 4 XE interfaces as source towards the 6500 cluster, config is on the drawing.

Now I did expect to see all the traffic going through the 2 x 10G interfaces on the XENServer and 2 x 1G for management, but I see a lot of "Acked missing segment" messages in the trace (see attached screenshot, it's taken from Cascade Pilot). The bandwidth over time never exceeds 25Mbit/s, with microbursts up to 600Mbit/s. I don't see any drops or interface errors on the N5k or the 6500.

The linux server where tcpdump is running:

tcpdump -i eth4 -n -B 1048576 -w trace.pcap -C 100M

tcpdump: listening on eth4, link-type EN10MB (Ethernet), capture size 65535 bytes
12909810 packets captured
12910471 packets received by filter
561 packets dropped by kernel

I have no idea why I see these acked missing segment messages, where do they get lost?

Cisco Employee

Hi Sven,A well-behaved TCP

Hi Sven,

A well-behaved TCP stack implementation should never acknowledge a segment that has not yet been received. The question, of course, is whether all packets captured by the ERSPAN session indeed made it to your machine performing the tcpdump and whether the machine was fast enough to store them. Is there any option of doing a local SPAN session and comparing the results?

I also wonder if other traffic analysis tools would report the same issue. Wireshark should be able to detect such occurences using the tcp.analysis.ack_lost_segment filter expression.

Best regards,

CreatePlease to create content