One-way packet-loss

Unanswered Question
Oct 30th, 2007

hi, there is the CE connected to a sub interface of a metro ethernet and when the CE ping the PE no packet-loss appears. One the other hand when the PE pings the CE there a packet loss. What might be the problem?

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
william.caban Tue, 10/30/2007 - 14:39

Many things. For clarification, can you provide some additional information?

- What type of media are we talking here? Ethernet? ATM? FR?

- What do you seen in the counters? Errors? Clean?

- Do you have any global ICMP rate-limit rules in your device?

- Do you have any rpf check on the subinterface? If so, try removing it and trying the pings again.

- If this is L3VPN, are you doing an extended ping from the VRF or loopback associated to the VRF?

- On the CE side, make sure you know the IP of the source interface and that the route entry is stable (i.e. no route flapping)?

v.matiakis Wed, 10/31/2007 - 00:42

Goodmorning,

The media type is LX and i am doing an extended ping from the PE through a sub-interface with a repeat count of 5000 and a datagram size of 500. The counters are clean and no ICMP rate-limit rules exist. The packet loss happens only from the PE'S side and only with a specific customer of this sub-interface. Other customers exist in other subifs and they have no problem.

v.matiakis Wed, 10/31/2007 - 00:58

Goodmorning,

The media type is LX and i am doing an extended ping from the PE through a sub-interface with a repeat count of 5000 and a datagram size of 500. The counters are clean and no ICMP rate-limit rules exist. The packet loss happens only from the PE'S side and only with a specific customer of this sub-interface. Other customers exist in other subifs and they have no problem.

kewlnetguy Thu, 11/01/2007 - 03:56

Hi,

This seems to be very specific issue, please note that ICMP is a two way progress if while pinging from CE to PE you observe no drops then it is highly likely that it is path issue with respect to source / Destination of IP.

Example :

When you ping from CE in default way - without specifying the source, the source for ICMP packet will be the outgoing interface. SO you have to check that you are taking the source IP as " S " and pinging the Destination IP " D " from CE.

Now from the PE you have to ping Destination IP " S " and source IP as " D ".

Please share the extended trace reports for Netpros to analyse and assist you.

Cheers

Prince

v.matiakis Thu, 11/01/2007 - 04:28

Here is the output off the extended ping if that is what you ask for

CE:

Target IP address: 192.168.1.173

Repeat count [5]: 500

Datagram size [100]:

Timeout in seconds [2]:

Extended commands [n]: y

Source address or interface: 192.168.1.174

Type of service [0]:

Set DF bit in IP header? [no]:

Validate reply data? [no]:

Data pattern [0xABCD]:

Loose, Strict, Record, Timestamp, Verbose[none]:

Sweep range of sizes [n]:

Type escape sequence to abort.

Sending 500, 100-byte ICMP Echos to 192.168.1.173, timeout is 2 seconds:

Packet sent with a source address of 192.168.1.174

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!

Success rate is 100 percent (500/500), round-trip min/avg/max = 1/1/4 ms

PE:

adr-core00#ping vrf Matiakis

Protocol [ip]:

Target IP address: 192.168.1.174

Repeat count [5]: 500

Datagram size [100]: 500

Timeout in seconds [2]:

Extended commands [n]: y

Source address or interface: 192.168.1.173

Type of service [0]:

Set DF bit in IP header? [no]:

Validate reply data? [no]:

Data pattern [0xABCD]:

Loose, Strict, Record, Timestamp, Verbose[none]:

Sweep range of sizes [n]:

Type escape sequence to abort.

Sending 500, 500-byte ICMP Echos to 192.168.1.174, timeout is 2 seconds:

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!

Success rate is 99 percent (498/500), round-trip min/avg/max = 1/1/4 ms

(note that the Ip addresses are changed)

kewlnetguy Fri, 11/02/2007 - 03:11

I am assuming the scenario for your PE and CE connectivity.

Lets say that PE and CE are connected as below:

PE has Gig port connected to a switch - now as you have already stated that the other customers have no drops for either PE to CE or CE to PE response.However only this customer in question has PE to CE drops - ( YOu have increased the payload size from default 100 to 500 while pinging from PE to CE in the pasted observation :) ).

If the Ethernet link of customer in question is not landing directly on the switch which is connected to the PE directly : i am assuming that it is connected on Access Switch ( the switch where Access Service Provider is terminating the link in SP site) and then after this the Access Switch is connected to a ring of switches for redundancy and various services.

In the above scenario - we need to check if the last mile access switch is serving to only this customer or any other customer as well. I recommend connecting another device - PC / Switch /Router and check the response from PE to CE and ensure if the response is same for the new test device.

In case the similar result is observed. The Issue is in the Service Provider Switch network.

In case the similar result is not observed. The other options to explore will remain is the switch port and the customer router port.

::: Assuming that L2 path for customer is via seprate path or some QoS or some edge device/port issue.

_____________

If i assume that the Customer router is connected direcly to the Single switch which is connected to the PE than also we should check the devices in steps.

______________

Anther round about. Enure that the interfaces are set with full duplex mode and maximum speed setting fixed manually and are not ruynning in auto negotiation mode.

_____________

Also check if any of the device in between or PE or CE Process CPU shorts high when PIng from PE is intiated.

I faced this once and the issue was in the QoS config. Two days head ache.

Do share your observation if you have more and are not to able to follow any of the above steps

Do share if my assumptions do not match the scenario in your case.

Happy troubleshooting!

Cheers!

Prince

bhartispbase Fri, 11/02/2007 - 04:56

will you pleas try any other ip address of customer end . sometimes router give low prority to wan to wan ping .

Please try to ping any Lan ip address of CE end in that case CE end , process switching come into picture .

so it will indicates that there will be no issue in performance ..

Regards

Shariq

v.matiakis Fri, 11/02/2007 - 05:59

Good afternoon gentlemen,

From the PE side every customer is on a sub-interface. The config is :

interface GigabitEthernet1/0/3.630

bandwidth 3000

encapsulation dot1Q 630

ip vrf forwarding Matiakis

ip address 192.168.1.173 255.255.255.252

no ip redirects

no ip directed-broadcast

no ip proxy-arp

no cdp enable

service-policy output Shaping_3Mbps

and the config of the customer side is:

interface FastEthernet0/0

ip address 192.168.1.174 255.255.255.252

duplex auto

speed auto

end.

v.matiakis Fri, 11/02/2007 - 06:04

I am sending this before i make a change because i want to make sure that this is the cause of the packet-loss. If anyone is really certain please tell! thanx

v.matiakis Fri, 11/02/2007 - 07:11

Last thing.... Every other customer with same configuration works just fine. Duplex and speed are set to auto.

william.caban Fri, 11/02/2007 - 09:05

I've read your post and I can not tell you what might be wrong in this case. Now, from my experience at Telcos I can tell you it is a good practice to set duplex and speed manually.

The autonegotiation does not always work and may cause instability at some links and in some other cases it cause some really odd behaviors. So, as a good practice, always set it manually with your customers.

Also, try doing pings with larger MTUs and the DF bit set in order to detect any logical MTU mismatch. Most of the vendors adjust the MTU of the port when it is set to Dot1Q and you have to take it into consideration. For example, in the old Cisco routers and in the Nortel Secure Routers, you can not change the MTU and if you set the port to dot1q it will substract 4 or 8 bytes. You as a provider might have a larger MTU that your customer, so you will see the issue in one-way only.

v.matiakis Wed, 11/07/2007 - 03:41

The status did not change even if i did specify the speed and the duplex.

Actions

This Discussion