Hi guys,
Our customer has a WAN network (DCoS) and has voice protected across it. I work for an integrator assisting the customer with their CPE routers end-to-end. Essentially CPE routers extend PSTN lines end-to-end over voice-ports and voip call legs between locations. On CPE routers we're seeing voice traffic being marked/matched end-to-end. We're pretty sure that the ISP is somehow shaping/causing delays on the voice traffic interstate but now we're trying to gather some evidence.
To help isolate, test and provide evidence we were planning on using UDP-Jitter in IP SLA. I've been trying to get this enabled on live routers without success due to the reason "No connection" in the initiators show output. An example of the "show ip sla statistics" on a production device is shown below:
SITE-1#show ip sla statistics
IPSLAs Latest Operation Statistics
IPSLA operation id: 100
Type of operation: udp-jitter
Latest RTT: NoConnection/Busy/Timeout
Latest operation start time: 17:26:47 BNE Mon Jul 2 2012
Latest operation return code: No connection
RTT Values:
Number Of RTT: 0 RTT Min/Avg/Max: 0/0/0 milliseconds
Latency one-way time:
Number of Latency one-way Samples: 0
Source to Destination Latency one way Min/Avg/Max: 0/0/0 milliseconds
Destination to Source Latency one way Min/Avg/Max: 0/0/0 milliseconds
Jitter Time:
Number of SD Jitter Samples: 0
Number of DS Jitter Samples: 0
Source to Destination Jitter Min/Avg/Max: 0/0/0 milliseconds
Destination to Source Jitter Min/Avg/Max: 0/0/0 milliseconds
Packet Loss Values:
Loss Source to Destination: 0
Source to Destination Loss Periods Number: 0
Source to Destination Loss Period Length Min/Max: 0/0
Source to Destination Inter Loss Period Length Min/Max: 0/0
Loss Destination to Source: 0
Destination to Source Loss Periods Number: 0
Destination to Source Loss Period Length Min/Max: 0/0
Destination to Source Inter Loss Period Length Min/Max: 0/0
Out Of Sequence: 0 Tail Drop: 0
Packet Late Arrival: 0 Packet Skipped: 0
Voice Score Values:
Calculated Planning Impairment Factor (ICPIF): 0
Mean Opinion Score (MOS): 0
Number of successes: 0
Number of failures: 1
Operation time to live: Forever
Number of failures continues to increase every 30 seconds or so. The configuration of SITE-1 is as follows:
SITE-1#show run | sec ip sla
ip sla 100
udp-jitter 10.2.2.2 16584 source-ip 10.1.1.1 source-port 16384
tos 30
owner NetFlow
timeout 60000
ip sla schedule 100 life forever start-time now
The configuration of SITE-2 (IP SLA Responder) is as follows:
SITE-2#show run | sec ip sla
ip sla responder
For the sake of privacy I've adjusted the IPs above as well as the router hostnames. The 10.2.2.2 IP lives on SITE-2's router and 10.1.1.1 lives on SITE-1's router.
If I do the above in GNS it works perfectly so I think something else is going on here. Either I've got a bug (doubt it) or something is preventing the UDP from connecting/responding for the IP SLA. Frustratingly, if I do a "show ip sla responder" on the SITE-2 router... it says it received the packets normally without error:
SITE-2#show ip sla responder
General IP SLA Responder on Control port 1967
General IP SLA Responder is: Enabled
Number of control message received: 123 Number of errors: 0
Recent sources:
10.1.1.1 [18:06:35.989 BNE Mon Jul 2 2012]
10.1.1.1 [18:06:30.989 BNE Mon Jul 2 2012]
10.1.1.1 [18:06:25.989 BNE Mon Jul 2 2012]
10.1.1.1 [18:05:35.989 BNE Mon Jul 2 2012]
10.1.1.1 [18:05:30.989 BNE Mon Jul 2 2012]
Recent error sources:
Permanent Port IP SLA Responder
Permanent Port IP SLA Responder is: Enabled
udpEcho Responder:
IP Address Port
If I enable trace debug ("debug ip sla trace") on SITE-1's router I get the following...
Jul 2 08:13:25.993: IPSLA-INFRA_TRACE:OPER:100 slaSchedulerEventWakeup
Jul 2 08:13:25.993: IPSLA-INFRA_TRACE:OPER:100 Starting an operation
Jul 2 08:13:25.993: IPSLA-OPER_TRACE:OPER:100 Starting jitter operation
Jul 2 08:13:25.993: IPSLA-OPER_TRACE:OPER:100 Ctrl msg: id=116, type=1, len=52, dest_ip=10.2.2.2, enablePort=16584, duration=60200
Jul 2 08:13:25.993: IPSLA-OPER_TRACE:OPER:100 table_id=0, topo_id=65535
Jul 2 08:13:30.993: IPSLA-OPER_TRACE:OPER:100 Timeout
Jul 2 08:13:30.993: IPSLA-OPER_TRACE:OPER:100 Ctrl msg: id=117, type=1, len=52, dest_ip=10.2.2.2, enablePort=16584, duration=60200
Jul 2 08:13:30.993: IPSLA-OPER_TRACE:OPER:100 table_id=0, topo_id=65535
Jul 2 08:13:35.993: IPSLA-OPER_TRACE:OPER:100 Timeout
Jul 2 08:13:35.993: IPSLA-OPER_TRACE:OPER:100 Ctrl msg: id=118, type=1, len=52, dest_ip=10.2.2.2, enablePort=16584, duration=60200
Jul 2 08:13:35.993: IPSLA-OPER_TRACE:OPER:100 table_id=0, topo_id=65535
Jul 2 08:13:40.993: IPSLA-OPER_TRACE:OPER:100 Timeout
Jul 2 08:13:40.993: IPSLA-OPER_TRACE:OPER:100 No connection
Jul 2 08:13:40.993: IPSLA-INFRA_TRACE:OPER:100 Updating result
