QoS for coloured VoIP packets doesn't work properly

[lhal]

Hello everyone,

I have come across a strange problem with QoS (Class of service for VoIP packets).

Please see below all the facts:

-I have a 2M leased line (1920 Kbps)

-I have given quality to VoIP traffic with set precedence 5.

-There are matched packets to class-map PREMIUM as expected however I get a drop rate 3000 bps while IP traffic doesn't exceed on a 5 min rate 73000 bps. (Traffic should be able to scale fine up to 30% of the available BW --which is much higher than 72000 bps--).

-To make things a bit more complicated we tried exactly the same QoS on another router (same platform, same IOS) with no probs.

-There are no errors on the line, the line has no CRC nor input/output errors.

-Configuration wise there is no problem, is it some kind of an IOS bug?

Has anyone come across a similar issue; are there any ideas what might cause this QoS misbehaviour?

Could it be a hardware fault (buffers etc.)

Thanks a lot in advance.

sh policy-map int s0/0 output class PREMIUM

Serial0/0

Service-policy output: QOS-CPE-OUT

Class-map: PREMIUM (match-any)

43392 packets, 24193568 bytes

5 minute offered rate 73000 bps, drop rate 3000 bps

Match: access-group 170

20782 packets, 11659128 bytes

5 minute rate 72000 bps

Match: ip precedence 5

22610 packets, 12534440 bytes

5 minute rate 1000 bps

Queueing

Strict Priority

Output Queue: Conversation 264

Bandwidth 30 (%)

Bandwidth 576 (kbps) Burst 14400 (Bytes)

(pkts matched/bytes matched) 43392/24193568

(total drops/bytes drops) 91/135264

QoS Set

precedence 5

Packets marked 43392

QOS configuration:

class-map match-any GOLD

match access-group 171

match ip precedence 3

class-map match-any PREMIUM

match access-group 170

match ip precedence 5

!

!

policy-map QOS-CPE-OUT

class PREMIUM

priority percent 30

set ip precedence 5

class GOLD

bandwidth percent 30

set ip precedence 3

ping 10.167.2.153 source 10.167.32.122 rep 100 size 1500

Type escape sequence to abort.

Sending 1000, 1500-byte ICMP Echos to 10.167.2.153, timeout is 2 seconds:

Packet sent with a source address of 10.167.32.122

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

----------------------------------------------------------------------

Success rate is 98 percent (988/1000), round-trip min/avg/max = 16/17/216 ms

[spremkumar]

hi

I dont think it can be a ios bug coz the same ios u r running in 2 different devices.

I would suggest to post out the ACLs both 170 and 171 which you are using here to match the traffic.

Is ACL 170 representing ur VOIP traffic ??

regds

[lhal]

Hi there, thanks for your reply,

Yes, basically my ACL 170 matches my source IP (which is basically a loopback address) in order to set precedence to 5 and represends my VoIP traffic.

ACL 170:

access-list 170 deny udp host 10.167.32.122 any eq 5060

access-list 170 permit ip host 10.167.32.122 any

ACL 171 matches for class GOLD.

ACL 171:

access-list 171 permit udp host 10.167.32.122 any eq 5060

access-list 171 permit ip host 80.106.200.122 172.30.94.0 0.0.1.255

I can see matched packets but the I see drop rate even when my traffic is low.

I appreciate your help

rgds

Anthony

[spremkumar]

Hi

You have mentioned the following statement under both the ACLs 170 and 171.

permit udp host 10.167.32.122 any eq 5060

ACL 170 is going to be ur premium so i would better prefer having a singel ACL entry instead of 2 .

Plz try to remove from 171 for which ur doing a gurantee of b/w instead 170 where ur giving priority for the matching traffic.

regds

[lhal]

Thanks for your advice,

I have removed second entries from both ACLs with unfortunatelly the same results (some packet-loss).

sh policy-map int s0/0 output class PREMIUM

Serial0/0

Service-policy output: QOS-CPE-OUT

Class-map: PREMIUM (match-any)

109485 packets, 41820900 bytes

5 minute offered rate 250000 bps, drop rate 0 bps

Match: access-group 170

100699 packets, 36949580 bytes

5 minute rate 249000 bps

Match: ip precedence 5

8786 packets, 4871320 bytes

5 minute rate 2000 bps

Queueing

Strict Priority

Output Queue: Conversation 264

Bandwidth 30 (%)

Bandwidth 614 (kbps) Burst 15350 (Bytes)

(pkts matched/bytes matched) 30529/36768810

(total drops/bytes drops) 408/612212

QoS Set

precedence 5

Packets marked 97468

Success rate is 99 percent (2975/3000), round-trip min/avg/max = 16/17/216 ms

I 've got no logical explanation for it, line is way below saturation and pretty clear of errors, could it be some sort of a hardware problem (hardware buffers or something like that)?

Thanks

[spremkumar]

Hi

Can you try something inline with the below mentioned config ??

Also can u increase the ping nos starting from 1000,2000 and go on and check for the drops..

ACL 170:

access-list 170 deny udp host 10.167.32.122 any eq 5060

access-list 170 permit icmp host 10.167.32.122 any echo

access-list 170 permit icmp host 10.167.32.122 any echo-reply

ACL 171:

access-list 171 permit ip host 80.106.200.122 172.30.94.0 0.0.1.255

regds

[lhal]

Hi, after applying ACLs 170 amd 171 as above and trying with ICMP packets at 1000 bytes and 2000 bytes I get the following:

a) ACL 170 has got matches as expected:

sh ip access-lists 170

Extended IP access list 170

10 deny udp host 10.167.32.122 any eq 5060

20 permit icmp host 10.167.32.122 any echo (9100 matches)

30 permit icmp host 10.167.32.122 any echo-reply

b) For 1000 bytes packet size:

ping 10.167.2.153 source 10.167.32.122 rep 2000 size 1000

----!!!! omitted----

Success rate is 100 percent (2000/2000), round-trip min/avg/max = 12/12/212 ms

sh policy-map int s0/0 output class PREMIUM

Serial0/0

Service-policy output: QOS-CPE-OUT

Class-map: PREMIUM (match-any)

140889 packets, 62456740 bytes

5 minute offered rate 117000 bps, drop rate 0 bps

Match: access-group 170

5060 packets, 5150240 bytes

5 minute rate 116000 bps

Match: ip precedence 5

34200 packets, 18958200 bytes

5 minute rate 0 bps

Queueing

Strict Priority

Output Queue: Conversation 264

Bandwidth 30 (%)

Bandwidth 614 (kbps) Burst 15350 (Bytes)

(pkts matched/bytes matched) 61933/57404650

(total drops/bytes drops) 408/612212

QoS Set

precedence 5

Packets marked 128872

c)ping 10.167.2.153 source 10.167.32.122 rep 2000 size 2000

----!!!! omitted----

Success rate is 98 percent (1962/2000), round-trip min/avg/max = 20/23/224 ms

sh policy-map int s0/0 output class PREMIUM

Serial0/0

Service-policy output: QOS-CPE-OUT

Class-map: PREMIUM (match-any)

144984 packets, 66586736 bytes

5 minute offered rate 182000 bps, drop rate 0 bps

Match: access-group 170

9100 packets, 9246400 bytes

5 minute rate 180000 bps

Match: ip precedence 5

34255 packets, 18992036 bytes

5 minute rate 0 bps

Queueing

Strict Priority

Output Queue: Conversation 264

Bandwidth 30 (%)

Bandwidth 614 (kbps) Burst 15350 (Bytes)

(pkts matched/bytes matched) 66028/61534646

(total drops/bytes drops) 408/612212

QoS Set

precedence 5

Packets marked 132967

Sorry for the wrong outputs, I include them in order to give a better picture; seems to me that packet loss appears once packet size becomes large, ie. no drops when packet size is 1000 bytes but then we get packet drops when packet size is 2000 bytes...

I am sure that VoIP packets would be OK since are small in size however in theory should be no problem assembling/re-assembling bigger than MTU size packets and that seems to be an issue.

what do you think?

Thanks for your help

Anthony

[lhal]

CORRECTION:

SORRY, I meant LONG OUTPUTS not WRONG OUTPUTS.

Thnx

Anthony

[spremkumar]

Hi

In my prev mail i did mention about the ping counts not the byte size and as per the o/p pasted it sounds good w/o any drops with 1000byte size even.

I dont think any buffer error or some h/w error with the box..

regds

[mheusinger · Accepted solution]

Hi,

in your first post you gave a picture of 100 pings with 1500 Bytes each and seeing some drops.

Now with 17 ms average roundtrip time and 1500 Byte I get around 700 kbps throughput, which is clearly above the 576 kbps configured for the voip class.

I would be surprised, if there would be NO drops in case the interface is overloaded.

I think you are somewhat tricked by your 5 MINUTES average load collection time. Sending f.e. at full line speed for 10 seconds and being idle for 4 minutes and 50 seconds would average to less than 4% usage. The policer however works normally in a 125 ms time window and would start to drop if you are above rate for even a second.

As it looks to me: get things more accurate by reducing load average to 30 seconds and get a decent traffic generator with adjustable output for reliable measurements.

Hope this helps

Martin

[lhal]

Hi,

happy holidays for everyone,

thanks for your reply, it makes sense what you are saying, I must have been tricked by the 5 min average since what you are saying is true; I have been tricked also by the fact that trying the same tests on other routers I did not get any similar drops, however on a second though I will test both routers once again with the same tests and compare results,

thanks again for your help

[lhal]

Hi, I have performed exact the same tests on two different routers with the same VoIP config (& obviously different IP addresses) and I get similar results, however even if I have changed the load-interval to 30 secs and my ICMP tests are far longer now (repeat 10000) I still get drops while 30secs average doesn't exceed 330Kbps.

Have you got any ideas why?

Please see below outputs from those two tests (output with echo packets display omitted, we see only results).

Router A

ping 10.167.2.153 source 10.167.32.122 rep 10000 size 1500

Success rate is 99 percent (9914/10000), round-trip min/avg/max = 16/17/244 ms

sh policy-map int s0/0 output class PREMIUM

Serial0/0

Service-policy output: QOS-CPE-OUT

Class-map: PREMIUM (match-any)

305064 packets, 175407016 bytes

30 second offered rate 341000 bps, drop rate 0 bps

Match: access-group 170

22200 packets, 32388800 bytes

30 second rate 339000 bps

Match: ip precedence 5

172135 packets, 95423516 bytes

30 second rate 0 bps

Queueing

Strict Priority

Output Queue: Conversation 264

Bandwidth 30 (%)

Bandwidth 614 (kbps) Burst 15350 (Bytes)

(pkts matched/bytes matched) 226108/170354926

(total drops/bytes drops) 408/612212

QoS Set

precedence 5

Packets marked 293047

Router B:

ping 10.167.2.169 source 10.167.32.126 repeat 10000 size 1500

sh policy-map int s0/0 output class PREMIUM

Serial0/0

Service-policy output: QOS-CPE-OUT

Class-map: PREMIUM (match-any)

634754 packets, 356868667 bytes

30 second offered rate 324000 bps, drop rate 2000 bps

Match: access-group 170

15944 packets, 13846523 bytes

30 second rate 322000 bps

Match: ip precedence 5

618810 packets, 343022144 bytes

30 second rate 0 bps

Queueing

Strict Priority

Output Queue: Conversation 264

Bandwidth 30 (%)

Bandwidth 595 (kbps) Burst 14875 (Bytes)

(pkts matched/bytes matched) 627783/356424848

(total drops/bytes drops) 61/91276

QoS Set

precedence 5

Packets marked 634754

Thanks a lot

[mheusinger]

Hi,

again it looks to me like an averaging issue. Ping sends packets ... and then waits. So for the packet sent the speed would be at least twice your average, and while the reply comes in your router is idle. So 50% of time "full" speed, 50% of time no traffic - neglecting round trip time. This would assume at least 660 kbps during packet sent and zero bps while the ICMP reply is sent back.

660 kbps is above the limit (614 or 595 kbps respectively) and therefore drops are to be expected.

Hope this helps

Martin

P.S.: rate helpful posts