Re: Rate Limit and QoS

Braunimus · ‎03-05-2012

Hello,

I am having an issue with VoiP phones giving me an insufficient bandwidth message. I have three remote locations connected to our main building using 2 Mb point to point ethernet solutions through TWC. Each remote location has a Cisco WS-C3560-24PS running IOS C3560-IPBASE-M, version 12.2(25) and have the cable modems plugged into port 1 on them. The remote buildings are labeled 192.168.101.xxx, 192.168.102.xxx, and 192.168.103.xxx. There are 14-16 VoiP phones in each remote building. The main building being in the subnet of 192.168.100.xxx. I have the 3560s connecting to a single port on a 2801 in the main building, all using the subnet of 192.168.253.xxx The phone server sits in our network at 192.168.100.203. I have created the ACLs, class maps, and policy maps on all of the equipment.

For the remote buildings I have the following:

ACL

===========

Extended IP access list VOIP

permit tcp any host 192.168.100.203 dscp ef

permit tcp any host 192.168.100.203 eq 5566

permit tcp any host 192.168.100.203 eq 5570

permit tcp any host 192.168.100.203 range 3998 3999

permit tcp any host 192.168.100.203 range 6800 6802

permit udp any host 192.168.100.203 dscp ef

permit udp any host 192.168.100.203 eq 5566

permit udp any host 192.168.100.203 eq 5567

permit udp any host 192.168.100.203 eq 20001

permit udp any host 192.168.100.203 range bootps tftp

permit udp any host 192.168.100.203 range 5004 5064

permit udp any host 192.168.100.203 range 6004 7039

permit ip any host 192.168.100.203 dscp ef

permit ip any any dscp ef

permit ip any host 192.168.100.203

Extended IP access list DEFAULT

permit ip any any

Class Map

===========

class-map match-any VOIP

match access-group name VOIP

class-map match-any DEFAULT

match access-group name DEFAULT

Policy-Map

===========

policy-map PER-PORT-POLICING

class VOIP

set dscp ef

police 128000 8000 exceed-action drop

class DEFAULT

set dscp default

I have the policy map applied to all ports with PCs and VoiP phones connected to them. Here is what I have on the ports fa0/2 - 24

interface FastEthernet0/2

switchport access vlan 101

switchport mode access

service-policy input PER-PORT-POLICING

srr-queue bandwidth share 10 10 60 20

srr-queue bandwidth shape 10 0 0 0

priority-queue out

spanning-tree portfast

Here is the setup on the interface connected to TWC

interface FastEthernet0/1

description "Connection to Time Warner BCE"

no switchport

ip address 192.168.253.101 255.255.255.0

speed 10

duplex full

srr-queue bandwidth share 10 10 60 20

srr-queue bandwidth shape 10 0 0 0

srr-queue bandwidth limit 20

priority-queue out

mls qos trust dscp

auto qos voip trust

The 2801 in the main building is running IOS C2801IPBASE-M, version 12.3(8). Here are the settings I have applied on it:

ACL

===========

access-list 101 permit ip any 192.168.101.0 0.0.0.255

access-list 102 permit ip any 192.168.102.0 0.0.0.255

access-list 103 permit ip any 192.168.103.0 0.0.0.255

ip access-list extended VOIP

remark "Phone System ports"

permit tcp host 192.168.100.203 any dscp ef

permit tcp host 192.168.100.203 eq 5566 any

permit tcp host 192.168.100.203 eq 5570 any

permit tcp host 192.168.100.203 range 3998 3999 any

permit tcp host 192.168.100.203 range 6800 6802 any

permit udp host 192.168.100.203 any dscp ef

permit udp host 192.168.100.203 eq 5566 any

permit udp host 192.168.100.203 eq 5567 any

permit udp host 192.168.100.203 eq 20001 any

permit udp host 192.168.100.203 range bootps tftp any

permit udp host 192.168.100.203 range 5004 5064 any

permit udp host 192.168.100.203 range 6004 7039 any

permit ip host 192.168.100.203 any dscp ef

permit ip any any dscp ef

ip access-list extended DEFAULT

permit ip any any

Class Map

===========

class-map match-any VOIP

match access-group name VOIP

class-map match-any DEFAULT

match access-group name DEFAULT

Policy Map

===========

policy-map WAN-POLICING

class VOIP

set dscp ef

priority 1536

class DEFAULT

set dscp default

interface FastEthernet0/1

ip address 192.168.253.100 255.255.255.0

service-policy output WAN-POLICING

speed 100

duplex full

rate-limit output access-group 101 2048000 64000 96000 conform-action transmit exceed-action drop

rate-limit output access-group 102 2048000 64000 96000 conform-action transmit exceed-action drop

rate-limit output access-group 103 2048000 64000 96000 conform-action transmit exceed-action drop

I have put a hub in to capture traffic via Wireshark to see if DSCP flags are being appropriately marked and I do see that all VoiP packets are getting marked with as EF. However, I have been receiving phone calls from people in the remote buildings stating that their phones will cut out, flash Insufficient Bandwidth on the LCD displays and then the call will cut back in. I am wondering if the 2801 is not applying QoS with the rate-limits in mind since it is set to 100 Mb, or is it an issue with trying to take 3 remote locations and bring them down into 1 port on the 2801?

rowseyba1 · ‎03-13-2012

Mark,

I may just be missing it, but I don't actually see where you applied your policy map on the out going interface of your WAN device on the first TWC connection. Also, have you confirmed that you aren't exceeding the BW on those links? Also, is TWC honoring your QoS marks on the point to point? What kind of VoIP phones are they?

Braunimus · ‎03-14-2012

The TWC connection is plugged into FastEthernet0/1 on the Cisco 3560 which will not allow me to put a service-policy on the outbound traffic. It will only allow me to apply it to input. So I was using the service-policy input on ports 2 - 24 to mark and limit the traffic for each user. I do have a NAGIOS monitoring system setup and I will get emails from it stating ping time is rather high at times (<200ms) and when I login to the 3560 I can see the traffic going out is usually over 1.7 Mb/sec. To check if TWC is not changing my packets I put a hub in on each of the lines and have used wireshark to capture the packets and look at them to make sure all of my markings are still there and they indeed are. We are using Mitel phones model number 8662.

rowseyba1 · ‎03-14-2012

Can you match the problem to those times of high utilization or does it happen randomly even if you aren't using all of your bandwidth?

The reason I ask is I have run into a situation where not being able to have QoS on both sides of the circuit has caused quality issues when BW utilization is high. Once we were able to put a service policy on both sides of the connection, it got better, but we still ended up having to increase the bandwidth a bit to compensate fully.

Edit: I did some research on your Insufficient Bandwidth message on the Mitel phones. It seems this may have something to do with latency on the network as well. The posts that I saw said something about the phone having to receive 60% of frames in a certain amount of time. If you google "mitel insufficient bandwidth" the first article will explain further.

Braunimus · ‎03-20-2012

Sorry it has taken me a while to respond. I have noticed that it is not always when bandwidth is high. I do have NAGIOS deployed and checking ping times on the end points to see if the latency reaches over 100ms and sometimes when there are phone issues the latency does get high. This also usually correlates to high utilization on the line. I did just have an issue today where latench was <20ms, and we were only using tops of 25% of the bandwidth. Phones worked for some while data applications from the PCs didn't and vica versa. I am beginning to think I am crazy.

The phone system was put in by another company and I am going to get on the phone with them to see if maybe they can help point me in a good direction to see if I can get this resolved. I do have a service policy applied outward on the Cisco 2801, but I am unable to apply an output policy on the Cisco 3560 interfaces. Reading the Cisco documentation has pointed me in the direction of applying a service policy to the input side of the client facing interfaces and letting srr-queue and priority-queue out handle pushing and prioritizing traffic on the WAN facing interface.

boulest · ‎03-20-2012

If I were you I would do the following

1- before I get the phone company involved. I would findout where the latency happning

a- start with both ends of the ciruits directly on the router interface and see if you have any latency.

b- remove all your bandwidth rate limiting and see if you still getting the bandwidth error on the phones.

c- find out if this happen on active calls or when the phones are ideal.

d- if on active calls, was it G711 or G729 call, was it a conf call, transfered call, or direct call.

e- do you have a way to monitor latency on the Metro E?

then I go to the phone company

Joseph W. Doherty · ‎03-21-2012

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

LAN switches, like the 3560s, are not ideal for precise QoS. At-a-glance, your remote 3560 configs looks okay although do be aware the interface limiter isn't exact. I.e. your 20% for the 10 Mbps could be higher.

On your HQ side, try a policy similar to:

class-map match-any Site1

match access-group 101

class-map match-any Site2

match access-group 102

class-map match-any Site3

match access-group 103

policy-map WAN-QoS

class VOIP

set dscp ef

priority percent 50

class class-default

set dscp default

fair-queue

policy-map WAN-Parent

class Site1

shape average 1800000

service-policy WAN-QoS

class Site2

shape average 1800000

service-policy WAN-QoS

class Site3

shape average 1800000

service-policy WAN-QoS

interface FastEthernet0/1

ip address 192.168.253.100 255.255.255.0

service-policy output WAN-Parent

!remote the rate-limiters