QoS markings

Answered Question
Nov 9th, 2009
User Badges:

Need help please with the following problem.

6509a(our own) >3550 (carriers) to another carrier owned 3550 to >6509b (our own).

6509a WS-SUP720-BASE pfc3a

6509b WS-X6K-S2U-MSFC2 pfc2

A port on 6509a, WS-X6724-SFP, is set to trust dscp markings. The port is set as a dot1q trunk. A port on the 6509b,WS-X6416-GBIC, is set to trust dscp markings. The port is set as a dot1q trunk. The carriers 2 3550's are in between carrying the traffice between the 2 6509's. The carrier is doing Q in Q. QoS dscp marked 46 packet leaves 6509a to carriers 3550 but never makes it across to other 6509b switch. Unable to determine what is stripping off the marking. Tried also changing trust to cos but did not resolve issue. Carrier does not have qos enabled on their switches so packet should pass thru in their q in q tunnel not remarked. Packet capture does verify packet at least going in to their 3550. What could be stripping off the marking.? Thanks.


Correct Answer by Giuseppe Larosa about 7 years 8 months ago

Hello John,


>> The trunk ports are set to trust dscp markings and not COS because the voip svrs are sending out a dscp 46 marking.


>> I guess I dont understand. I thought I am sending a tagged framed over the trunk..vlan 10? Is there a way to view the tagged raw frames to see the 802.1q header?


native vlan in 802.1Q is sent untagged unless you have an explicit option to send them tagged


this can be in part the cause of your problem as the provider switch expects to receive tagged frames to add the outer customer vlan-id (XXX) and to send the frame with double 802.1Q header to the other side.


Hope to help

Giuseppe



  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (2 ratings)
Loading.
Giuseppe Larosa Tue, 11/10/2009 - 05:38
User Badges:
  • Super Silver, 17500 points or more
  • Hall of Fame,

    Founding Member

Hello John,

if you can demonstrate your traffic is leaving with the correct marking on the inner 802.1Q tag only the L2 service provider devices can be the ones to remark to zero.


a possible solution could be that the Service provider enables QoS and trust all ports on both C3550.


Hope to help

Giuseppe


JOHN APONTE Tue, 11/10/2009 - 09:49
User Badges:

Tnx for the reply. The service provider did enable qos and trust ports but that failed also. The service provider has a TAC case open but TAC engr are saying this should work the way its configured. The TAC engr has now asked to see our 'show tech' from both the 6509's so we are waiting a response...but in the meantime... The service provider using their packet generator can craft a dscp 46 marked packet and can send it across and we see it ingress the 6509b switch when they are directly connected to the far end 3550. The service providers link is a backup link with the primary being another link from another service provider. The primary link is using the same HD/SW and the dscp 46 packet makes it across ok, the difference is I think the primary link is not doing any type of q in q. We have been working this for probably a month now with the backup service provider with no sucess.

Giuseppe Larosa Tue, 11/10/2009 - 10:25
User Badges:
  • Super Silver, 17500 points or more
  • Hall of Fame,

    Founding Member

Hello John,


>> The primary link is using the same HD/SW and the dscp 46 packet makes it across ok, the difference is I think the primary link is not doing any type of q in


yes I think the problem is caused by QinQ inplementation on C3550 that misses the so called QoS transparency.


I imagine you have set a span session to capture the traffic you send to the provider.

by setting as a trunk the destination port you should be able to see your own QoS marking in 802.1Q header


Hope to help

Giuseppe


JOHN APONTE Tue, 11/10/2009 - 11:28
User Badges:

We were wondering how would we see the .1q header also. We have spanned everything we are sending to the 3550 but cannot see the .1q header. How can we set a span to see the layer 2 header markings? To be more clear...we have a span session on 6509a port 6/6, the port to 3550, and our monitor port is on the same box on port 5/1...distributed omnipeek capture svr. When we span from 6/6 to 5/1 we see the marked packets egressing 6/6 and heading towards the 3550 but do not see the .1q header info in the packet. How can we see true layer 2 trunk header stuff in the packet? Thanks!!

Giuseppe Larosa Tue, 11/10/2009 - 11:40
User Badges:
  • Super Silver, 17500 points or more
  • Hall of Fame,

    Founding Member

Hello John,

the destination port of the local SPAN session has to be configured to trunk


int giga x/y

switchport

switchport enc dot1q

swichport mode trunk



see

>> SPAN copies Layer 2 Ethernet frames, but SPAN does not copy source trunk port ISL or 802.1Q tags. You can configure destination ports as trunks to send locally tagged traffic to the traffic analyzer.


http://www.cisco.com/en/US/docs/switches/lan/catalyst6500/ios/12.2SXF/native/configuration/guide/span.html#wp1059824


Hope to help

Giuseppe


JOHN APONTE Tue, 11/10/2009 - 11:44
User Badges:

The 6509a is a CATOS box. So if I change from monitor port to trunk how does the traffic that I want to mirror get to that port? I should have mention earlier that both 6509a and 6509b are running catos software...a is running 8.5 and b is running 7.6 Is there commands to span the trunks to view the .1q headers ?

JOHN APONTE Tue, 11/10/2009 - 12:46
User Badges:

Giuseppe, here is another explanation about our QoS stripping problem...


Several weeks ago we noticed that one of our Carriers does not pass our L3 DSCP markings across the link between our campuses. The link looks like this:


DS104B<--->Carriers DeSoto 3550 premise switch <------->Carriers Canoga 3550 premise switch<---> CS01A

Sup720 gig6/8 L2TP Tunnel Sup2 Gig 6/6

.1Q trunk Routed VLAN 10 .1Q trunk Routed VLAN 10


This .1Q trunk actually forms across the carrier's network, but we only allow VLAN 10, which is the routed VLAN.

What we're seeing is when span and packet capture the traffic leaving ds104b gig6/8 (DeSoto end), we see the DSCP markings. However, when we span and packet capture traffic on cs01a (Canoga end), we do not see them. The reverse is also true. On the DeSoto end the carrier's port we attach two is gi0/2: Here's his config:


interface GigabitEthernet0/2

switchport access vlan XXX

switchport mode dot1q-tunnel

l2protocol-tunnel drop-threshold cdp 1024

l2protocol-tunnel drop-threshold stp 1024

l2protocol-tunnel drop-threshold vtp 1024

l2protocol-tunnel cdp

l2protocol-tunnel stp

l2protocol-tunnel vtp

no cdp enable

spanning-tree bpdufilter enable


They claim that with this setup, our VLAN 10 leaving DeSoto should be fully encapsulated leaving the carriers switch, and de-encapsulated at the Canoga end, with a VLAN 10 frame, so the DSCP markings should be untouched. But, it's not working. We have set our switchports talking to the carrier's access switch as simple access ports with VLAN 10, but no effect. The carrier took off the Q in Q, to no effect.


The carrier's technician used a packet generator connected to his DeSoto access switch another port, shaped it as a UDP packet with DSCP 46, wrapped it in a VLAN 10 tag, and we were able to see it at the far end at Canoga with the packet capture). Interestingly enough we were also able to see it on the packet capture on the DeSoto trunk port connected the carrier's access switch. This happened with either L2TP or access port only config'ed on the carrier's access switch.


Other things to note: we are trusting DSCP markings inbound on both the DeSoto port 6/8 and the Canoga port 6/6. Also, we have a WAN connection between the two campuses where we do the same routed VLAN thing, but our trunks are isl. Also, we don't think that link is Q in Q.


Giuseppe Larosa Wed, 11/11/2009 - 09:28
User Badges:
  • Super Silver, 17500 points or more
  • Hall of Fame,

    Founding Member

Hello John,


>> This .1Q trunk actually forms across the carrier's network, but we only allow VLAN 10, which is the routed VLAN.


what is important is that vlan10 hasn't to be the native vlan in your setup so even if it is the only one permitted it should be tagged.


for a working QinQ scenario your switch ports have to be trunk ports unconditionally and vlan 10 has to be not the native vlan.


your config should look like:


int gi x/y

switchport

switchport trunk enc dot1q

switchport mode trunk

switchport trunk allowed vlan 10


by the way a tunnel port should have STP bpdufilter automatically enabled.


check with them what vlan is XXX it should be your customer vlan-id to be used by you only



Edit:

I had missed the message about the fact that your C6509 are CatOS


your ports should be configured as

set vlan YY m/n

set trunk m/n on dot1q

set trunk m/n 10


note: if I remember correctly the native vlan on catos is that defined by YY so take care to have YY <> 10.


about span destination I will look at catos config guide.


Hope to help

Giuseppe


JOHN APONTE Wed, 11/11/2009 - 10:29
User Badges:

Here is how the trunk ports are configured on the 6509a and 6509b switches. Should the native vlan be changed from 10 to 1 ?

sh trunk 6/6


Port Mode Encapsulation Status Native vlan

-------- ----------- ------------- ------------ -----------

6/6 on dot1q trunking 10


Port Vlans allowed on trunk

-------- ---------------------------------------------------------------------

6/6 10


Port Vlans allowed and active in management domain

-------- ---------------------------------------------------------------------

6/6 10


Port Vlans in spanning tree forwarding state and not pruned

-------- ---------------------------------------------------------------------

6/6 10

Giuseppe Larosa Wed, 11/11/2009 - 23:49
User Badges:
  • Super Silver, 17500 points or more
  • Hall of Fame,

    Founding Member

Hello John,

this is a key point to send frames with an 802.1Q header you need to change the native vlan of the port.

CatOS misses a specific command for setting the native vlan.

native vlan is equal to the vlan the port would be if it wouldn't be a trunk.

you can change it with:

set vlan YY 6/6


with YY<> 10



Hope to help

Giuseppe


JOHN APONTE Thu, 11/12/2009 - 08:06
User Badges:

The native vlan for both trunk ports on 6509a and 6509b are both set for vlan 10.

Giuseppe Larosa Thu, 11/12/2009 - 08:10
User Badges:
  • Super Silver, 17500 points or more
  • Hall of Fame,

    Founding Member

hello John,

so you are sending untagged frames


this can be the root cause of the problem if we are speaking of CoS marking it can be passed only if an 802.1Q header is present in the frame.


I would suggest to change it if the objective is L2 marking.


Edit:

reviewing the thread I see you are looking for L3 marking preservation on the path.


So I'm afraid this is not the root cause but the provider switch can only trust CoS in my humble opinion I would make a try.


However, it is strange to setup 802.1 QinQ and then sending untagged frames over it.


Hope to help

Giuseppe



JOHN APONTE Thu, 11/12/2009 - 08:16
User Badges:

The trunk ports are set to trust dscp markings and not COS because the voip svrs are sending out a dscp 46 marking.

JOHN APONTE Thu, 11/12/2009 - 08:22
User Badges:

I guess I dont understand. I thought I am sending a tagged framed over the trunk..vlan 10? Is there a way to view the tagged raw frames to see the 802.1q header?

Correct Answer
Giuseppe Larosa Thu, 11/12/2009 - 09:02
User Badges:
  • Super Silver, 17500 points or more
  • Hall of Fame,

    Founding Member

Hello John,


>> The trunk ports are set to trust dscp markings and not COS because the voip svrs are sending out a dscp 46 marking.


>> I guess I dont understand. I thought I am sending a tagged framed over the trunk..vlan 10? Is there a way to view the tagged raw frames to see the 802.1q header?


native vlan in 802.1Q is sent untagged unless you have an explicit option to send them tagged


this can be in part the cause of your problem as the provider switch expects to receive tagged frames to add the outer customer vlan-id (XXX) and to send the frame with double 802.1Q header to the other side.


Hope to help

Giuseppe



JOHN APONTE Thu, 11/12/2009 - 15:03
User Badges:

Changed the native vlan from 10 to 1 and it works!! We can see the markings on the other switch. Thanks for all your help!!!

Giuseppe Larosa Thu, 11/12/2009 - 23:48
User Badges:
  • Super Silver, 17500 points or more
  • Hall of Fame,

    Founding Member

Hello John,

I'm happy to hear that it worked.


you have been kind to provide feedback this helps to improve the forums making this a complete story.


Best Regards

Giuseppe



Actions

This Discussion