Another 6500/7600 QoS/DSCP/CoS Queueing Question

[bshellrude]

OK... a little confused, think I know what needs to happen, and what's going on, but there is some real un-certainty I'm hoping people can help out with.  Here's the basic setup:

A---|6500|--10G--|7604|---10G----|7604|----10G----|6500|----B

You get the point.  Traffic traversing from A -> B or vica versa.

All links through the core are L3/Routed, not L2/Vlan/.1q/ISL

Traffic is marked at the edge using an ingress policy map.

Traffic is confirmed via SPAN that it contains both CoS and DSCP/ToS leaving the 6500s in either direction headed for the 7600 core

Traffic is ALSO confirmed via SPAN that upon being *received* from the other side by the 6500, that DSCP is maintained but CoS is gone/0.

Considering that only 6708-10G modules apparently allow dscp values mapped to queues/thresholds, that leaves me with looking to queue on the ingress (for priority VoIP traffic) with cos-to-queue/thresh mapping as well as on the egress with cos-to-queue mappings.  Obviously that's not possible (at least on the ingress) if the 7600's aren't preserving the CoS on egress from the port.

This leaves me wondering, if the 7600's are even queueing the egress traffic based on the supposed internal DSCP-to-CoS mapping that's supposed to happen before the queues/scheduler.  Interfaces are all configured as "trust dscp" right now.  So according to CISCO docs, should be rewriting CoS to 0 on ingress, and using trusted dscp values to determine internal DSCP, which in turn should be used with DSCP-CoS map to appropriately queue on the egress... I'm skeptical this is actually happening... and unfortunately don't really have any way to verify (that I know of) because the show commands on the 6500/7600 are pretty primitive regarding QoS stats...

So, we were re-thinking this and thought maybe the thing to do to solve this is to:

- trust cos instead of dscp

- enable dscp transparency (no dscp rewrite) so it is preserved on the egress side of the switch

And so by doing that it would:

- use CoS for ingress queueing

- use CoS for egress queueing

- And preserve both the original CoS and DSCP/ToS values

Would that be correct?

The other two config options I was thinking about were:

- queueing-only mode

- mpls propagate-cos (though I don't believe this would do what I want, but rather just propagate non-existent MPLS EXP bits)

Any help would be greatly appreciated... I've read so many different docs now my head is swimming

[Jon Marshall · Accepted solution]

Couple of caveats -

1) all of the below applies to pre IOS 15 as i don't have any experience with that so it may be different

2) i haven't used a 7600 but i have used the 6500 a lot but and both share a lot of the linecards and i suspect you are referring to those sort of linecards.

The main issue is that the CoS value is contained within the 802.1q tag added to non native vlans on a trunk link. But your links are L3 so there is no CoS value to preserve.

This creates two problems for you -

1) ingress queueing. On ingress the queues are CoS based which means you need a CoS value to assign the packets into the relevant queues. On the 7600s you are obviously not seeing a CoS value for the reason explained. Now you can use a policy map together with a service policy to classify and mark incoming traffic. But, as far as i am aware, you can only set the IP Precedence or DSCP markings in a policy map on ingress traffic. Some cards like the ES cards for the 7600 do support setting a CoS value but i believe they are the exception rather than the norm.

2) egress queueing. You are right in what you say ie. you can trust the incoming DSCP/IPP value and then, assuming the linecard does not support DSCP based egress queueing, the 7600 can derive a CoS value based on the internal DSCP value and then place it into the correct egress queue.

Again though, without a trunk there is no CoS value being written into the packet.

I completely agree that it can be very difficult to tell exactly what the 6500 is doing in terms of internal marking etc. It is one of the big frustrations with the 6500.

Hope some of that has helped.

Edit - the only way you could trust CoS on ingress as far as i can see is to make the links trunk links ie. you use a dedicated vlan for each interconnect and only allow that vlan on the link. Then you simply transfer the IP addresses assigned to the physical ports to the SVI for the new vlan on each switch. You would have to make sure that the vlan you allowed across the link was not the native vlan because you need a tag to be added.

Jon

[bshellrude]

Thanks Jon, that confirms a number of things for me....

I'm still confused a little bit about one thing.  Why when I SPAN outbound on one of the 6500 interfaces towards the 7600's do I see CoS Markings?  Is that because span is capturing the packets before the .1q tag is stripped on egress from the port?

[Jon Marshall]

I'm still confused a little bit about one thing.  Why when I SPAN outbound on one of the 6500 interfaces towards the 7600's do I see CoS Markings?  Is that because span is capturing the packets before the .1q tag is stripped on egress from the port?

Don't know to be honest. Are the L3 links actually configured with the IP address on the physical interface ?

Jon

[bshellrude]

They are true L3 routed ports yes... 

[Jon Marshall]

So the packets coming into the 6500 are marked with CoS ie. the packets come in on a trunk link ?

If so i cannot see how when the 6500 sends it on to the 7600 that there would be a CoS marking because there is no place to put it in the header

Jon

[bshellrude]

Well no not necessarily comin in on a trunk, no... 

But when doing a SPAN capture on the egress interface we do see CoS markings... I guess those must be the internally derived CoS and are being captured before actually being queued and transmitted (which I can only assume would be where the 1q header is stripped)... I guess it's also plausible that we're seeing them on account of the SPAN destination port being set to include encapsulation...???

[Jon Marshall · Accepted solution]

I don't see how the CoS value can be in any packets captured on an egress interface unless they already had them in.

If you are spanning the L3 egress interface then there should not be any CoS markings in the headers.

Jon

[bshellrude]

Well... I would tend to agree....except....

This capture taken with traffic headed to an Anycast RP (which happens to be our 7600s)... and captured on the egress port T6/4 on a 6500...

But the VLAN looked funny to me... remembered that CISCO has reserved VLANs... Quick google gave me "show vlan internal usage" which provided the following, vlan 1020 incidentally being the outbound interface assigned in the output below... so... once again I'm left wondering...

VLAN Usage

---- --------------------

1006 online diag vlan0

1007 online diag vlan1

1008 online diag vlan2

1009 online diag vlan3

1010 online diag vlan4

1011 online diag vlan5

1012 PM vlan process (trunk tagging)

1013 Control Plane Protection

1014 NDE 0

1015 L3 multicast partial shortcuts for VPN 0

1016 Egress internal vlan

1017 Multicast VPN 0 QOS vlan

1018 vrf_0_vlan0

1019 IPv6 Multicast Egress multicast

1020 TenGigabitEthernet6/4

[Jon Marshall]

You have me wondering now as well

I believe that switches like the 6500 use a sort of trick in effect to implement routed ports ie. internally they still use a vlan (one of the internally used vlans) and then just allocate the port into that vlan as far as i know with an equivalent SVI although you can't see it.

But i assumed the above would still mean that if the port was only in one vlan it would not be tagged but your packet clearly is.

Perhaps i am mistaken and the actual port is treated as a trunk port but i can't see the logic of that as it only needs to be in one vlan.

I'll do a bit of digging but thanks for posting that, i learn something new every day.

Jon

[bshellrude]

Ha... I know right....

Incidentally, looks like the 7600 does the same thing...

VLAN Usage

---- --------------------

1006 online diag vlan0

1007 online diag vlan1

1008 online diag vlan2

1009 online diag vlan3

1010 online diag vlan4

1011 online diag vlan5

1012 PM vlan process (trunk tagging)

1013 Control Plane Protection

1014 L3 multicast partial shortcuts for VPN 0

1015 Container0

1016 Egress internal vlan

1017 Multicast VPN 0 QOS vlan

1018 IPv6 Multicast Egress multicast

1019 GigabitEthernet1/2

1020 GigabitEthernet1/1

1021 TenGigabitEthernet2/4

1022 TenGigabitEthernet2/2

1023 TenGigabitEthernet2/3

1025 NDE

1026 TenGigabitEthernet2/1

[Jon Marshall]

I expect the 7600 is no different in that respect.

The confusion for me is that the port should be an access port not a trunk. Like i say, i will have a dig around but i suspect this may be an internal Cisco document kind of thing.

Maybe you could try trusting CoS on ingress on the 7600 and see if you can map different values to different queues.

Jon

[DIEGO ZORRILLA]

Hi guys, are you still working on this.

I´m having some issues with frames going into a 7600 with ToS value, but when going out from the 7600 dont have any more the ToS value.

This is happening when the source of the ip packets is a juniper