With Matt Blanshard and Jane Gao
Welcome to the Cisco Support Community Ask the Expert conversation. This is an opportunity to ask your toughest layer 2 questions to two of the technical leaders of the San Jose LAN Switching team, Matt Blanshard and Jane Gao. Learn more about Spanning Tree, VTP, Trunking, Resilient Ethernet Protocol, IGMP Snooping, Private VLANS, Q-in-Q Tunneling, QoS, various switching platforms including all desktop switches, Metro Ethernet switches, 4500 and 6500 switches, Blade Center switches, and Nexus 7000 switches.
Matt Blanshard began his Cisco career as an intern in 2007. He is now a technical leader at the Cisco Technical Assistance Center on the LAN Switching team. He holds a bachelor's degree from the University of Phoenix in computer science, and has CCNA certification.
Jane Gao is a technical leader in the Lan Switching Technical Assistance Center (TAC) team in San Jose. She has been working with LAN switching technologies and supporting Cisco switching platforms Jane's Bio since 2009. Ms. Gao was previously a technical leader in the Wireless TAC team in San Jose. Prior to joining Cisco Ms. Gao was working in software development. She has a Master of Science degree in Computer Science from DePaul University in Chicago.
Remember to use the rating system to let Matt and Jane know if you have received an adequate response.
They might not be able to answer each question due to the volume expected during this event. Remember that you can continue the conversation on the Lan Switching and Routing discussion forum shortly after the event. This event lasts through August 12, 2011. Visit this forum often to view responses to your questions and the questions of other community members.
Hello Matt and Jane,
It is a pleasure meeting you.
I have a question about the Catalyst 2960 platform. Over the past years, many features typical for 3550/3560/3750 have been brought to the 2960 series, such as Dynamic ARP Inspection, IP Source Guard, stacking (for 2960-S), static routing, etc. Considering this, I would like to ask:
Thank you very much for any information!
Good to meet you as well! Currently at this time there is plan to backport private vlan functionality to the 2960 switches. For question number two it looks like the VACL funtionality being slipped in is a mistake. VACL is still unsuported on the 2960 switches though it does appear to work, it is an untested feature.
Thank you very much for your answers. And please consider this as my personal expression of a huge interest in having both Private VLANs and VACLs supported on the 2960, should there be any doubt whether there is a customer interest for these features on 2960 Catalysts
There are currently enhancement requests in place for both feature in place. I will forward your interest as well along with this thread to the product team who handles the roadmap for these products and see if we can get them added. I suspect VACL wouldn't be a big deal for them to add as it looks like it already has happened on accident it would just require a commitment to test it.
Private vlan's is entirely a different matter as it introduces functionality that might not be possible with the current hardware. Again I will forward this request on and get it added to the enhancement request.
Thank you very much for passing these suggestions along. I am aware that both functionalities may require a specialized hardware support. In any case, thank you for clarifying this, and I am looking forward to see what functionality can and will be implemented.
Firstly, thanks for hosting this session.
I am just starting to read up on Fabric Path on the Nexus switches. I can appreciate the huge benefits it offers not just in terms of bandwidth but also in the flexibility of being able to extend L2 and have a much larger flat broadcast domain. I have always found L3 in the DC to be a limiting factor in terms of flexibility.
Where i am not clear is how you would intergrate services into this architecture. If you have one very large L2 flat network interconnected using Nexus switches this means you can literally have 1000s of servers L2 adjacent. So how do you firewall, load-balance etc. between the servers.
Take firewalling as an example. It is very common in a DC to segment servers eg. database servers. Now i can see how you could use a firewall in transparent mode but -
1) even your most powerful firewalls pale in comparison to the bandwidth we are talking about with fabric path
2) Nexus switches do not support service modules. So how would you "insert" a firewall between servers without losing the advantages of fabric path.
I may well have misunderstood some of the architecture so please feel free to tell me
I am really glad you asked this question since fabricpath is one of the things I love talking about
First when it comes to segmentation at layer 2 most of the reasons that there is segmentation at layer 2 in a traditional DC design is to limit the size of the l2 domain due to the many undesirable downsides of having a large flat l2 design. Since fabricpath is a way to eliminate these downsides it opens up the possibility to have any application talk to any other application without the need to involve layer 3 routing.
In a situation where you would need firewalling between the segments there is no way currently to firewall inside the fabricpath domain, so you will need to fallback to traditional layer 3 routing between segments if you want to firewall in between. As you pointed out even using a firewall in transparent mode would not be practical due to the huge amount of bandwidth fabricpath provides us with, along with the low latency switching which injecting a firewall into would kill.
As for your second question when it comes to load balancing fabricpath supports equal cost multiple path routing between multiple switches, so it can load balance between switches, up to 16 paths per mac address. If you are talking more like traditional Microsoft NLB and the like it fully supports multicast, but you would need to be sure to use IGMP mode on the servers to ensure that they are sending their IGMP joins so they get the traffic.
Hope this answers your question and I am clear on this post. If not reply on the parts you are unclear on and I will be happy to clarify further!
I am really glad you asked this question since fabricpath is one of the things I love talking about
Matt, you may regret that after my 100th question on it
Seriously though many thanks for the answer but could we go into this a bit more. When i talked about load-balancing i was thinking more server load-balancing ie. ACE module etc. To that list you could also IPS/IDS etc.
One of the key aspects in a DC is not just the sheer bandwidth/latency but services such as the ones mentioned above. Because one of the primary reasons for fabric path, at least as far as i can see, is to allow a much larger flat L2 network this means you can scale up to many 1000s of servers into one vlan in effect. But, speaking from purely personal experience, in all the DCs security is one of the major components. A simple example of a virus spreading for example is much easier on a L2 vlan with no protection.
So i may be missing something here, because to me a L2 domain that large without any sort of IPS/IDS/firewalling seems like a disaster waiting to happen. Yes you could fall back to L3 segregation but then you seem to me to be losing a lot of the advantage that fabric path gives you.
So i guess my follow up questions would be -
1) Do cisco see this as an issue and are there plans to look at firewalling/IPS integration into the Nexus switches. I suspect not because i'm not sure whether IPS or firewalling could ever actually keep up with that bandwidth ie. potentially it could always be the limiting factor
2) Appreciate you may not be able to answer this but what is the take up on fabric path so far and is security seen as an issue by these people
3) If Cisco do see this as an issue and traditonal stateful firewalling/IPS etc. are not the solution are there any other technologies on the horizon that could address some of the issues.
Or if this isn't an issue then please feel free to tell me !
Jon, keep the questions coming
As to address your question about firewalling this is my opinion and not Cisco's official position, but in order for firewalling to keep up we need a major technology leap to keep up with the bandwidth that fabricpath is allowing in the network. I don't know what/if plans are being made to address this, but I do see it as a potential problem. I just don't see in the near future producing a firewall that's capable of keeping up with terabits of traffic.
For the ACE side of things I do think there are ways to achieve load balancing without necessarily putting the traffic through the ACE engine so I do see that technology keeping up to some degree, but by using alternative methods of load balancing (round-robin arp and a few others).
Also to point out is that service modules for the Nexus are being developed, but I do am not privy to any of the performance details so my prediction on the firewall could also be totally wrong
Keep the fabricpath questions coming!
Service modules for the Nexus switches ! - I won't push you for any more information on that one
I agree about the firewalling/bandwidth issue and i suspect this will put more emphasis on end point securiity rather than using the network to secure the devices.
So moving on to my next set of questions !
I'm finding it difficult to find any docs that go into the specifics of how fabric path works. I have an overiew in the sense that it is basically implementing routing within L2 networks. The routing protocol it uses is IS-IS and in effect it uses switch tags or addresses as the source and destination ie. a packet from hostA to hostB would have a header added to the packet with the source address of the switch hostA is attached to and the destination address of the switch hostB is attached to. Is that correct so far ?
My initial thoughts were that IS-IS was used to exchange information about all available mac-addresses to all switches within the fabric domain with switch tags added to "routes" for each mac-address. But from one of the docs i have read on CCO -
Cisco FabricPath needs to learn at the edge of the fabric only a subset of the MAC addresses present in the network, allowing massive scalability of the switched domain.
I'm not sure i understand how this works. Does this imply
1) that the core switches within the fabric path "domain" need to know all mac-addresses present in that L2 network
2) if the edge switches only know a subset then how does an edge switch know which destination switch to use as the destination address in the packet. When an edge switch receives a packet from a device on the edge it presumably does a lookup for the destination mac-address and then assigns a destination switch tag to the header it adds. If it only has a subset of mac-addresses then how does it know how to address the packet if the destination mac-address is not part of this subset ?
Do you have any links that i may have missed to a more detailed explanation of how all this works ?
Once again, many thanks for answering all my questions.
This is going to be a long post
ISIS is used to establish routing adjacencies between switches. What is actually being advertised though is a bit different. Inside fabricpath we use what is called conversational mac learning. Broadcast frames do not trigger mac learning. Inside the core of the fabricpath there is no mac learning at all since all frames will have a destination switch id. The fabricpath core is very similar to a MPLS core where all forwarding being done there is label switching only. First I am going to go over how flooded frames are handled.
Inside the topology there will be two loop-free trees built which are used for multi-destination frames (broadcast, multicast, unknown unicast). One tree is typically used for all broadcast and unknown unicast while multicast is balanced across both trees. When a frame is received with an unknown destination mac it is sent out to the respective tree. All of the switches in this tree will receive this packet but only the switch which has the destination mac learned as a local mac will install the remote source mac of the frame. This means the edge switches will only learn mac addresses for traffic it is interested in instead of from every broadcast frame and greatly reduces the amount of mac learning required.
So a real scenario will look like this topology:
In this scenario even though the frames will be traversing both Sw1 and Sw2 they will never learn the mac of either Host A or Host B. ISIS is mostly used to establish the trees, handle multicast forwarding, and create switch id's to forward the frames to. You can also see that in a true core situation where there are dedicated core switches they will never learn any mac's because they will never have any mac's as local.
Unfortunately all the documentation that I have is internal to Cisco, but I know there are some white papers coming out on this which will explain it further. As you can see though this greatly reduces the overhead on switches and since fabricpath uses ECMP it will load balance traffic from Host A and Host B through sw1 and sw2.
Excellent reply, thanks very much.
I'm going to take a bit of time to absorb it but just to be absolutely clear in my head a few quick questions -
The fabricpath core is very similar to a MPLS core where all forwarding being done there is label switching only
i understand what you mean in the sense that with MPLS the routes outside the MPLS cloud are not known to the P routers in the same way the core switches in fabric path do not need to know the mac-addresses of hosts attached to the fabric path edge.
However in an MPLS network the tags are actually changed at each P router hop as they go through the MPLS network. I'm assuming there is no actual tag changing in fabric path ie. each core switch does not swap a tag, it simply looks at the destination switch tag and forwards it ?
One other thing is not entirely clear. Although fabric path builds a loop free topology using IS-IS which means no STP needed, broadcasts etc.must still be flooded to every switch within the fabric topology. So every switch must still process every broadcast sent. So when you talk of greatly reducing the overhead of switches are you talking purely about the cam tables switches maintain or is there some additional benefit i am not seeing ?
No tag switching inside the core just looks at the destination switch id and forwards it accordingly. As for the reduction it's only in the amount of mac's that have to be learned since there's no way to stop broadcast/unknown unicast without breaking basic network functionality.
Hi Matt & Jane,
You had been assigned as subject matter expert till August 12th in public forum also you are from the TAC. Your intro page start out as questions to raise in this area "Spanning Tree, VTP, Trunking, Resilient Ethernet Protocol, IGMP Snooping, Private VLANS, Q-in-Q Tunneling, QoS." I know Cisco had tons of information in the web but you must have favorite link for your area of expertise to support TAC issue.So, to benefit audience, Please provide URL pointer in a single page for your technology above for the following questions:
1) How is the handshaking protocol for "X"?
2) What you need to configure to work for X?
3) How do you verify with show command and debug to check your protocol X handshake?
4) Recover outagage issue in shorter time for each technology to follow the "Flow Chart" poster.
You may already have this or you have a good two weeks to research and put in one ducument that will help entire audience for any questions to arise. Update often as any NEW features introduce. Your TAC manager will appreciate your hard work as it will cut down tons of un-warranted tickets.
I have two questions.
First one pertains to Security (VACL), and second pertains to QoS.
When configuring a VACL, there is an option to match frames based on LSAP values.
Where in the DocCD can I find what the supported values are?
For example, if I were to allow/disallow spanning-tree, how can I find out what values to insert into the permit statement?
mac access-list extended STP
permit any any lsap 0x???? 0x0
2. Cat QoS
Would you recommend configuring an expedite/priority queue for EF/VoIP, or only reserve bandwidth for it?
In another word, between the follow two options, which one would you recommend, and why:
srr-queue bandwidth shape 4 0 0 0
We normally do the former, w/ priority-queue, but is there any chance of starving out the other three queues?
Should we remove the priority-queue, and only shape queue 1 instead?
When using a VACL with the current architecture of our switches it won't block STP packets because they are reserved packets and ignore ingress ACL's. If you want to block those you will need to configure spanning-tree bpdufilter on the port.
For the qos question, we always recommend configuring the priority-queue out. When you have that combined with a shaper it's a strict priority queue and is shaped to keep from starving the other queue's.
Thanks for the response.
STP was just used as an example.
I may need to match things other than STP.
Do you happen to know where I can find a list of the supported LSAP values that can be used in a VACL?
Also you were probably thinking about the priority queue in an MQC policy on routers when you replied.
Is the recommendation still to configure a priority-queue on a switch?
According to this document, when expedite queue is configured, it supercedes the SRR shape configuration.
“The expedite queue is a priority queue, and it is serviced until empty before the other queues are serviced.”
“If the egress expedite queue is enabled, it overrides the SRR shaped and shared weights for queue 1.”
Hi Matt and Jane,
I have a question about creating a SPAN port on a Cisco 871. I understand the commands needed to accomplish this.
My problem lies with trying to use my WAN port (FastEthernet4) as the source of the SPAN port - the cli won't accept "FastEthernet4". I can use "FastEthernet3" or 2 or 1.....but not 4, which is the WAN interface that connects to my DSL modem.
This is what I type: "monitor session 1 source interrface FastEthernet4" - and like I said, the cli won't accept it.
Am I doing something wrong? Somehow I remember that I had this problem a few years ago, but I cant remember how or if I was able to solve it.
I am using IOS 12.4-11T. My DSL assigns a dynamic IP address, and I use "IP negotiated". I also tried using "Dialer0" as the interface, but it wouldnt work either.
It doesnt make sense to me that I would be unable to use SPAN to monitor the WAN interface...out of all of the interfaces, this seems like one of the most important ones to use SPAN with. Also, my ASA5505 is able to use its WAN interface as a source for SPAN.
Hi Matt and Jane,
What Spanning tree enhancements are in plan to improve convergence times. Currently STP convergence is noticeable when running rapid PVST. Any suggestions or design recommendation to make sub-second STP convergence in Data Centers??
There is a new IEEE draft out called trill which is the next step in STP replacement. Cisco's implementation of that is called fabricpath and uses ISIS and eliminates the need for traditional STP. When using trill/fabricpath reconvergence/recovery from link loss/device crash is very short, in the several hundred millisecond range (200-300 or less).
At this point though not much is being done to enhance regular STP any longer, especially since it's standards based and not much modification can be done to it.
Hi Matt and Jane,
First off thanks for taking your time to answer a few questions. I'll jump right into it. Obviously the catalyst platforms do not support the full functionality of the "show policy-map interface" command and I understand they may never due to various platform limitations. My question concerns whether or not there are more diagnostics in the works to allow us to see policy hits and or at least true marking. As it stands currently I have to use a sniffer or hackish loopback setups with a "transit vlan" and "show mls qos interface statistics" in order to see what the final marking of traffic inbound to a particular port is.
The following setup allows me to troubleshoot. "show mls qos interface 3 statistics" shows me inbound markings coming from the client, "show mls qos interface 1 statistics" shows me actual markings after the policy-map on inteface 3 has been applied.
Port 1 : client vlan 100 (the vlan all of my real, non-test clients are in)
Port 2 : transit vlan 101
Port 3 : transit vlan 101
| 1 | 2 | 3 |
| | |
------- ---------- Client PC
I have more and more customers utilizing catalyst switches exclusively now that metro ethernet has become more common and this is increasingly become a trouble shooting barrier for me in these setups. Am I just missing soemthing, is there another existing way to do this?
You are right that there are limitations on the command "show policy-map interface" for some platforms, it's unsupported on some or supported with certain limitations on others.
Can you please let us know what platform(s) you are mostly interested in? We may have different set of commands for QoS statistics depending on the platforms. But sniffer capture is one of the common tools we use in TAC for troubleshooting when there's any doubt on the commands counter outputs as well.
Hello Jane and Matt,
I have a question about the 2960C switches. We have a need to terminate a single mode fiber from an isp and change that to multi mode to connect to our network. Is it possible to use something like the 2960C with SFP ports to make this switch? I'm thinking we could use the gbic for single mode to connect to the isp and use the gbic for multi mode to connect to our network. Will this work.
I have been told by some people that the SFP Ports on the 2960C are only Uplink Ports and as such would not work for this situation and other people have told me this will work fine. Othe people have told me I would need a 3750 to be able to do this.
In your opinion what would be the best way to handle this. We would like to use something like an 8-port switch to do this, so we could branch off to another router later if needed...
Thanks for your time..
You would be able to use a 2960C for this. Just create a layer 2 vlan and make both ports an access port on the 2960C. Alternatively you could look at using a media convertor to convert between single mode and multi-mode fiber as an alternate solution as well.
Hello Jane and Matt,
I would like to ask about the exact usage of DHCP Relay Option 82 in DHCP Snooping and the exact mechanism of how a DHCP Snooping-enabled switch forwards a server's response to a client.
Originally, I thought that the Option 82 is the only and definitive indicator for a switch where should a server's response be forwarded, as the Option 82 contains both the MAC address and the Port ID of the switch that originally received the client's request. However, after debugging, it turned out that the DHCP Snooping behaves in a more complex way, and here are my observations/questions:
Thank you very much!