Welcome to the Cisco Networking Professionals Ask the Expert conversation. This is an opportunity to explore the value of FCoE and I/O consolidation with Cisco expert Dante MalagrinÃ², Ph.D. Dante heads the Data Center Solutions Marketing team for Cisco in Europe and Emerging Markets where he also leads the Infrastructure and Architecture teams. Dante is a recognized expert in the data center and information technology industry and has been on the Board of Directors for Storage Networking Industry Association (SNIA) Europe. Dante originally joined Cisco in 1999 as a software engineer in the areas of high-speed networking and LAN switching. He participated in the development of the Catalyst 6000 family of LAN switches. Dante joined Andiamo Systems in 2001 as a technical marketing manager responsible for competitive analysis and interoperability testing with storage partners. In 2002, after the acquisition of Andiamo Systems by Cisco, he became product manager for the Cisco MDS 9000 family of intelligent SAN switches.
Remember to use the rating system to let Dante know if you have received an adequate response.
Dante might not be able to answer each question due to the volume expected during this event. Our moderators will post many of the unanswered questions in other discussion forums shortly after the event. This event lasts through May 16, 2008. Visit this forum often to view responses to your questions and the questions of other community members.
Please give a brief description what u r refering to by " Figre channel over ethernet and I/O Consolidation "
it's a long answer to give, but I would recommend beginning from the following links:
I would also highly recommend reading the following book from Silvano Gai, which describes in a fair amount of details what Fibre Channel over Ethernet is and the key ingredients of I/O Consolidation (i.e. the combination of Data Center Ethernet and Fibre Channel over Ethernet.)
I'm interested in when you think host adapters will be available to the "mass market" and when FCoE will simply become an extra tick box when installing a driver/module on your system?
And also, will this make it to the SMB?
Host adapters are already available from a wide variety of companies. You can already purchase a CNA (Converged Network Adapter) from QLogic, Emulex and Intel. FCoE is implemented in such a way that when you install it on your server, it appears as if it was a virtual NIC or a virtual HBA, so that the OS and the apps cannot tell the difference.
Here are a couple of links where you can learn more:
The real question now becomes: how soon will the OSMs (Original Storage Manufacturers) like EMC, Network Appliance, HP and IBM will be supporting this solution?
They are all actively looking into how to support and qualify the technology in their solution offering. I cannot comment on the specifics right now as there are NDA relationships in place, but I can say that we expect fully qualified solutions to be available to the "mass" market by the second half of this calendar year.
Finally as far as SMBs are concerned, I see two paths: for small SMBs, with little to no knowledge about storage management and Fibre Channel, iSCSI represents a probably more viable option especially in terms of pricing. For "larger" SMBs, where maybe there is some installed base of Fibre Channel and there is enough knowledge around storage management to be able to adopt FCoE, I would say that this solution is a perfect fit.
In both cases, I would say that Data Center Ethernet (mostly because of its lossless characteristics) represents a key differentiator. For more color on this, take a look at the blog discussion I had with Marc Farley a couple of weeks ago:
and some of the subsequent discussions.
Im interested on your thoughts/reasoning on placing such an important data transport in a data centre environment (such as storage protocols) on a relative unreliable (or see ungaranteed) transport environment. Ethernet networking is a relatively unreliable transport medium that relise on higher layers of the OSI model to produce a reliable transport. Yes, we use iSCSI today, but this already is accepted as Fibre Channels poorer brother and the risks associated are accepted as fair game.
I would also like to explore further if placing FCoE on data switches is a good idea in high performance enviroments (as obviously this isn't targeted towards to cost concious SAN's such as the iSCSI protocol). Almost all network switches have some form of over subscription and given this, even with QoS enabled is putting SAN traffic into such an environment ideal or an acceptable risk. I also ask this in terms of reliability. To be honest, many networking experts would agree networks are no where near as reliable as they should be, especially when working with Spanning Tree, VLAN's, dot1q trunks, etherchannel, etc. the Networking technologies we use today to build and expand our highly redundant and high perfomance networks are still disruptive to configure. While outages are kept what we accept to be very short in the networking world (maily due to our outage window being approx 45-90secs which relates to the TCP timeout window), in a SAN environment, even lost packets have a high risk of data loss.
Then of course there are the inherint security risks with networks as well.
Is FCoE such a good idea given the risks inherent with networks today, or is there a longer term plan to harmonise all networking technologies such as STP, DOT1q, QoS, LACP, etc so that implementation is negotiated and activated in seamless transition which would bring networking up to the level of reliability we currently enjoy with a Fibre Channel based SAN environment?
you pose a bunch of loaded questions and I am not sure I will be able to shortly answer all of them. Let me try to start and maybe we can go more in depth based on your specific feedback.
First of all, you are right in saying that Ethernet "as is" cannot be used for I/O consolidation. You need to apply some specific enhancements to Ethernet and make it lossless and more reliable. These are the enhancements that we generally refer to as Data Center Ethernet (http://www.cisco.com/go/dce)
Regarding the oversubscription question, it is common practice to build oversubscribed Fibre Channel networks as well as Ethernet networks. Obviously, the design of a Unified Fabric would have to be different than the design of a regular Ethernet network and probably design principles will be similar to FC designs.
Bottom line is that you raise valid points (although I think you are overestimating the reliability gap between Ethernet and Fibre Channel) and the answer to those points is not in the FCoE protocol itself, but in the collection of Data Center Ethernet standards.
FCoE does not have intrinsic limitation of bandwidth as it essentially is a layer 2 encapsulation protocol.
In terms of products, the Nexus 5000, which is the first FCoE compliant network switch supports 10GbE speeds.
From what I've read so far FCoE seems nowhere near ready (because the core standards necessary are just proposals) so when do you think we might actually see a fully working, standardised, implementation of FCoE?
Also, Cisco says that "FCoE is not a requirement for deploying Data Center Ethernet, but rather an option that takes advantage of a fabric that is enabled for Data Center Ethernet." So, why would you deploy DCE if you're not going to use FCoE? - what else does it do that regular ethernet can't?
you raise valid point, but I think that "nowhere near ready" is a bit of a strong statement for a technology supported by a very large partner ecosystem, which includes Intel, QLogic, Emulex, Broadcom, Netxen, Dell, EMC and many more. Also, standards are draft, but that does not mean that they are just "proposals". All of the hardware elements of the protocol have been defined and there is only a small set of software-related capabilities that still needs to be defined. The FCIA has already been demonstrating FCoE in multiple occasions, including the latest SNW in Orlando (http://www.fibrechannel.org/NEWS/fcia080407.html). The expectation is that the full FCoE standard will be ratified by the end of this calendar year or early next one, but all of the shipping hardware availbale today will be compatible with the final version when available. This is very common practice with networking protocols and it's not the first time it happens.
The second part of your question is extremely good and valid. FCoE is the primary (and initial) application for Data Center Ethernet, but not the only one. iSCSI, NAS, Voice, server clustering traffic (i.e. VMotion) are only some examples of other protocols that would benefit of a lossless Ethernet transport such as the one provided by Data Center Ethernet.
Hope it helps.
Thanks for the detailed response - we actually use FC exclusively in our DC's and the idea of a single fabric is appealing, but the work involved in getting there would be staggering. Really, at this point we're just looking at the technology to see if the benefits would justify the massive investment required.
One thing that interests me is that you use voice as benefitting from DCE - is that because you no longer need QoS?
let me clarify that I am not suggesting any short term implementation of voice over DCE. What I wanted to say is that once you have a standard-based lossless transport at level 2, you can use it for other applications than just Fibre Channel. QoS (or bandwidth management) will always be required and, as a matter of fact, DCE provides that. The lossless nature of the transport allows for improved performance and potentially reduced implementation overheads.
I can understand the part about the potentially reduced implementation overheads, but what exactly do you mean by "improved performance"? Has this been quantified yet?
sorry for having been vague in my reply. The performance improvement have not been quantified, yet, but the idea is that TCP operates at higher sustained throughput if the underlying link is lossless. Slow-start and Fast-Retransmit algorithms do not have to kick in if there is no packet loss. Obviously, this requires Data Center Ethernet to handle congestion, which is exactly what happens with BCN an QCN.
This is exactly what I have concerns about as I haven't been able to find any research on this topic. If the end-node gets paused won't the network connection just freeze?
It doesn't seem intuitively obvious to me that a lossless link will have higher throughput using TCP when the mechanism to avoid loss is to stop transmitting.
I probably don't have enough understanding of the new protocols yet - do you have anything which could help me here? Are there any publications which have looked at these issues?
You are right. It is not obvious and I do not have any references to actual testing, yet.
One thing I'd like to clarify though is that congestion is not handled by stopping traffic, but by a congestion management mechanism (defined in IEEE 802.1Qau) that gets the originator of the traffic to slow down. The big difference with TCP and the Windows-based mechanism is that you never have retransmissions (as you never loose packets) and so you maximize the usage of the bandwidth (while guaranteeing fairness and avoiding deadlock).
In terms of reference, I would recommend Silvano Gai's book on Data Center Networks and Fibre Channel over Ethernet as well as watching the TechWiseTV event that we have scheduled for May 29th.
Hope it helps.