Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
cancel

ASK THE EXPERT – INFINIBAND AND CLUSTER COMPUTING

Welcome to the Cisco Networking Professionals Ask the Expert conversation. This is an opportunity to discuss with Cisco expert Nikhil Kelshikar about InfiniBand network the interconnect technology which is being used to build large high performance clusters. Nikhil is a technical marketing engineer with the Server Networking and Virtualization Business Unit at Cisco Systems, Inc. He has worked with the Cisco Server Fabric Switches and InfiniBand Technology. Nikhil has experience working with High Performance Computing Linux Clusters.

Remember to use the rating system to let Nikhil know if you have received an adequate response.

Nikhil might not be able to answer each question due to the volume expected during this event. Our moderators will post many of the unanswered questions in other discussion forums shortly after the event. This event lasts through January 13, 2006. Visit this forum often to view responses to your questions and the questions of other community members.

20 REPLIES
Silver

Re: ASK THE EXPERT – INFINIBAND AND CLUSTER COMPUTING

Hi Nikhil,

What is InfiniBand technology ?

I am not familiar with it at all.

Thanks,

Tom

Community Member

Re: ASK THE EXPERT – INFINIBAND AND CLUSTER COMPUTING

InfiniBand is an interconnect technology. InfiniBand can be used to build high bandwidth, low latency network. InfiniBand offers 10Gbps bandwidth to the host and can offer CPU to CPU communication latencies of less than 5 microseconds. InfiniBand (IB) is a standards based technology. IB is ideally suited to build high-performacne compute clusters.

More details about Cisco InfiniBand products are found here: http://www.cisco.com/en/US/products/ps6418/index.html

Community Member

Re: ASK THE EXPERT – INFINIBAND AND CLUSTER COMPUTING

Hi Nikhil,

Can you tell me if Cisco is planning to integrate Infiniband technology on Catalyst 65XX platfrorms in the future?

Colin.

Community Member

Re: ASK THE EXPERT – INFINIBAND AND CLUSTER COMPUTING

No, There is no pending plans in the roadmap to do so. InfiniBand products are a separate product line - Server Fabric Switches (SFS). The SFS 3000 Series switches offer ethernet gateways for connectivity to a Catalyst 65xx.

Bronze

Re: ASK THE EXPERT – INFINIBAND AND CLUSTER COMPUTING

Hello Nikhil,

Can you tell me the different pieces that are required to build a InfiniBand Cluster?

Thanks,

Bob

Community Member

Re: ASK THE EXPERT – INFINIBAND AND CLUSTER COMPUTING

To build an InfiniBand Linux Cluster one would need to buy InfiniBand Host Channel Adapters (HCAs) which are the adapter cards for the server. One would also need Cisco Server Fabric Switches (SFS), which are InfiniBand switches and InfiniBand cables.

Community Member

Re: ASK THE EXPERT – INFINIBAND AND CLUSTER COMPUTING

What Upper Layer Protocols does InfiniBand Support?

Community Member

Re: ASK THE EXPERT – INFINIBAND AND CLUSTER COMPUTING

InfiniBand supports IP over InfiniBand (IPOIB), SCSI RDMA Protocol (SRP), Sockets Direct Protocol (SDP), Message Passing Interface (MPI), User Direct Access Programming Library (uDAPL). This implies that most application code doesn't need to change to work over InfiniBand.

Community Member

Re: ASK THE EXPERT – INFINIBAND AND CLUSTER COMPUTING

I have a couple of questions about the protocol support (and Cisco's support for them):

1. On the Linux platform, are you using the Linux Infiniband stack built-into some versions of the 2.6 kernel, or do the Cisco HCA's use a proprietary implementation?

2. On both the Linux and Windows platform, what level of SRP support is incorporated into the drivers? For example, I have seen that some drivers support a direct "virtual SCSI miniport" driver which exposes the SRP resources, while others do not.

3. Do the Cisco Infiniband to Fiber gateways support native SRP functionality, or iSCSI functionality?

Community Member

Re: ASK THE EXPERT – INFINIBAND AND CLUSTER COMPUTING

Hello Ivan,

1. On the Linux platform, are you using the Linux Infiniband stack built-into some versions of the 2.6 kernel, or do the Cisco HCA's use a proprietary implementation?

A. Cisco HCAs can use the OpenIB based InfiniBand stack built into the 2.6.14 + Linux Kernel. This is a community supported stack created by the OpenIB consortium. Cisco is a key contributor to this development.

However the current stack shipped by Cisco is based on the Mellanox VAPI 4.1 implementation.

3. Do the Cisco Infiniband to Fiber gateways support native SRP functionality, or iSCSI functionality?

A. The Fiber channel gateways support native SRP not iSCSI.

Community Member

Re: ASK THE EXPERT – INFINIBAND AND CLUSTER COMPUTING

Hi Nikhil,

Can you tell me what are High Performance Compute Clusters?

Thanks,

Kerry

Community Member

Re: ASK THE EXPERT – INFINIBAND AND CLUSTER COMPUTING

Scientific and technical computing requires a lot of compute processing power to solve the complex problems. Traditionally to solve a complex computing problem end-users purchase machines with many processors. However, with the availbility of faster computer networks, users network thousands of commodity servers with a Gigabit Ethernet and/or InfiniBand-based network to build a cluster of computers. This cluster can be used to run a large parallel application.

High Performance Compute Clusters are such compute clusters which are networked together to solve a complex computing task. HPC Clusters are used in Research, Academia, Financial Enterprises, Manufacturing and many more. HPC Clusters are used to solve problems such as Car Crash simulations, airplane wing design, etc.

Silver

Re: ASK THE EXPERT – INFINIBAND AND CLUSTER COMPUTING

Nikhil,

Based on what I understand so far, aren't High Performance Compute Clusters mainly just for big government labs and universities?

Can you give some examples of some type of High Performance Compute Cluster applications?

Thanks,

Tom

Community Member

Re: ASK THE EXPERT – INFINIBAND AND CLUSTER COMPUTING

Depends on what business you are in.

Many applications that were traditionally "supercomputer-based" are being moved into these HPC platforms. For example, I have personally worked on systems in the Bioinformatics space (used by a lot of drug companies) and stress modeling (used by some types of engineering and architecture firms).

Additionally, some of these technologies are starting (just starting) to appear in more traditional enterprise-class applications (for example, Oracle RAC is a good example).

Part of the challenge with Infiniband has always been that it was a niche product, supported only by a bunch of "one-product-wonder" companies. Now that a Cisco is backing the technology, I would expect to see this market take off more.

Now, the REAL question is 10Gig Ethernet vs Infiniband - which is a whole different debate.

59
Views
0
Helpful
20
Replies
CreatePlease to create content