Quick question from a networking newbie about best practices - if I have a number of 1U rack mounted servers, say 24 or 48, each with multiple Ethernet ports, e.g 4 - RJ45 ports for each of primary data, redundant data, console / management, KVM in this example, would it be best to cable each same port type of all the servers to a different edge switch or leaf, eg all primary data go to switch 1, all redundant data go to switch 2, etc or is it acceptable to take same port types from the servers and 'spread' them to different switches?
For example servers # 1 through 6 primary data are connected to switch 1, servers # 7 through 12 to switch 2, etc. This would ultimately mean that a particular switch could contain more than one Ethernet cable from a particular server, for example might be cabled up with primary data, redundant data, KVM in different RJ45 ports, of course.
Thinking through this, I guess it makes tracing cables in case of faults developing much more complicated, but there might be advantages in cabling lay-out depending on how switches and servers are laid out in the cabinet. All switch ports are addressable, I guess, so there's no reason from a data flow viewpoint? There might be slightly better failsafe properties - if switch 3 was used for all console / management went down then all console / management would fail?
A colleague of mine said that there might be advantages in balancing and network traffic performance. I have not heard of this aspect, and couldn't find any info so far referring to it.
I'd appreciate any comments, or links to info on this.
First, the KVM/management ports (ILO in HP server terms) - we've used a separate network for these "out of band" ports, on separate (cheaper) switches, usually 2950s. Saves expensive core ports. Yeah, if the 2950 dies you have no ILO/management but it won't die at the same time as the main ports/switch (if it does, go check your UPS!!!)
For primary/backup traffic ports, put one on one core switch and the other on the other, for redundancy. Maybe also say PROD PRI/BKP go to SW1/SW2, but TEST PRI/BKP go to SW2/SW1. If you mix this with HSRP for "PROD" server Vlans active on SW1, and "TEST" server Vlans active on SW2, you can balance load AND have failover.
As the previous poster mentioned, the dual NIC on the server should be split between two core switches. IE Server #1 connects to Swtich #1 and Switch #2.
To make tracing out cabling a little easier for troubleshooting and future installs you should create a standard practice.
The way mine is setup is server #1 connects both ports to patch panel A-1 ports 1 and 2 (I always group them). Patch panel A-1 connects to patch panel A-2 (B-1 connects to B-2 etc, patch panel A represents rack A and so forth).
Patch panel A-2 port 1 goes to lets say a 6500 labeled switch #1 port 3/1, and the redundant connection goes to switch #2 port 3/1. So it's an exact mirror, and of course i put descriptions on the 6500's.
If you have the appropriate supervisor modules in a 6500 you can have them load balance between the two switches for you.
Also be careful with setting up the NIC teaming on the servers, you can create a broadcast storm and bring down your core switches pretty fast.
[toc:faq]The ProblemOn traditional switches whenever we have a trunk
interface we use the VLAN tag to demultiplex the VLANs. The switch needs
to determine which MAC Address table to look in for a forwarding
decision. To do this we require the switch to do...
[toc:faq]Introduction:Netdr is a tool available on a RSP720, Sup720 or
Sup32 that allows one to capture packets on the RP or SP inband. The
netdr command can be used to capture both Tx and Rx packets in the
software switching path. This is not a substitut...
IntroductionOSPF, being a link-state protocol, allows for every router
in the network to know of every link and OSPF speaker in the entire
network. From this picture each router independently runs the Shortest
Path First (SPF) algorithm to determine the b...