Cannot get my 3560G switch to talk to ESX 3.5 servers and the EMC NAS.

Answered Question
Dec 23rd, 2009
User Badges:

I have been trying for the past couple of weeks to get my 3560G switch to talk to a couple of ESX 3.5 servers, it's VC, and the EMC NAS. The goal is to implement 2 Etherchannels on the physical switch and NIC teaming on the vSwitch. One port-channel will have 1 VLAN traversing it, and the other port-channel will have 2 VLANs traversing it. VLANs were set up on the vSwitch to mirror the VLANs I have on the 3560G. I want to (have to) use load-balancing. Also, I have read other forums and websites to get some ideas on how to set this up, and each solution is different from the other. The problem seems to reside mostly in the VLAN configuration on the physical switch IMHO, because if I just make the switch a dummy switch, everything can talk without issue. The following information, I already know, not because it is my requirement, but rather pieces that must be present for the whole configuration to succeed.


All interfaces in an Etherchannel must belong to the same VLAN or be configured as trunks. I take this to mean that for the second port-channel, I have to configure the interfaces as trunks, since I need 2 different VLANs to traverse the link.  -I have tried configuring them as both trunks and access ports, and neither seemed to work.


vSwitch must be configured for "Route based on IP hash."


port-channel load-balance should use the src-dst-ip option.


flowcontrol receive desired should be set


system mtu jumbo 9000 should be set


Here is a snippet of my configuration:






port-channel load-balance src-dst-ip

no file verify auto

spanning-tree mode pvst

spanning-tree extend system-id

!

vlan internal allocation policy ascending

!

interface Port-channel1     ------Port-channel2 is configured similarly to this one.

switchport trunk encap dot1q

switchport trunk native vlan 2

switchport trunk allowed 3     -----Po2 allows vlans 4,5 instead

switchport mode trunk

!

interface Gigabitethernet0/1     ------gi0/1 - 20 are configured similarly to this one.

switchport trunk encap dot1q

switchport trunk native vlan 2

switchport trunk allowed 3     -------gi0/13 - 14 allow vlan 4 and gi0/15 - 20 allow vlan 5

switchport mode trunk

flowcontrol receive desired

channel-group 1 mode on

!

Interface Gigabitethernet0/21 - 28 are shutdown

!

interface Vlan1

no ip address

shutdown

!

interface Vlan2

ip address x.x.1.2 255.255.255.0

!

interface Vlan3 - 5 are configured identically

no ip address

!

ip default-gateway x.x.1.1






This should be enough to give you an idea on where I am and where I need to be. Any information you can provide would be greatly appreciated!

Correct Answer by Reza Sharifi about 7 years 4 months ago

Hi,


Does you config work without using etherchannel?

On the ESX, I think the native vlan is 0, have you define that?

Also, you are using the 3560 as a layer-2 layer-3 device right?


Reza

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (1 ratings)
Loading.
Correct Answer
Reza Sharifi Wed, 12/23/2009 - 15:10
User Badges:
  • Super Bronze, 10000 points or more
  • Cisco Designated VIP,

    2017 LAN

Hi,


Does you config work without using etherchannel?

On the ESX, I think the native vlan is 0, have you define that?

Also, you are using the 3560 as a layer-2 layer-3 device right?


Reza

klambert1218 Wed, 12/23/2009 - 18:25
User Badges:

Hi Reza,


To be perfectly honest, I haven't tried it without Etherchannel because Etherchannel is a must for the system on which it is going to be used. As far as the ESX is concerned, a colleague of mine is responsible for that part of this task, although I'm sure he has it set up the correct way, I will make sure I inquire about that. Currently, the 3560G is functioning as a L2 device. I don't think I need to configure any ports as L3 because inter-VLAN communication is not necessary at this particular moment in time.

klambert1218 Mon, 12/28/2009 - 08:18
User Badges:

The native vlan on the ESX is set to vlan 2 also. Still nothing works.

klambert1218 Wed, 01/20/2010 - 10:16
User Badges:

I have since made changes on the ESX, like Route based on IP hash. I have also made more changes on the Cisco configuration. Some progress has been made, but not everything is as I need it to be.

klambert1218 Wed, 01/20/2010 - 10:21
User Badges:

This question is NOT answered. Still looking for assistance in this matter.

Reza Sharifi Wed, 01/20/2010 - 12:37
User Badges:
  • Super Bronze, 10000 points or more
  • Cisco Designated VIP,

    2017 LAN

Hello Klambert,


Can you post your latest switch config?

Also, can you post diagram showing switch connectivity to VM server?


Reza

klambert1218 Thu, 01/21/2010 - 07:09
User Badges:

This is a snippet of pertinent information for my switch configuration. Also, I have attached a Visio diagram of how the switch connects to the ESX servers and the SAN (please excuse the crudeness).



port-channel load-balance src-dst-ip

no file verify auto

spanning-tree mode pvst

spanning-tree extend system-id

!

vlan internal allocation policy ascending

!

interface Port-channel1     ------Port-channels 2 and3 are configured similarly to this one.

switchport trunk encap dot1q

switchport trunk native vlan 2

switchport trunk allowed 3     -----Po2 allows vlan 4 and Po2 allows 5 instead

switchport mode trunk

switchport nonegotiate

flowcontrol receive desired

!

interface Gigabitethernet0/1     ------gi0/1 - 22 are configured similarly to this one.

switchport trunk encap dot1q

switchport trunk native vlan 2

switchport trunk allowed 3     -------gi0/13 - 16 allow vlan 4 and gi0/17 - 22 allow vlan 5

switchport mode trunk

switchport nonegotiate

flowcontrol receive desired

channel-group 1 mode on

!

Interface Gigabitethernet0/23 - 28 are shutdown

!

interface Vlan1

no ip address

shutdown

!

interface Vlan2

ip address x.x.1.2 255.255.255.0

!

interface Vlan3 - 5 are configured identically

no ip address

!

ip default-gateway x.x.1.1

Attachment: 
aeryilmaz Thu, 05/12/2011 - 13:55
User Badges:

Hi Klambert,


Curious if you ever got your issue resolved. I have a similar setup as you - ESX 3.5 servers, NICs are trunked and port-channeled. These ESX Hosts are currently passing traffic on various VLANs, but... I have added a new VLAN and attempting to connect these ESX hosts to a Storage appliance on that new VLAN. I ran a layer 2 trace and confirmed the path looks good from the ESX access switch and my Storage access switch, but they cannot talk to each other.


I can post more details, but curious what your resolution was?


Thanks in advance!

Actions

This Discussion

Related Content