SG300-20 Vlan Setup for ESXi server and workstations

Answered Question
Nov 1st, 2013
User Badges:

Hi,


I'm fairly new to networking so please be gentle.  I'm trying to setup a number of vlans for my home lab.


I've recently moved jobs and taken an Oracle Apps & Middleware role so need to start picking up Apache, EBusiness Suite, Load Balancers etc so need to segragate my network to allow the various configurations I want to setup in my ESXi lab.


My setup is detailed below:-


I have a dryatek 2860n router which is my gateway to the internet setup on IP 192.168.1.1


My Cisco switch has been configured on 192.168.1.2 and setup to use Layer 3.


I have a number of PCs connected to my switch which I want to use to administer my ESXi server and have access to the various VLANS.


The VLANS I require are as follows


VLAN 1192.168.1.x/24
Default / Internet Uplink






VLAN 1210.0.12.x/24
Workstations
VLAN 1310.0.13.x/24
Server MGMT Interface
VLAN 1410.0.14.x/24
Server Public Interface
VLAN 1510.0.15.x/24
Server Private Interface
VLAN 2010.0.20.x/24
Storage



My esxi server has two network interfaces one which will have the MGMT, Public and Private traffic configured as virtual interfaces in ESXi and one to run my Storage traffic/nfs mounts to a QNAP NAS I wish to run on my network


These are how I have the ports


Port VLAN Membership




g1VLAN1

g13 -20VLAN 12



Needs access to VLAN 1, 13, 14, 15, 20

g9


VLAN 13, 14, 15


g10VLAN 20

g7 - 8 VLAN 20
LAG Configured for QNAP NAS




g13-20 are my workstations that need to be on VLAN 12, but also need to be able to connect to 13, 14, 15, 20 over SSH, RDP, NFS


g9 is the ESXi MGMT Interface that need to have traffic from VLANS 13, 14, 15


g10 is the ESXi Storage Interface that needs access to VLAN 20 only


g7/g8 are connect to the QNAP this ideally I want to setup as LAG.   When I get more interfaces in my ESXi server I will eventually team them to match.


I have configured a ip interface in my CISCO switch to 10.0.12.1 as gateway for my workstations and created a static route in my router to allow traffic back to the switch.  This doesnt quite work as of yet.


I have also setup a default route to 0.0.0.0



I've followed a number of guides but struggling to get my head round the concepts and how to achieve the above configuration.


Ideally I want to configure this through the CLi as Ive had no end of issues with the web interface of the Cisco switch.


I believe g9 need to be TRUNK, and the rest ACCESS is that correct.


How do the workstatations access the other VLANS??? 



Any help would be appreciated


Thanks


Paul

Correct Answer by Tom Watts about 3 years 9 months ago

Hi Paul, to break this down a bit.


Host A is connecting to port 13.

config t

int gi0/13

switchport mode access

switchport access vlan 12


ESXI connects to port 9

config t

int gi0/9

switchport mode trunk

switchport trunk allowed vlan add 13-15  (keep in mind that vlan 1 is untagged here and is the IP interface for your server)


This translates to


ESXI = 192.168.1.x /24 gateway 192.168.1.2

interface vlan 1

ip address 192.168.1.2 255.255.255.0

no ip address dhcp



Host A = 10.0.12.x /24 10.0.12.1

interface vlan 12

name Workstations

ip address 10.0.12.1 255.255.255.0



With this basic configuration Host A communicates to ESXI (no other config on the switch)


Please try to get the basic connectivity first then can work on routes and DHCP.


-Tom
Please mark answered for helpful posts

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (1 ratings)
Loading.
Tom Watts Fri, 11/01/2013 - 14:02
User Badges:
  • Green, 3000 points or more

Hi Paul, the switch needs to be in layer 3 mode and each VLAN should have an IP address assigned to it to achieve the intervlan communication.



-Tom
Please mark answered for helpful posts

Paul Robinson Sat, 11/02/2013 - 04:57
User Badges:

Hi Tom,


I've enabled Layer 3 and this is the configuration I have so far.


prcswitch01#show run

config-file-header

prcswitch01

v1.2.7.76 / R750_NIK_1_2_584_002

CLI v1.0

file SSD indicator encrypted

@

ssd-control-start

ssd config

ssd file passphrase control unrestricted

no ssd file integrity control

ssd-control-end cb0a3fdb1f3a1af4e4430033719968c0

!

port jumbo-frame

vlan database

vlan 12-15,20

exit

voice vlan oui-table add 0001e3 Siemens_AG_phone________

voice vlan oui-table add 00036b Cisco_phone_____________

voice vlan oui-table add 00096e Avaya___________________

voice vlan oui-table add 000fe2 H3C_Aolynk______________

voice vlan oui-table add 0060b9 Philips_and_NEC_AG_phone

voice vlan oui-table add 00d01e Pingtel_phone___________

voice vlan oui-table add 00e075 Polycom/Veritel_phone___

voice vlan oui-table add 00e0bb 3Com_phone______________

bonjour interface range vlan 1

hostname prcswitch01

username cisco password encrypted 98b7744a0a178b0acf1860e2a20a507d4a172033 privilege 15

ip ssh server

!

interface vlan 1

ip address 192.168.1.2 255.255.255.0

no ip address dhcp

!

interface vlan 12

name Workstations

ip address 10.0.12.1 255.255.255.0

!

interface vlan 13

name Management

ip address 10.0.13.1 255.255.255.0

!

interface vlan 14

name Public

ip address 10.0.14.1 255.255.255.0

!

interface vlan 15

name Private

ip address 10.0.15.1 255.255.255.0

!

interface vlan 20

name Storage

ip address 10.0.20.1 255.255.255.0

!

interface gigabitethernet2

switchport mode access

!

interface gigabitethernet3

switchport mode access

switchport access vlan 12

!

interface gigabitethernet4

switchport mode access

switchport access vlan 12

!

interface gigabitethernet5

switchport mode access

switchport access vlan 12

!

interface gigabitethernet6

switchport mode access

switchport access vlan 12

!

interface gigabitethernet7

switchport mode access

switchport access vlan 12

!

interface gigabitethernet8

switchport mode access

switchport access vlan 12

!

interface gigabitethernet9

switchport trunk allowed vlan add 13-15

!

interface gigabitethernet10

switchport mode access

switchport access vlan 12

!

interface gigabitethernet11

switchport mode access

switchport access vlan 12

!

interface gigabitethernet12

switchport mode access

switchport access vlan 12

!

interface gigabitethernet14

switchport mode access

!

interface gigabitethernet15

switchport mode access

!

interface gigabitethernet16

switchport mode access

!

interface gigabitethernet17

switchport mode access

!

interface gigabitethernet18

switchport mode access

!

interface gigabitethernet19

switchport mode access

!

interface gigabitethernet20

switchport mode access

!

ip route 0.0.0.0 0.0.0.0 192.168.1.1



Not sure if this is correct or not.  From my understanding g9 needs to be Trunk as it has multiple VLAN traffic.  The workstation VLANs just need to be ACCESS is that correct?


I also confused over the whole TAGGED and UNTAGGED.  Reading a few articles I believe they should all be UNTAGGED.  With possibly the exception of the interface connected to my ESXi which can be TAGGED as ESXi attached VLAN IDs to packets???

Paul Robinson Sat, 11/02/2013 - 05:07
User Badges:

Ive not added all VLANs yet, want to get the workstations talking to ESXi server first and will then add in VLAN 20


Want to setup a LAG for my NAS, need to get my head around the basics first


Here is my routing table on the router


Key: C - connected, S - static, R - RIP, * - default, ~ - private

*            0.0.0.0/ 0.0.0.0          via 217.32.143.32     WAN1

S~         10.0.12.0/ 255.255.255.0    via 192.168.1.2       LAN1

C~       192.168.1.0/ 255.255.255.0    directly connected    LAN1

S    109.151.244.121/ 255.255.255.255  via 109.151.244.121   WAN1

*      217.32.143.32/ 255.255.255.255  via 217.32.143.32     WAN1

Tom Watts Sat, 11/02/2013 - 05:09
User Badges:
  • Green, 3000 points or more

Hi Paul, one part in the configuration I notice is Jumbo Frames have been enabled. I also notice a default IP route, I'm not sure that is needed - yet.


Anyway... assuming the ESXI server has the VLANs assigned to it's network card and it is 802.1q tagging, then yes, you are correct.


When a packet is untagged it will not contain a VLAN ID within the packet, instead the switch will forward it based off the bridge entry associating the VLAN with the MAC address. When there is a tagged packet, there is an additional 4 bytes added to the packet which will show the 802.1q within the packet.


So the thinking is correct. Next, you need to ensure the default gateway of the connected clients is specified correctly. If you're connecting a computer to VLAN 12 then your gateway needs to be set 10.0.12.1  with mask 255.255.255.0 and of course an IP address in the same subnet.


I am assuming your ESXI server is connecting to port 9. Your ESXI would need to be set up on the IP gateway

192.168.1.2  with mask 255.255.255.0 and of course an IP address in this subnet.



-Tom
Please mark answered for helpful posts

Paul Robinson Sat, 11/02/2013 - 06:01
User Badges:

Hi Tom,


Thanks for the quick reply.


I was hoping to use Jumbo frames as the nics on my ESXi server, Qnap NAS and workstations all support it.  I was under the impression this should give a little bit more performance.  Suppose the only way is to test with it off and on and put a load across my network from ESXi server to NAS.



I have one workstation setup using


IP 10.0.12.10,

Subnet 255.255.255.0

Gateway 10.0.12.1


I am able to ping the router but not get internet as of yet.  Not sure why that doesnt work as have added the reverse route.  I'm not sure if this is DNS releated as im pretty sure i was able to ping googles DNS 8.8.8.8.  Will give it another try later.


At the moment I only have the management interface setup on my ESXi with the following details


IP 10.0.13.90

Subnet 255.255.255.0

Gateway: 10.0.13.1


Does the gateway need to be changed to the CISCO IP 192.168.1.2 in order for the workstations to ping/connect to it?


I've done a lot of the configuration so far through the GUI, but ideally would like to be able to configure it through the CLI so could do with a hand with the commands I need to use.  In case I ever reset my router.  I would like to configure it from a fresh via the CLI


Thanks


Paul

Tom Watts Sat, 11/02/2013 - 06:35
User Badges:
  • Green, 3000 points or more

Hi Paul, whatever the native VLAN is for the ESXI server is, needs to be the default gateway of the ESXI server.

So I am assuming right now the ESXI is connecting to your port 9 because it is the one with tagged VLANs, If that's the case (and please correct me, or let me know what IP you want the ESXI server to have) then in that scenario the ESXI server gateway would be the untagged VLAN in that case being VLAN 1 or 192.168.1.2.


The host connections need to have a default gateway of the VLAN they are a member of.

So if your host connection is in VLAN 12 your gateway is 10.0.12.1.


What kind of router are you using?



-Tom
Please mark answered for helpful posts

Paul Robinson Sat, 11/02/2013 - 06:50
User Badges:

Tom,


Yes Port 9 is what the ESXi server is connected to


I just checked it and its on


IP: 10.0.13.10

SUB: 255.255.255.0

GW: 10.0.13.1


I'm able to ping it from my workstation so thats a good sign and have setup SSH which im able to connect to.  The server doesnt require internet access. 


Port 9 is setup as follows 


GE9Trunk1UP, 13T, 14T, 15T1UP, 13T, 14T, 15T


My workstation is now


IP: 10.0.12.10

SUB: 255.255.255.0

GW: 10.0.12.1


I had the DNS set to 192.168.1.1, but have changed it to use googles DNS 8.8.8.8 and can now get out to the outside world.  I'm not quite sure what I should be using for DNS.  Should it not pick this up from the router or do I have to manually put in a DNS ... for example BT's DNS which is my ISP


I'm going to setup my other ESXi VLANs and see if i can ping them.  Hopefully they will be all ok once ive configured my vnics.


Paul

Paul Robinson Sat, 11/02/2013 - 06:52
User Badges:

Sorry forgot to mention my router is a Draytek 2860n.


Sorry another question ... is it possible to configure DHCP for the workstation VLAN.


Do I set my router to give out IP's on the 10.0.12.X range, then configure relay on the switch (Not sure what to do with this either)

Tom Watts Sat, 11/02/2013 - 08:13
User Badges:
  • Green, 3000 points or more

Hi Paul, your switch is currently running 1.2.7.76 software. Upgrade to the 1.3.0.62 which will add DHCP capability on the switch.


The Draytek router will need to support a static route pointing back to the switch SVI because it will not know about any additional subnet except the one it directly connects to (unless you can somehow have the route table populate).


-Tom
Please mark answered for helpful posts

Paul Robinson Sat, 11/02/2013 - 08:44
User Badges:

Tom,


I upgraded to 1.3.0.62 but had issues with stability.


Whenever I tried to add an IP4 Interface it would hang my switch, the only way I could get on was to reset the router.


Reverting back to the old firmware I had no issues adding the Interfaces so something is not quite right.


Is it worth me raising another support question?


Thanks


Paul

Tom Watts Sat, 11/02/2013 - 09:20
User Badges:
  • Green, 3000 points or more

Paul, I think the IPv4 interface getting hung may have had something to do with sequence of events when configuring.

There is not any bug for configuring IPv4 interface on any of the releases but a common caveat being the default VLAN must have a static IP assigned first before assigning any additional VLAN an IP address.



-Tom
Please mark answered for helpful posts

Paul Robinson Sat, 11/02/2013 - 09:55
User Badges:

Tom,


Ive switched to the latest firmware but when trying to enable DHCP server on VLAN 12 interface it hangs.  Here's my running config.


I have a static IP set for the switch but a bit confused as to why the default VLAN 1 says NO IP ADDRESS DHCP


prcswitch01#show run

config-file-header

prcswitch01

v1.3.0.62 / R750_NIK_1_3_647_260

CLI v1.0

set system mode router



file SSD indicator encrypted

@

ssd-control-start

ssd config

ssd file passphrase control unrestricted

no ssd file integrity control

ssd-control-end cb0a3fdb1f3a1af4e4430033719968c0

!

port jumbo-frame

vlan database

vlan 12-15,20

exit

voice vlan oui-table add 0001e3 Siemens_AG_phone________

voice vlan oui-table add 00036b Cisco_phone_____________

voice vlan oui-table add 00096e Avaya___________________

voice vlan oui-table add 000fe2 H3C_Aolynk______________

voice vlan oui-table add 0060b9 Philips_and_NEC_AG_phone

voice vlan oui-table add 00d01e Pingtel_phone___________

voice vlan oui-table add 00e075 Polycom/Veritel_phone___

voice vlan oui-table add 00e0bb 3Com_phone______________

bonjour interface range vlan 1

hostname prcswitch01

username cisco password encrypted 98b7744a0a178b0acf1860e2a20a507d4a172033 privilege 15

ip ssh server

!

interface vlan 1

ip address 192.168.1.2 255.255.255.0

no ip address dhcp

!

interface vlan 12

name Workstations

ip address 10.0.12.1 255.255.255.0

!

interface vlan 13

name Management

ip address 10.0.13.1 255.255.255.0

!

interface vlan 14

name Public

ip address 10.0.14.1 255.255.255.0

!

interface vlan 15

name Private

ip address 10.0.15.1 255.255.255.0

!

interface vlan 20

name Storage

ip address 10.0.20.1 255.255.255.0

!

interface gigabitethernet2

switchport mode access

!

interface gigabitethernet3

switchport mode access

switchport access vlan 12

!

interface gigabitethernet4

switchport mode access

switchport access vlan 12

!

interface gigabitethernet5

switchport mode access

switchport access vlan 12

!

interface gigabitethernet6

switchport mode access

switchport access vlan 12

!

interface gigabitethernet7

switchport mode access

switchport access vlan 20

!

interface gigabitethernet8

switchport mode access

switchport access vlan 20

!

interface gigabitethernet9

switchport trunk allowed vlan add 13-15

!

interface gigabitethernet10

switchport mode access

switchport access vlan 12

!

interface gigabitethernet11

switchport mode access

switchport access vlan 12

!

interface gigabitethernet12

switchport mode access

switchport access vlan 12

!

interface gigabitethernet14

switchport mode access

!

interface gigabitethernet15

switchport mode access

!

interface gigabitethernet16

switchport mode access

!

interface gigabitethernet17

switchport mode access

!

interface gigabitethernet18

switchport mode access

!

interface gigabitethernet19

switchport mode access

!

interface gigabitethernet20

switchport mode access

!

exit

ip default-gateway 192.168.1.1


Paul

Paul Robinson Sat, 11/02/2013 - 13:18
User Badges:

Thinking about it NO IP ADDRESS DHCP is probably correct ... switch is just saying it has no ip assigned by dhcp


Anyhow, it seems that it is hanging when working with interfaces, any ideas?


Wonder if its worth try through the cli?  Although not sure what commands are required.



Have found another support note on the DNS issue I was having.  The switch doesnt forward DNS requests, I was setting the DNS as the router but the switch wont pass these requests onto the clients from the look of it.


I'll get the DNS address for BT from my router and apply that to my workstations once I have got DHCP working

Paul Robinson Sun, 11/03/2013 - 03:31
User Badges:

Tom,


I reflashed my switch and all seems ok now.


Have configured DHCP which is working fine, have used googles DNS server for all my clients.


Just have my other VLANS to configure hopefully they will be easy to setup.


Thanks for all your help


Paul

Correct Answer
Tom Watts Sun, 11/03/2013 - 03:39
User Badges:
  • Green, 3000 points or more

Hi Paul, to break this down a bit.


Host A is connecting to port 13.

config t

int gi0/13

switchport mode access

switchport access vlan 12


ESXI connects to port 9

config t

int gi0/9

switchport mode trunk

switchport trunk allowed vlan add 13-15  (keep in mind that vlan 1 is untagged here and is the IP interface for your server)


This translates to


ESXI = 192.168.1.x /24 gateway 192.168.1.2

interface vlan 1

ip address 192.168.1.2 255.255.255.0

no ip address dhcp



Host A = 10.0.12.x /24 10.0.12.1

interface vlan 12

name Workstations

ip address 10.0.12.1 255.255.255.0



With this basic configuration Host A communicates to ESXI (no other config on the switch)


Please try to get the basic connectivity first then can work on routes and DHCP.


-Tom
Please mark answered for helpful posts

Paul Robinson Mon, 11/04/2013 - 05:16
User Badges:

Hi Tom,


That all seems to be working now, I can connect to my ESXi server that is on 10.0.13.10 from my workstations on VLAN 12.


I am just configuring my storage VLAN.20


The only issue is, ESXi only allows the use of one default gateway associated to the management interface for outgoing traffic ... this is set to 10.0.13.1 which is ok for VLANs 13,14,15 vnics using the trunk on port 9 but not the other interface I have connected to VLAN 20 which I want to use for storage traffic.


When I configure my second vswitch which is attached to another physical nic on port 10, I am unable to ping my nas


vswitch 0       vmk0     IP 10.0.13.10      ----> physical nic on port 9

                                 GW 10.0.13.1


vswitch 1      vmk1     IP 10.0.20.10      -----> physical nic on port 10

                                Uses Default GW as above


Port 10 has been configured as VLAN 20, Access


With ESXi forcing traffic to go through a single gateway, do I need to make gi10 TRUNK (VLANS 1, 13, 20) to allow traffic to flow. 


I was hoping I could allocate a seperate gateway to each virtual device but that doesnt seem the case.


Thanks


Paul

Paul Robinson Mon, 11/04/2013 - 09:32
User Badges:

Tom,


I've amended the gi10 to be TRUNK and allow VLANS 13,20  and can now reach my NAS.


ESXi will send any traffic for VLAN 20 through the second nic which is what I want.


Going to try and configure the NAS using LAG a little later.


DHCP is now also working on VLAN 12.


I'll mark this question as answered and open another question if I have any issues with configuring LAG.


I however have found some notes on another site which seem easy enough to follow.



Thanks for all your help


Paul

Actions

This Discussion