Inter-VLAN routing w/ multiple devices

Unanswered Question
Sep 13th, 2010

I've got a client here in our datacenter with some custom equipment, and they're asking me to set up Inter-VLAN routing. This would be simple enough if the VLANs were all on a single device, but with this setup [and 5 intervening years since I last did anything like this in a lab env] I believe I am in over my head.


Currently from our public network they have PIX 515E firewall put through to a Cat 2950 put through to a Cat3550. The public IP addresses stop at the PIX and the NAT rules take over to their internal network, the 2950 is a 'dumb' switch with no configuration at all, [see note] and the 3550 currently has 6 VLANs. The endgame right now is to configure inter-VLAN routing between VLANs 1 and 6. The rub is that VLAN1 [their 24/7 heavily used production network] has its default gateway on the PIX, and VLAN 6's gateway resides on the 3550. If both gateways were on the 3550 I could do this with the docs I already have, but with the gateway on the PIX box along with 800+ lines of config I've no clue. If the PIX could handle the routing this would be simplified, but as far as I know this is not the case.


The one thing that we need to do, if in any way possible, is avoid any downtime for VLAN 1. It's hard enough to schedule maintenance downtime for just one of their servers, but taking down their production network would be a logistical nightmare.


note: The client is migrating from their previously flat network on the 2950, to the new configuration on the 3550. Once everything has been switched over to the appropriate VLAN the trunk port on the 3550 will be moved over to the PIX and the 2950 removed.


Any help or suggestions are VERY much appreciated, and please let me know if you need any more detailed informaton about the configuration.

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
darren.g Mon, 09/13/2010 - 18:46

What software level is on the 3550 switch?


If it's got even base-IP, it shoudln't be difficult to configure a VLAN interface (as opposed to a VLAN) for both VLAN1 and VLAN6 intot he 3550, and put a static route into the PIX to route the VLAN6 segment via the VLAN1 IP address.


It's been a while since I played with a 3550, but something like this should work


3550_Switch#conf t

3550_Switch(config)#

3550_Switch(config)#int vlan6

3550_switch(config-if)#description Intra-VLAN routing point from VLAN 1

3550_switch(config-if)#ip address 192.168.6.254 255.255.255.0

3550_switch(config-if)#exit

3550_switch(config)#int vlan1

3550_switch(config-if)#description Intra VLAN routing point from VLAN 6

3550_switch(config-if)#ip address 192.168.1.254 255.255.255.0

3550_switch(config-if)#exit


Then, on your PIX, add a static route for network 192.168.6.0/24 and point it to 192.168.1.254. Depending on what routes exist in the 3550 switch at the moment, you may have to fiddle with routing to get stuff from VLAN 6 to get to the PIX. You'd do that with an "ip route xx.x.x.x y.y.y.y z.z.z.z" type line in the 3550.


So basically, the PIX should say 'OK, VLAN 1 is locally connected, so I just send all traffic for VLAN1 out

the local port, but for VLAN 6 I have to send traffic via IP address 192.168.1.254.


Obviously, you need to add your own IP addresses and what not.


If you don't have an image on the 3550 which has basic IP routing, then you'll most likely have to get into 802.1q trunking between the PIX and the 2950/3550.


Hope I've explained this clearly enough.


Cheers.

fubes2000 Fri, 09/17/2010 - 12:59

Sorry for the delay, I've taken on alot of work for people who have quit, gone on vacation, or been injured all just in the last month.


As far as the software version on the 3550, it is 12.2. Right now the VLANs on the 3550 are:

interface Vlan1
ip address 192.168.1.254 255.255.255.0
!
interface Vlan6
ip address 192.168.14.1 255.255.255.0


The routes on the 3550 look like:

Gateway of last resort is not set

C    192.168.13.0/24 is directly connected, Vlan5
C    192.168.14.0/24 is directly connected, Vlan6
C    192.168.10.0/24 is directly connected, Vlan3
C    192.168.11.0/24 is directly connected, Vlan4
C    192.168.1.0/24 is directly connected, Vlan1

and the routes on the PIX are:

S    0.0.0.0 0.0.0.0 [1/0] via 1.2.3.4, outside
C    192.168.1.0 255.255.255.0 is directly connected, inside
S    192.168.14.0 255.255.255.0 [1/0] via 192.168.1.254, inside
C    208.123.216.0 255.255.252.0 is directly connected, outside


I've also done a bit of testing, plugging in a machine to each VLAN, assigning it an IP from that network, and mucking about with the gateway to see what happens. Some strange things have been happening as seen below. For reference, VLAN1 is 192.168.1.0/24 and VLAN6 is 192.168.14.0/24.

VLAN 1, Gateway 192.168.1.1
     192.168.1.     1     Y
                    254     N
                    44     Y
                    43     Y

               14. X     N
               
VLAN 1, Gateway 192.168.1.254
     192.168.1.     1     Y
                    254     N
                    44     Y
                    43     Y
               14. 1     N
                    11     Y
                    12     Y
                    
VLAN 6, Gateway 192.168.14.1
     192.168.1.     1     Y
                    254     Y
                    44     N
                    43     N
               14. 1     Y
                    11     Y
                    12     Y

VLAN 6, Gateway 192.168.1.1
     192.168.1.     1     Y
                    254     Y
                    44     N
                    43     N
               14. 1     Y
                    11     Y
                    12     Y


One thing that struck me as weird is that even when I couldn't ping a host from inside the same network [ie. 192.168.1.254] the ARP table was still populated for that address.


The two basic goals for me are to:

  1. get 192.168.1.X speaking to 192.168.14.X [done if we change the gateway to 192.168.1.254]
  2. get 192.168.14.X speaking to 192.168.1.X [nope]


Any further help, or pointing in the right direction is much appreciated.

Jon Marshall Sat, 09/18/2010 - 03:06

What version of software is the pix 515E running. If it 6.x or less then you cannot do what you want because the pix is not allowed to route the traffic back out of the same interface it received it on.


If it 7.x or greater then this restriction has been lifted.


Can you confirm what version of code the pix is running ?


Jon

fubes2000 Mon, 09/20/2010 - 08:59

The PIX reports Version 7.0(7), hopefully this is just a matter of  putting in an config setting to allow rerouting packets back out of  the inside interface because, like I said before, I cannot take this  network down even for a few minutes at midnight to change the gateways  around without a fight and at least a week of advance notice.

Jon Marshall Mon, 09/20/2010 - 11:29

fubes2000 wrote:


The PIX reports Version 7.0(7), hopefully this is just a matter of  putting in an config setting to allow rerouting packets back out of  the inside interface because, like I said before, I cannot take this  network down even for a few minutes at midnight to change the gateways  around without a fight and at least a week of advance notice.


Wade


Then you can either do a Darren suggested ie. move vlan 1 default-gateway to switch which in my opinion if you do need to firewall vlan 1 clients from other internal subnets is the way to go or you can use the hairpinning feature on the pix which will allow it to reroute traffic back out of the same interface but even then there is a fair bit of extra config and it can get complicated what with NAT etc.


The pix way could be done without downtime in theory but i would agree with Darren and take an outage.


Jon

darren.g Mon, 09/20/2010 - 15:57

Wade.


I'm with Jon on this one - using the PIX as the primary router is a really bad idea - it's not what the box is designed for, and you can run into all kinds of problems getting it to work.


While yes, the software version you are running will perform what you want - egressing packets onth e same interface as they ingressed from - it's not recommended, and it can open a whole can of security holes in your network you don't want to think about (since you have to put in a rule which leave the port wide open, because you want to allow all kinds of conversation between VLAN1 and the other VLAN's).


I know what I'd be doing - giving up a bit of sleep and arranging a midngiht visit to make the changes - because you're opening yourself to a whole lot of headaches if you do it the other way. The outage should be short if you plan it properly - you can lay out your changes in a text file, then cut-and-paste them into the switch/firewall in seconds. If the business wants the increased functionality, they'll agree to the change window. If not,t hey can't blame you. :-)


Cheers.

darren.g Sun, 09/19/2010 - 15:23

hi Wade.


What you've got there is most likely a firewall problem, not a routing problem. Because the PIX is set to be the default gateway for VLAN1, it's entirely possible that it's dropping the packets destined for the other VLAN's, either because, as Jon said below the software version on the PIX doesn't allow packets to egress the same interface they ingressed from, or because you have ruleset issues which simply drop the traffic.


I'd make the following changes to get this working.


Swap the IP addresses of the "inside" interface on the PIX with the VLAN1 interface on the 3550, and change your routing in the 3550 to point a default route to the PIX from all VLAN's, not just VLAN1.


So you'd end up with the following


PIX "inside" interface - IP address 192.168.1.254.

3550 VLAN interface - IP address 192.168.1.1


In the PIX, add the following static routes


192.168.1.0/24 via 192.168.1.1

192.168.10.0/24 via 192.168.1.1

192.168.11.0/24 via 192.168.1.1

192.168.13.0/24 via 192.168.1.1

192.168.14.0/24 via 192.168.1.1


And on the 3550, add the following


ip route 0.0.0.0 0.0.0.0 192.168.1.254


You'll also need to change any NAT rules on the PIX which refer to the IP address rather than the interface name - without knowing the configuration, I don't know how much work would be involved in doing this. There might not be any.


This has the effect of requiring the smallest change, removes the routing function from the PIX (which it really shouldn't be doing anyway!), and allows for additional VLAN's to be connected in the 3550 if required without any further interruption.


Your PIX routing table should then look like this


S    0.0.0.0 0.0.0.0 [1/0] via 1.2.3.4, outside
C    192.168.1.0 255.255.255.0 is directly connected, inside
S    192.168.14.0 255.255.255.0 [1/0] via 192.168.1.1, inside

S    192.168.10.0 255.255.255.0 [1/0] via 192.168.1.1 inside

S    192.168.11.0 255.255.255.0 [1/0] via 192.168.1.1 inside

S    192.168.13.0 255.255.255.0 [1/0] via 192.168.1.1 inside
C    208.123.216.0 255.255.252.0 is directly connected, outside


You shouldn't need to add a route for 192.168.1.0/24 ebcause it's a connected interface, and packets destined for this network from the outside should jsut be send out the 'inside' interface.


I know you originally said you didn't want any outages, but you're not really going to be able to fix this without some form of outage, from what I can see. Provided all your kit is reasonably close togetehr physically, and you can swap console cables quick, this should not be too long.


The best bit is you don't need to change the clients on VLAN1 - they will just keep routing via 192.168.1.1 - it's just the device with that address will change. The most the clients are going to need is an ARP table flush.


Hope this makes it clear - if not, feel free to let me know.


Cheers

Actions

This Discussion

Related Content