Network design question

[Edward Mario]

A small company I work for currently have the following design for their rack in the data center.

The ASA acts as a router on a stick as well as an edge device. The Catalyst 3750 is operating in layer 2 only.

There are multiple blade enclosures as well as physical servers.

The ASA also does a lot of router ACL (reflexive ACL) between the subinterfaces.

The ASA also handles anyconnect (vpn) client termination as well as site to site tunnel termination

Now, this design has served them well (good enough) for the first few years.

However, a gradual increase in network traffic as well as the non modularity of the design above has finally convinced the brass to agree that a discussion about a network upgrade (overhaul) is needed.

A few issues that have occured lately:

- non modularity, any disruption on the firewalls cause the whole network to be down

- performance, suffice to say routing 10gbps traffic over 1gbps link to the ASA was not great.

- no PBR, BGP, load balancing etc

I've been thinking of a few designs lately that addresses the flaws above, and have one in mind that seems to be the sweet spot price/performance wise (we dont exactly have unlimited budget)

New design

The  new design preserves the current Cisco ASA 5520 and relegates it as an edge device that handles internet connection as well as site to site tunnels with other ASAs on other sites. A new L3 switch stack (Catalyast 4500X? As the Nexus do not do layer 3) will handle layer 3 switching in hardware which should address the bottleneck in the previous design.

The MPLS router is a separate project but will be connected directly to the L3 switch. The MPLS router, ASA, and L3 switch will run dynamic routing between them (OSPF perhaps).

A few questions:

1. Do you think this is a reasonable design? Keeping in mind that the ASA can't really go away at this stage due to site to site tunnels from other sites configured on it.

2. I know that layer 3 switches (Catalyst) do not do reflexive ACL, so I think this can be approximated with the "established" keyword. I am also aware that this is not a direct substitute, but it's a necessary evil to get hardware switching going. Are there any cost effective 10Gbps routers that are as fast and not cost too much? This way, I can replace the layer 3 switch with a router instead.

3. And lastly, I'm not a network designer, can I get your opinion or thoughts on a good design for this network? Remember, it needs to be "cheap" enough to be accepted.

Thanks

Ed

[Jon Marshall · Accepted solution]

Ed

Some Nexus switches do L3, not sure which ones but i belive the 5k, 7k and 9k are all L3 capable with the right supervisor/daughter cards.

As for your questions -

1) Yes it is a reasonable design. Using the ASA to route between vlans is a performance bottleneck and to be honest a bit of a configuraiton headache. For anything more than a small site (or a site that has very strict security requirements) a L3 switch is a much better choice.

2) The 6500 does support reflexive acls. Don't know about the 6800. The 4500 doesn't as far as i am aware. Can't say whether Nexus do or don't. If this is important to you then a L3 switch, other than the 6500, may not be the right choice.

That said a router will always have the limitation of having to use subinterfaces because it just does not have the number of interfaces available that a L3 switch has. Note there are things like the 7600 which technically is a router but it is really just a 6500 equivalent with more emphasis on some L3 things.

Also bear in mind that routers, while more flexible, are generally a lot more expensive for the same performance equivalent of a L3 switch.

A lot also depends on how much 10Gbps you want/need within the DC. Nexus switches are designed for high density 10Gbps deployments but they obviously come at a cost. A 6500 with the right supervisor and cards can also provide quite a lot of 10Gbps but there is often oversubscription.

A 4500X may well meet your requirements but it is hard to say without knowing how much throughput and 10Gbps ports you actually need.  Again there willl probably be oversubscription on the cards.

I'm not trying to confuse the issue but there are 2 key elements you need to decide on at the moment -

1) how many 10Gbps ports do you need and do you need them to run at wire speed or can they be oversubscribed. If you are currently running everything through the ASA then even if you simply went with a 4500X you should see a marked improvement in terms of inter vlan routing.

2) how important is the security aspect to you. You can use standard acls but as you say a 4500X does not support reflexive acls. So it can get quite tricky if you need to allow traffic from one vlan to another but not the other way around ie. traffic can only be initiated from one side but you do want to allow the return traffic.

Jon

[Edward Mario]

Hey Jon,

Thanks for sharing your thoughts on the subject.

I guess I have to present two alternatives: router based or L3 switch based to the board and let them decide whether the security is worth the additional cost.

I did look at the 6500 though it was just too bigh for the current available rack space and hiring an additinal rack is an added cost.

I will look at the Nexus you mentioned.

The other wrinkle is that at the moment, the site is heavily locked down with RACLs for every single service needed. There are upwards of 5K ACL lines. These RACLs are maintained by a few sysadmins whenever they need to add/remove server(s). Whatever solution I can come up with it needs to have a GUI style management to manage the ACLs, ala Cisco ASDM for ASA.

Do you know whether the Nexus provided something similar?

Thanks

Ed

[Jon Marshall]

Ed

I have no idea to be honest. I only mentioned Nexus because you mentioned 10Gbps but for a small company they are probably overkill. It depends on how much actual throughput you need for your servers internally. I have no direct experience with Nexus switches so if you want further info in terms of GUIs etc. you may want to start a separate thread.

I do have experience of 4500/6500s but again not with a GUI. I use the CLI only and coming from a Unix background i tend to use scripts to manage things like acls if needed but it depends on what you actually want to do with the acls.

If you choose a router then you will not have enough interfaces for the servers so you would still need a L2 switch, which could be your 3750 but then if it is you really don't need to be looking at the Nexus switches.

Sorry i can't help with the GUI side of things. I suspect Cisco do something for switches but it may well be integrated into their management suite so you may end paying for an awful lot you don't use.

Jon