Broadcast Domain Size

[matthew.mcbride]

Hello,

I am looking for real-world experiences and suggestions from the network engineering community concerning the size of Ethernet broadcast domains. The IEEE standard for 802.3 recommends no more than 1024 hosts in a single broadcast domain. Some protocols and operating systems are more chatty than others generating higher volumes of broadcast type traffic.

My question is, how large can a broadcast domain grow? Is a /22, according to the 802.3 standard a good thing to stick to or does it depend on the individual environment? Can a broadcast domain grow to a /18 or larger without any significant impact on the hosts in that domain thus affecting network stability, reliability, and availabilty? I'm just trying to get a feel for what other engineers have experienced over the course of their careers. Any suggestions are welcome.

Thanks for your time.

-m2

[twojciac]

It depends how chatty your hosts are and what your requirements are :)

My personal experience is on a typical desktop network you should strive to get no larger than a /23. For special networks, like a wireless network where you want seamless roaming in an entire building, a /22 might be a firm requirement to meet the demand. For datacenters where you might have hosts with a number of virtual interfaces, the requirement for a /21 or /20 is acceptable because the number of actual hosts still remains low.

With L3 switched networks, there's a limited benefit of large broadcast domains. Why do you want large broadcast domains?

[bwindle]

At one of my networks they run a /27 with about 450ish devices. They see about 400 ARP packets a minute, and TONS of other Windows broadcast junk. I would be very hesitant to stick 16k Windows machines in the same broadcast domain... the ARPs alone would be somewhere around 230 a second (assuming the /18 is full of machines)

[matthew.mcbride]

I completely agree that the size depends on the type of traffic within the broadcast domain.

Right now, we have a pure Linux implementation of 200+ servers in a data center environment. These servers contain anywhere from 1 to 400 virtual IP addresses for a hosting environment. Due to legacy design and our current vendor for the virtual private server (VPS) software requirement, these hosts need to remain in the same broadcast domain. But the problem is it just keeps growing.

A packet capture on this vlan reveals no L3 broadcast traffic but at L2 there are obviously a good amount of ARP broadcasts. Right now the vlan has the equivalent of approximately a /19. When looking at counters and such for L2 information on various switches, it appears we average about 1-2% ARP packets to unicast packets.

I've taken some steps to increase the default time-out value for the IP ARP cache from 4 hours to 24 hours to see I could aleave the amount of ARP broadcasts. This seems to have helped a bit.

Anyway, I am concerned the broadcast domain has outgrown itself; a new design and/or solution is definitely in order. We haven't had any issues to this point . In fact, this vlan is even running HSRP with multiple secondary interfaces; yikes, another thorn in my side.

Thanks for your response. I'm still looking for more details from the engineering community to see if anyone else out there has run a broadcast domain of this size and if so, what problems have you seen?

Obviously to pull this off you have to have a very controlled environment at L2 and L3.

Thanks

-m2