×

Warning message

  • Cisco Support Forums is in Read Only mode while the site is being migrated.
  • Cisco Support Forums is in Read Only mode while the site is being migrated.

Mac address tables filling up on end switches

Answered Question
Jan 14th, 2014
User Badges:

We have an odd problem on our network where our mac-address tables on our endpoint switches are filling up very quickly. At first I thought it could be mac-flooding, and started to investigate which part of the network this could be coming from, however when looking at the mac-address-table count for each vlan I'm seeing quite an even spread of macs in each vlan, no obvious spiked anywhere. Another curious thing is that vlans that have only ever had 3 devices (2 servers, and the vlan interface) in are showing a much higher mac-address count than they should.


Is it possible that this is something is looping somewhere?


This is an example snippet from one of the endpoint 2960 switches. We have a mixture of 2950 and 2960s, and a 6509 core switch.


2960switch#sh mac-address-table count

Mac Entries for Vlan 1:

---------------------------

Dynamic Address Count  : 271

Static  Address Count  : 0

Total Mac Addresses    : 271

Mac Entries for Vlan 700:

---------------------------

Dynamic Address Count  : 262

Static  Address Count  : 0

Total Mac Addresses    : 262

Mac Entries for Vlan 703:

---------------------------

Dynamic Address Count  : 264

Static  Address Count  : 0

Total Mac Addresses    : 264


Mac Entries for Vlan 704:

---------------------------

Dynamic Address Count  : 265

Static  Address Count  : 0

Total Mac Addresses    : 265


Mac Entries for Vlan 705:

---------------------------

Dynamic Address Count  : 260

Static  Address Count  : 0

Total Mac Addresses    : 260


This continues across all the vlans, of which there are 50. Vlan705 for example is one where there have never been more than 3 mac addresses on that segment.


I can provide configs from the core.

Correct Answer by Jon Marshall about 3 years 7 months ago

Andi


How exactly have you configured the bridge group ?


It sounds like what you have done is bridged all the vlans together. If you have then that is why you are seeing all the mac  addresses in all vlans. I'm not familiar with the WLC but i wouldn't have thought that is what you want to do. If you have redundant connections in your network this could create potential STP problems.


Jon

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 4 (4 ratings)
Loading.
Jon Marshall Tue, 01/14/2014 - 04:33
User Badges:
  • Super Blue, 32500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

Andi


What are those mac addresses ie. have you tried to identify if they are valid on your network or not ?


You mentioned STP, do you have redundant uplink to a pair of switches doing the inter vlan routing ?


It is unlikely to be a L2  loop as generally these tend to take down networks quite quickly.


Jon

devils_advocate Tue, 01/14/2014 - 07:10
User Badges:
  • Gold, 750 points or more
  • Community Spotlight Award,

    Small Business, November 2015

Have you tried to trace any of the addresses?


Pick a MAC address on Vlan705 that is not one of the 3 known ones and see which port the CAM table shows for it. I suspect it will be an uplink/downlink port to another switch so login to the next switch in the chain and do the same thing until you find the port the MAC address is actually plugged into.

AndiMorris Tue, 01/14/2014 - 07:34
User Badges:

Interesting, thanks both.

The mac addresses seem to be genuine devices on the network, but not ones that are actually in that vlan, so it's almost like a leak.


Tracing this up the chain I get right back to the core, where I see:

cyhr1#sh mac-address-table vl 705

Codes: * - primary entry



  vlan   mac address     type    learn qos            ports

------+----------------+--------+-----+---+--------------------------

*  705  000e.d612.3456    static  No    --  Router


I'm curious by the learn=no parameter here.

paul driver Tue, 01/14/2014 - 07:44
User Badges:
  • Green, 3000 points or more

Hello


Is this the count history after you have cleared the cam table?

Do you have any port-security configured?


If not, try enabling port sec on just one switch for testing  with say a max default of 10  mac entries per port


int ran xxx

switchport port-security maximum 10

switchport port-security violation restrict

switchport port-security


or enable DAI


res

Paul


Please don't forget to rate any posts that have been helpful.

Thanks.

AndiMorris Tue, 01/14/2014 - 07:49
User Badges:

Hi,

that is the count history not long after clearing. I'm finding the address table fills up in a matter of minutes, and is spread across all the vlans.


We don't have any port security configured, and this is something we will definitely look at doing from now on to stop this affecting us like this in the future, however for this purpose it's treating the symptom rather than the cause, and I'd really like to find out what is wrong here.

devils_advocate Tue, 01/14/2014 - 07:47
User Badges:
  • Gold, 750 points or more
  • Community Spotlight Award,

    Small Business, November 2015

That MAC address was manufatured by Cisco Systems so I suspect its a built in address for a Router/Switch.


If you do a #show interface vlan705, does the MAC address match the entry listed for this SVI?

AndiMorris Tue, 01/14/2014 - 07:51
User Badges:

Yes, it is the (obfuscated) mac address of our core switch. Running a show interface vlan 705 as you suggest does show the same mac address.

devils_advocate Tue, 01/14/2014 - 07:59
User Badges:
  • Gold, 750 points or more
  • Community Spotlight Award,

    Small Business, November 2015

Ahhh I would expect to see that in the CAM table of any switch which has Vlan705 on it but it doesn't explain the other 200+ addresses.


A CAM table for a Vlan with only a few devices in will likely have more than those devices to account for built in addresses etc but it shouldn't be that many.


Try tracing another one?


Thanks

AndiMorris Tue, 01/14/2014 - 08:10
User Badges:

Apologies, I misunderstood you.


Tracing another valid mac address up the chain, I again get to the core, however this time I see:


cyhr1#sh mac-address-table | inc 0024.d675.cca6

*  704  0024.d675.cca6   dynamic  Yes   --  Bridge

*  710  0024.d675.cca6   dynamic  Yes   --  Po10

*  729  0024.d675.cca6   dynamic  Yes   --  Bridge

*  728  0024.d675.cca6   dynamic  Yes   --  Bridge

*  731  0024.d675.cca6   dynamic  Yes   --  Bridge

*  721  0024.d675.cca6   dynamic  Yes   --  Bridge

*  727  0024.d675.cca6   dynamic  Yes   --  Bridge

*  746  0024.d675.cca6   dynamic  Yes   --  Bridge

*  751  0024.d675.cca6   dynamic  Yes   --  Bridge

*  742  0024.d675.cca6   dynamic  Yes   --  Bridge

*  743  0024.d675.cca6   dynamic  Yes   --  Bridge


Vlan 710 is the wireless client vlan, and is where I would expect to find this mac address. The Po10 does go to the WLC controller.


I can't explain the Bridges though.

paul driver Tue, 01/14/2014 - 08:13
User Badges:
  • Green, 3000 points or more

Hello


Can you post the out from sh sdm prefer this will show the TCAM resource


res

Pau







Please don't forget to rate any posts that have been helpful.

Thanks.

AndiMorris Tue, 01/14/2014 - 08:17
User Badges:

This is the output of that command from my endpoint 2960:


sh sdm prefer

The current template is "default" template.

The selected template optimizes the resources in

the switch to support this level of features for

0 routed interfaces and 255 VLANs.



  number of unicast mac addresses:                  8K

  number of IPv4 IGMP groups:                       0.25K

  number of IPv4/MAC qos aces:                      0

  number of IPv4/MAC security aces:                 0.25K


My 6509 core does not know the command.

devils_advocate Tue, 01/14/2014 - 08:17
User Badges:
  • Gold, 750 points or more
  • Community Spotlight Award,

    Small Business, November 2015

I have not seen an interface listed as 'Bridge' before so this is where my knowledge ends I am afraid.


Perhaps one of the more experienced guys on here can help with this one?

AndiMorris Tue, 01/14/2014 - 08:46
User Badges:

I think you may have pointed me in the right direction actually. Removing my wireless vlan out of the bridge-group 1 has dramatically decreased the amount of entries in my end user switch TCAM.


I think tomorrow I will remove all vlans from the bridge group. I don't really know why we have it setup anyway.

devils_advocate Tue, 01/14/2014 - 08:55
User Badges:
  • Gold, 750 points or more
  • Community Spotlight Award,

    Small Business, November 2015

Does your Wireless use a controller or are your Access Points autonomous?


Thanks

Correct Answer
Jon Marshall Tue, 01/14/2014 - 11:21
User Badges:
  • Super Blue, 32500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

Andi


How exactly have you configured the bridge group ?


It sounds like what you have done is bridged all the vlans together. If you have then that is why you are seeing all the mac  addresses in all vlans. I'm not familiar with the WLC but i wouldn't have thought that is what you want to do. If you have redundant connections in your network this could create potential STP problems.


Jon

AndiMorris Tue, 01/14/2014 - 11:51
User Badges:

Yes, all of our vlans are in the same bridge group. I don't know why it's like this, perhaps something from a legacy setup. I've now removed a few vlans from the group with no obvious degradation in service, so I'll continue to remove the vlans out of the group.

AndiMorris Wed, 01/15/2014 - 02:08
User Badges:

Removing the vlans from the bridge group has resolved this.


Thanks everyone for your help.

Actions

This Discussion

Related Content