%FIB-2-FIBDOWN: CEF has been disabled due to a low memory condition. It can be re-enabled by configuring "ip cef [distributed]"

[Jeffrey Simon]

I have a 3560 connected to an edge device and I keep etting this error.  I have turned off proxy arp and turned off ip cef under the interface, however I lot connectivity to the internet when I did that.  I can't seem to find any documentation on this error. I understand what it means, I just don't know how to resolve the problem.

4d18h: %FIB-2-FIBDOWN: CEF has been disabled due to a low memory condition. It can be re-enabled by configuring "ip cef [distributed]"

4d18h: %FIB-2-FIBDOWN: CEF has been disabled due to a low memory condition. It can be re-enabled by configuring "ip cef [distributed]"

4d18h: %FIB-2-FIBDOWN: CEF has been disabled due to a low memory condition. It can be re-enabled by configuring "ip cef [distributed]"

4d18h: %FIB-2-FIBDOWN: CEF has been disabled due to a low memory condition. It can be re-enabled by configuring "ip cef [distributed]"

4d18h: %FIB-2-FIBDOWN: CEF has been disabled due to a low memory condition. It can be re-enabled by configuring "ip cef [distributed]"

4d19h: %FIB-2-FIBDOWN: CEF has been disabled due to a low memory condition. It can be re-enabled by configuring "ip cef [distributed]"

4d19h: %FIB-2-FIBDOWN: CEF has been disabled due to a low memory condition. It can be re-enabled by configuring "ip cef [distributed]"

4d19h: %FIB-2-FIBDOWN: CEF has been disabled due to a low memory condition. It can be re-enabled by configuring "ip cef [distributed]"

4d19h: %FIB-2-FIBDOWN: CEF has been disabled due to a low memory condition. It can be re-enabled by configuring "ip cef [distributed]"

4d19h: %FIB-2-FIBDOWN: CEF has been disabled due to a low memory condition. It can be re-enabled by configuring "ip cef [distributed]"

Any assistance would be appreciated.

Thanks!

Jeff

[Jeffrey Simon]

Also, I am not sure all of this should be in my ARP cache... I am fairly sure this is why my switch is running out of memory.

Internet  10.0.30.31              1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  10.0.30.230             1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  10.60.4.20              1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  17.149.36.204           0   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  17.158.10.42            1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  17.172.233.99           0   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  23.10.224.170           1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  23.11.129.212           0   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  23.11.218.85            1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  23.67.243.24            0   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  54.225.100.8            0   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  64.38.239.200           1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  64.50.236.214           1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  65.98.60.155            1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  65.175.128.102          1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  66.27.60.10             0   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  66.35.36.129            1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  66.135.211.140          0   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  66.162.25.122           1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  66.211.178.169          0   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  66.211.180.58           0   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  67.20.126.75            1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  69.16.168.244           1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  69.171.245.80           1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  71.242.0.12             1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  71.250.0.12             1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  74.125.26.99            0   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  74.125.26.103           0   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  74.125.26.104           0   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  74.125.26.105           0   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  74.125.26.106           0   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  74.125.26.147           0   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  74.125.131.109          1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  74.125.226.228          1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  76.73.4.58              1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  128.135.4.9             1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  128.175.60.118          1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  128.255.70.89           1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  129.21.171.98           1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  140.247.173.13          1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  149.20.4.71             1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  157.55.33.29            0   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  173.194.73.109          1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  173.194.73.125          1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  173.252.71.156          1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  192.150.16.37           1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  192.168.1.1             1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  198.129.224.35          1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  204.8.241.230           0   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  204.9.55.82             1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  204.11.168.20           1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  204.157.3.70            1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  204.228.227.21          1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  208.53.158.34           1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  208.68.208.49           1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  209.118.59.250          1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  216.17.3.16             1   0015.c678.cc01  ARPA   GigabitEthernet0/21

Internet  216.200.136.9           1   0015.c678.cc01  ARPA   GigabitEthernet0/21

[Reza Sharifi]

Hi

Have a look at this doc for troubleshooting tips:

FIB-2-FIBDOWN

FIB-2-FIBDOWN : CEF has been disabled due to a low memory condition.It can be re-enabled by configuring "ip cef [distributed]"

Before you re-enable the CEF, identify the cause and fix the issue.       This error might be caused by one of these issues:

• The number of not-directly connected routes that the desktop default           template allows is exceeded.

  If this template is used, the maximum number of 2000 most likely will           be exceeded.

  As a workaround, issue the sdm prefer           routing command, and reload the switch.           Ideally, this workaround resolves the problem. For more information, refer to           Configuring           SDM Templates.

• The number of MAC addresses learned by the switch has exceeded the           amount of space allocated in the hardware to store MAC addresses.

  In this case, the show mac-address-table           count output shows 0 free entries.

  As a workaround, change the Switch Database Management (SDM) template           to allow for more space in the unicast MAC address region or prune unnecessary           VLANs in order to reduce the number of MAC addresses that are learned by the           switch. This issue is documented in the Cisco bug ID           CSCef89559 (registered customers only)         .

http://www.cisco.com/en/US/products/hw/switches/ps5023/products_tech_note09186a00807ccc79.shtml

Also, can you post

sh run

HTH

[NormMuelleman]

I have a big security concern here with the arp output.

ARP is address resolution protocol, right? So there should be a 1 to 1 mapping of an internal mac address to a learned IP address within your network.

WHY is ONE DEVICE arping to SOOOOO many IP addresses? You have either a hub plugged into your network, or you have a bot that has cloned one of your mac addresses calling home or going out.

I would immediately shut that down, and find out what that mac address is. I'd also consider port blocking that mac address for the time being.

Something has gotten into your network, and it's not pretty....just my thoughts...

[Jeffrey Simon]

Sorry if I didn't make this more clear.  Int gig0/21 is connected to my router.  So all of those external arps are just using the default route to my router to get out. 

[Richard Burts · Accepted solution]

Jeff

In your second post you comment that you wonder if the very large arp cache might be what is causing the low memory problem. I suspect that you are right.

The behavior that you describe where the switch arps for every remote IP address suggests that you have configured a default route which just points at the exit interface (ip route 0.0.0.0 0.0.0.0 Gig0/2). This does cause the switch to arp for every remote address and it does depend on the next hop router enabling proxy arp. This explains why you lost Internet connectivity when you disabled proxy arp on the router.

I suggest that you re-configure the default route on the switch to specify a next hop address. I believe that this will redcuce the memory consumption due to the very large arp table.

HTH

Rick

[Jeffrey Simon]

Well, I don't know what exactly fixed this but somewhere along the way it got fixed. Just went to look into this again tonight and everything looks fine. I think no proxy arp on the routers LAN facing interface and giving the arp table some time to clear out is what fixed it but I can't be sure...

Sent from Cisco Technical Support iPhone App

[Richard Burts]

Jeff

Turning off proxy arp would help resolve the problem of low memory. But in your original post you say that you tried turning off proxy arp and that you then lost connectivity tp the internet when you did that. This would actually be the behavior that I would expect. Did you also change the default route on the switch? I would expect that this would be the effective solution to the problem.

In any case, thanks for posting back to the forum and indicating that the problem is now solved.

HTH

Rick

[Jeffrey Simon]

Well whatever changes I made seem to have caused another problem. =(

%PLATFORM_UCAST-4-PREFIX:  One or more, more specific prefixes could not be programmed into TCAM and are being covered by a less specific prefix, and the packets may be software forwarded

%PLATFORM_UCAST-4-PREFIX:  One or more, more specific prefixes could not be programmed into TCAM and are being covered by a less specific prefix, and the packets may be software forwarded

%PLATFORM_UCAST-4-PREFIX:  One or more, more specific prefixes could not be programmed into TCAM and are being covered by a less specific prefix, and the packets may be software forwarded

%PLATFORM_UCAST-4-PREFIX:  One or more, more specific prefixes could not be programmed into TCAM and are being covered by a less specific prefix, and the packets may be software forwarded

%PLATFORM_UCAST-4-PREFIX:  One or more, more specific prefixes could not be programmed into TCAM and are being covered by a less specific prefix, and the packets may be software forwarded

Any thoughts?

[Richard Burts]

Jeff

This link has a discussion of the error message that you are receiving

https://supportforums.cisco.com/thread/2004437

So you might want to look into the SDM template specification.

HTH

Rick