Our ASR crashed today. What should be the problem?
------------------ show version ------------------ Cisco IOS Software, IOS-XE Software (X86_64_LINUX_IOSD-ADVENTERPRISEK9-M), Version 15.1(1)S2, RELEASE SOFTWARE (fc1) Technical Support: http://www.cisco.com/techsupport Copyright (c) 1986-2011 by Cisco Systems, Inc. Compiled Tue 26-Apr-11 13:53 by mcpre Cisco IOS-XE software, Copyright (c) 2005-2011 by cisco Systems, Inc. All rights reserved. Certain components of Cisco IOS-XE software are licensed under the GNU General Public License ("GPL") Version 2.0. The software code licensed under GPL Version 2.0 is free software that comes with ABSOLUTELY NO WARRANTY. You can redistribute and/or modify such GPL code under the terms of GPL Version 2.0. For more details, see the documentation or "License Notice" file accompanying the IOS-XE software, or the applicable URL provided on the flyer accompanying the IOS-XE software. ROM: IOS-XE ROMMON ASR1004-79-ADSL uptime is 1 hour, 30 minutes Uptime for this control processor is 1 hour, 31 minutes System returned to ROM by reload at 20:46:21 TEB Tue Dec 6 2011 System restarted at 20:49:25 TEB Tue Dec 6 2011 System image file is "bootflash:asr1000rp2-adventerprisek9.03.02.02.S.151-1.S2.bin" Last reload reason: Reload Command This product contains cryptographic features and is subject to United States and local country laws governing import, export, transfer and use. Delivery of Cisco cryptographic products does not imply third-party authority to import, export, distribute or use encryption. Importers, exporters, distributors and users are responsible for compliance with U.S. and local country laws. By using this product you agree to comply with applicable laws and regulations. If you are unable to comply with U.S. and local laws, return this product immediately. A summary of U.S. laws governing Cisco cryptographic products may be found at: http://www.cisco.com/wwl/export/crypto/tool/stqrg.html If you require further assistance please contact us by sending email to firstname.lastname@example.org. cisco ASR1004 (RP2) processor with 4279411K/6147K bytes of memory. 8 Gigabit Ethernet interfaces 32768K bytes of non-volatile configuration memory. 8388608K bytes of physical memory. 1925119K bytes of eUSB flash at bootflash:. 78085207K bytes of SATA hard disk at harddisk:. Configuration register is 0x2102
Fix your EIGRP adjacency first. Why is the adjacency constantly going up and down?
Before this crash, the ASR was up for about 1 week and 4 days. So I will have to presume that it must've been crashing. So fix the underlying issue before we tackle the next possibility.
I'm not discounting an IOS bug, however, seconds before the crashed occur nothing is showing that it's caused by an IOS bug. Instead, the logs is showing that your EIGRP adjacencies keep going down and up and then the appliance crashed.
If you believe that unstable EIGRP adjacencies are not causing the issue (particularly the logging side) then disable logging (not recommendable).
Otherwise another option is to upgrade your IOS.
Eigrp is stable. Before the crash, I think the main dmvpn tunnel was down, when it went up, routers' control plane did not handle the all eigrp requests and crashed. But it should not be crashed.
Now that I can open the file on my PC I can see this;
Dec 6 2011 18:48:18.173 : %ASR1000_INFRA-4-NO_PUNT_KEEPALIVE: Keepalive not received for 300 seconds
Dec 6 2011 18:48:18.173 : %ASR1000_INFRA-2-FATAL_NO_PUNT_KEEPALIVE: Keepalive not received for 300 seconds resetting
The reason for reset is that the RP lost contact with the ESP (forwarding engine) and reloaded the system.
I think your version is current enough that what the ESP was doing at the time was also captured (enhancement added due to this CSCtl92129) but you will still need TAC to look into it further.
It would be rediculous route flaps crash Cisco core routers. We have quite a few crashes on our main gateway ASR1006 with dual RP1 which cause P1 network outage for us.
Our network staff start to lose confidence on the ASR, if there are choice please don't use it as the core router.
I've got a few ASR 1002 and I've never had a crash.
Granted the IOS is always (and I mean ALWAYS) gets upgraded.
Our routers are used for internet gateway with 1 Gbps connection.