06-30-2013 11:05 AM
Hi Guys, i am new to XR IOS, just recently deployed some 9001 XR on our network, had a prblem recently where one devices interfaces loose traffic, all the neighbours go down as per the below log, i am trying to find the reason for this as it keeps happening on 5Hour intervals. Thanks
RP/0/RSP0/CPU0:Jun 30 03:25:08.437 : eem_ed_oir[190]: Messge received content : Event 1 NodeId: 0x828
RP/0/RSP0/CPU0:Jun 30 03:25:08.440 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB35: Bundle-Ether1.35: state Standby -> Init
RP/0/RSP0/CPU0:Jun 30 03:25:08.442 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB32: Bundle-Ether1.32: state Active -> Init
RP/0/RSP0/CPU0:Jun 30 03:25:08.443 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB31: Bundle-Ether1.31: state Standby -> Init
RP/0/RSP0/CPU0:Jun 30 03:25:08.444 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB15: Bundle-Ether1.15: state Standby -> Init
RP/0/RSP0/CPU0:Jun 30 03:25:08.447 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB30: Bundle-Ether1.307: state Active -> Init
RP/0/RSP0/CPU0:Jun 30 03:25:08.448 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB34: Bundle-Ether1.34: state Active -> Init
RP/0/RSP0/CPU0:Jun 30 03:25:08.449 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB33: Bundle-Ether1.33: state Standby -> Init
RP/0/RSP0/CPU0:Jun 30 03:25:08.449 : invmgr[255]: %PLATFORM-INV-6-NODE_STATE_CHANGE : Node: 0/0/CPU0, state: BRINGDOWN
RP/0/RSP0/CPU0:Jun 30 03:25:08.450 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB60: Bundle-Ether1.360: state Standby -> Init
RP/0/RSP0/CPU0:Jun 30 03:25:08.452 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB49: Bundle-Ether1.49: state Active -> Init
RP/0/RSP0/CPU0:Jun 30 03:25:08.453 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB35: Bundle-Ether1.305: state Standby -> Init
RP/0/RSP0/CPU0:Jun 30 03:25:08.455 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB12: Bundle-Ether1.312: state Active -> Init
RP/0/RSP0/CPU0:Jun 30 03:25:08.455 : mpls_ldp[1043]: %ROUTING-LDP-5-GR : GR session 172.29.0.8:0 (instance 1) disconnected
RP/0/RSP0/CPU0:Jun 30 03:25:08.457 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB11: Bundle-Ether1.311: state Standby -> Init
RP/0/RSP0/CPU0:Jun 30 03:25:08.457 : mpls_ldp[1043]: %ROUTING-LDP-5-NBR_CHANGE : Neighbor 172.29.0.8:0, DOWN (Interface state down)
RP/0/RSP0/CPU0:Jun 30 03:25:08.458 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB16: Bundle-Ether1.16: state Active -> Init
RP/0/RSP0/CPU0:Jun 30 03:25:08.460 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB29: Bundle-Ether1.29: state Active -> Init
RP/0/RSP0/CPU0:Jun 30 03:25:08.461 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB27: Bundle-Ether1.27: state Standby -> Init
RP/0/RSP0/CPU0:Jun 30 03:25:08.463 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB79: Bundle-Ether1.709: state Standby -> Init
RP/0/RSP0/CPU0:Jun 30 03:25:38.635 : shelfmgr[395]: %PLATFORM-SHELFMGR_HAL-6-BOOT_REQ_RECEIVED : Boot Request from 0/0/CPU0, RomMon Version: 1.30
RP/0/RSP0/CPU0:Jun 30 03:25:38.637 : shelfmgr[395]: %PLATFORM-MBIMGR-7-IMAGE_VALIDATED : Remote location 0/0/CPU0: : MBI tftp:/disk0/asr9k-os-mbi-4.2.3.CSCue55886-1.0.0/lc/mbiasr9k-lc.vm validated
RP/0/RSP0/CPU0:Jun 30 03:25:38.640 : shelfmgr[395]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/0/CPU0 ASR9001-LC state:MBI-BOOTING
RP/0/RSP0/CPU0:Jun 30 03:26:56.660 : shelfmgr[395]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/0/CPU0 ASR9001-LC state:MBI-RUNNING
RP/0/RSP0/CPU0:Jun 30 03:27:13.194 : bgp[1048]: %ROUTING-BGP-5-ADJCHANGE : neighbor 172.29.0.11 Down - BGP Notification sent, hold time expired (VRF: default)
RP/0/RSP0/CPU0:Jun 30 03:27:19.774 : bgp[1048]: %ROUTING-BGP-5-ADJCHANGE : neighbor 172.29.0.6 Down - BGP Notification sent, hold time expired (VRF: default)
RP/0/RSP0/CPU0:Jun 30 03:27:40.594 : bgp[1048]: %ROUTING-BGP-5-ADJCHANGE : neighbor 172.29.0.1 Down - BGP Notification sent, hold time expired (VRF: default)
RP/0/RSP0/CPU0:Jun 30 03:27:40.806 : bgp[1048]: %ROUTING-BGP-5-ADJCHANGE : neighbor 172.29.0.2 Down - BGP Notification sent, hold time expired (VRF: default)
RP/0/RSP0/CPU0:Jun 30 03:27:43.563 : bgp[1048]: %ROUTING-BGP-5-ADJCHANGE : neighbor 172.29.0.8 Down - BGP Notification sent, hold time expired (VRF: default)
RP/0/RSP0/CPU0:Jun 30 03:27:45.142 : bgp[1048]: %ROUTING-BGP-5-ADJCHANGE : neighbor 172.29.0.12 Down - BGP Notification sent, hold time expired (VRF: default)
LC/0/0/CPU0:10: init[65540]: %OS-INIT-7-MBI_STARTED : total time 5.229 seconds
LC/0/0/CPU0:Jun 30 03:27:00.002 : insthelper[61]: %INSTALL-INSTHELPER-7-PKG_DOWNLOAD : MBI running; starting software download
LC/0/0/CPU0:Jun 30 03:27:41.274 : sysmgr[90]: %OS-SYSMGR-5-NOTICE : Card is COLD started
LC/0/0/CPU0:Jun 30 03:27:41.757 : init[65540]: %OS-INIT-7-INSTALL_READY : total time 51.189 seconds
LC/0/0/CPU0:Jun 30 03:27:42.383 : sysmgr[352]: %OS-SYSMGR-6-INFO : Backup system manager is ready
RP/0/RSP0/CPU0:Jun 30 03:27:51.664 : bgp[1048]: %ROUTING-BGP-5-ADJCHANGE : neighbor 172.29.0.5 Down - BGP Notification sent, hold time expired (VRF: default)
RP/0/RSP0/CPU0:Jun 30 03:27:55.337 : shelfmgr[395]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/0/0 A9K-MPA-20X1GE state:UNPOWERED
RP/0/RSP0/CPU0:Jun 30 03:27:55.449 : shelfmgr[395]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/0/0 A9K-MPA-20X1GE state:POWER UP
LC/0/0/CPU0:Jun 30 03:27:56.275 : lda_server[64]: %L2-SPA-5-STATE_CHANGE : SPA in bay 0 type A9K-MPA-20X1GE Initing
RP/0/RSP0/CPU0:Jun 30 03:27:57.309 : shelfmgr[395]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/0/0 A9K-MPA-20X1GE state:INITIALIZED
LC/0/0/CPU0:Jun 30 03:27:57.435 : kdscrape-lc[62]: KERNEL-DUMP: Copied LC1.130630-012501.e500 to /harddisk:/dumper
LC/0/0/CPU0:Jun 30 03:27:57.542 : kdscrape-lc[62]: KERNEL-DUMP: Copied LC1.130630-012501.crashinfo.by.p40x0mc to /harddisk:/dumper
RP/0/RSP0/CPU0:Jun 30 03:27:57.585 : invmgr[255]: %PLATFORM-INV-6-IF_OIRIN : xFP OIR: 0/0/0 port_num: 0 is inserted, state: 1
RP/0/RSP0/CPU0:Jun 30 03:27:59.193 : invmgr[255]: %PLATFORM-INV-6-IF_OIRIN : xFP OIR: 0/0/0 port_num: 1 is inserted, state: 1
RP/0/RSP0/CPU0:Jun 30 03:28:00.795 : invmgr[255]: %PLATFORM-INV-6-IF_OIRIN : xFP OIR: 0/0/0 port_num: 2 is inserted, state: 1
LC/0/0/CPU0:Jun 30 03:28:01.269 : pfm_node_lc[291]: %FABRIC-FIA-1-DATA_SB_SERDES_1_FAIL_0 : Set|fialc[139329]|Fabric Interface(0x103b000)|Data SB Serdes Link 1 Failure on FIA 0
LC/0/0/CPU0:Jun 30 03:28:01.271 : pfm_node_lc[291]: %FABRIC-FIA-1-DATA_SB_SERDES_0_FAIL_0 : Set|fialc[139329]|Fabric Interface(0x103b000)|Data SB Serdes Link 0 Failure on FIA 0
RP/0/RSP0/CPU0:Jun 30 03:28:01.974 : invmgr[255]: %PLATFORM-INV-6-IF_OIRIN : xFP OIR: 0/0/0 port_num: 3 is inserted, state: 1
RP/0/RSP0/CPU0:Jun 30 03:28:03.577 : invmgr[255]: %PLATFORM-INV-6-IF_OIRIN : xFP OIR: 0/0/0 port_num: 4 is inserted, state: 1
RP/0/RSP0/CPU0:Jun 30 03:28:05.181 : invmgr[255]: %PLATFORM-INV-6-IF_OIRIN : xFP OIR: 0/0/0 port_num: 5 is inserted, state: 1
RP/0/RSP0/CPU0:Jun 30 03:28:06.785 : invmgr[255]: %PLATFORM-INV-6-IF_OIRIN : xFP OIR: 0/0/0 port_num: 6 is inserted, state: 1
RP/0/RSP0/CPU0:Jun 30 03:28:08.386 : mpls_ldp[1043]: %ROUTING-LDP-5-GR : GR session 172.29.0.1:0 (instance 1) timed out
RP/0/RSP0/CPU0:Jun 30 03:28:08.389 : invmgr[255]: %PLATFORM-INV-6-IF_OIRIN : xFP OIR: 0/0/0 port_num: 7 is inserted, state: 1
RP/0/RSP0/CPU0:Jun 30 03:28:08.458 : mpls_ldp[1043]: %ROUTING-LDP-5-GR : GR session 172.29.0.8:0 (instance 1) timed out
LC/0/0/CPU0:Jun 30 03:28:09.559 : prm_server_ty[303]: %PLATFORM-QOSHAL-6-QOS_MODE_CHANGE : Line-card QoS mode set to child-shaping-enabled.
LC/0/0/CPU0:Jun 30 03:28:09.563 : prm_server_ty[303]: %PLATFORM-QOSHAL-6-QOS_MODE_CHANGE : Line-card QoS mode set to lowburst-disabled.
RP/0/RSP0/CPU0:Jun 30 03:28:09.993 : invmgr[255]: %PLATFORM-INV-6-IF_OIRIN : xFP OIR: 0/0/0 port_num: 8 is inserted, state: 1
LC/0/0/CPU0:Jun 30 03:28:10.011 : sysmgr[90]: %OS-SYSMGR-7-DEBUG : sysmgr_admin_plane_check:SYSMGR_PLANE_ADMIN Notification sent.
RP/0/RSP0/CPU0:Jun 30 03:28:11.180 : invmgr[255]: %PLATFORM-INV-6-IF_OIRIN : xFP OIR: 0/0/0 port_num: 9 is inserted, state: 1
LC/0/0/CPU0:Jun 30 03:28:13.005 : syslog_dev[88]: hsbi_fpd_agent[198]: FPD ltrace_file_name => fpd-agent/hsbi_fpd_agent
LC/0/0/CPU0:Jun 30 03:28:15.980 : rsi_agent[318]: %OS-RSI_AGENT-6-CARD_ROLE_CHANGE : Based on the card configuration/type, the AFI IPv4 role of the card has changed from Invalid to Customer Facing
LC/0/0/CPU0:Jun 30 03:28:15.984 : rsi_agent[318]: %OS-RSI_AGENT-6-CARD_ROLE_CHANGE : Based on the card configuration/type, the AFI IPv6 role of the card has changed from Invalid to Not Interested
RP/0/RSP0/CPU0:Jun 30 03:28:16.600 : invmgr[255]: %PLATFORM-INV-6-IF_OIRIN : xFP OIR: 0/0/0 port_num: 11 is inserted, state: 1
RP/0/RSP0/CPU0:Jun 30 03:28:33.471 : shelfmgr[395]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/0/0 A9K-MPA-20X1GE state:OK
RP/0/RSP0/CPU0:Jun 30 03:28:33.488 : invmgr[255]: %PLATFORM-INV-6-NODE_STATE_CHANGE : Node: 0/0/0, state: OK
RP/0/RSP0/CPU0:Jun 30 03:28:33.495 : invmgr[255]: %PLATFORM-INV-6-OIRIN : OIR: Node 0/0/0 inserted
RP/0/RSP0/CPU0:Jun 30 03:28:33.496 : eem_ed_oir[190]: Messge received content : Event 0 NodeId: 0x828
RP/0/RSP0/CPU0:Jun 30 03:28:33.498 : eem_ed_oir[190]: Messge received content : Event 0 NodeId: 0x828
LC/0/0/CPU0:Jun 30 03:28:34.330 : pfm_node_lc[291]: %FABRIC-FIA-1-DATA_SB_SERDES_1_FAIL_0 : Clear|fialc[139329]|Fabric Interface(0x103b000)|Data SB Serdes Link 1 Failure on FIA 0
LC/0/0/CPU0:Jun 30 03:28:34.331 : pfm_node_lc[291]: %FABRIC-FIA-1-DATA_SB_SERDES_0_FAIL_0 : Clear|fialc[139329]|Fabric Interface(0x103b000)|Data SB Serdes Link 0 Failure on FIA 0
LC/0/0/CPU0:Jun 30 03:28:44.631 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface TenGigE0/0/2/3, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:44.632 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface TenGigE0/0/2/2, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:44.633 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface TenGigE0/0/2/1, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:44.633 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface TenGigE0/0/2/0, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:44.637 : cfgmgr-lc[134]: %MGBL-CONFIG-6-OIR_RESTORE : Configuration for node '0/0/2' has been restored.
LC/0/0/CPU0:Jun 30 03:28:47.602 : rsi_agent[318]: %OS-RSI_AGENT-6-CARD_ROLE_CHANGE : Based on the card configuration/type, the AFI IPv4 role of the card has changed from Customer Facing to Core Facing
LC/0/0/CPU0:Jun 30 03:28:47.603 : rsi_agent[318]: %OS-RSI_AGENT-6-CARD_ROLE_CHANGE : Based on the card configuration/type, the AFI IPv6 role of the card has changed from Not Interested to Core Facing
RP/0/RSP0/CPU0:Jun 30 03:28:51.005 : shelfmgr[395]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/0/CPU0 ASR9001-LC state:IOS XR RUN
RP/0/RSP0/CPU0:Jun 30 03:28:51.010 : invmgr[255]: %PLATFORM-INV-6-NODE_STATE_CHANGE : Node: 0/0/CPU0, state: IOS XR RUN
LC/0/0/CPU0:Jun 30 03:28:51.022 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/19, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:51.022 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/18, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:51.022 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/17, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:51.023 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/16, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:51.023 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/15, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:51.024 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/14, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:51.024 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/13, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:51.024 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/12, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:51.025 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/11, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:51.025 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/10, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:51.025 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/9, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:51.026 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/8, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:51.026 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/7, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:51.030 : ifmgr[201]: %PKT_INFRA-LINK-3-UPDOWN : Interface GigabitEthernet0/0/0/6, changed state to Down
LC/0/0/CPU0:Jun 30 03:28:51.031 : ifmgr[201]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface GigabitEthernet0/0/0/6, changed state to Down
LC/0/0/CPU0:Jun 30 03:28:51.031 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/5, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:51.032 : ifmgr[201]: %PKT_INFRA-LINK-3-UPDOWN : Interface GigabitEthernet0/0/0/4, changed state to Down
LC/0/0/CPU0:Jun 30 03:28:51.032 : ifmgr[201]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface GigabitEthernet0/0/0/4, changed state to Down
LC/0/0/CPU0:Jun 30 03:28:51.032 : ifmgr[201]: %PKT_INFRA-LINK-3-UPDOWN : Interface GigabitEthernet0/0/0/3, changed state to Down
LC/0/0/CPU0:Jun 30 03:28:51.032 : ifmgr[201]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface GigabitEthernet0/0/0/3, changed state to Down
LC/0/0/CPU0:Jun 30 03:28:51.033 : ifmgr[201]: %PKT_INFRA-LINK-3-UPDOWN : Interface GigabitEthernet0/0/0/2, changed state to Down
LC/0/0/CPU0:Jun 30 03:28:51.034 : ifmgr[201]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface GigabitEthernet0/0/0/2, changed state to Down
LC/0/0/CPU0:Jun 30 03:28:51.034 : ifmgr[201]: %PKT_INFRA-LINK-3-UPDOWN : Interface GigabitEthernet0/0/0/1, changed state to Down
LC/0/0/CPU0:Jun 30 03:28:51.034 : ifmgr[201]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface GigabitEthernet0/0/0/1, changed state to Down
LC/0/0/CPU0:Jun 30 03:28:51.035 : ifmgr[201]: %PKT_INFRA-LINK-3-UPDOWN : Interface GigabitEthernet0/0/0/0, changed state to Down
LC/0/0/CPU0:Jun 30 03:28:51.035 : ifmgr[201]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface GigabitEthernet0/0/0/0, changed state to Down
LC/0/0/CPU0:Jun 30 03:28:51.043 : cfgmgr-lc[134]: %MGBL-CONFIG-6-OIR_RESTORE : Configuration for node '0/0/0' has been restored.
LC/0/0/CPU0:Jun 30 03:28:57.029 : rsi_agent[318]: %OS-RSI_AGENT-6-CARD_ROLE_CHANGE : Based on the card configuration/type, the AFI IPv4 role of the card has changed from Core Facing to Standard
LC/0/0/CPU0:Jun 30 03:28:57.030 : rsi_agent[318]: %OS-RSI_AGENT-6-CARD_ROLE_CHANGE : Based on the card configuration/type, the AFI IPv6 role of the card has changed from Core Facing to Standard
LC/0/0/CPU0:Jun 30 03:28:58.163 : ifmgr[201]: %PKT_INFRA-LINK-3-UPDOWN : Interface GigabitEthernet0/0/0/6, changed state to Up
LC/0/0/CPU0:Jun 30 03:28:58.165 : ifmgr[201]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface GigabitEthernet0/0/0/6, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.171 : BM-DISTRIB[1148]: %L2-BM-6-ACTIVE : GigabitEthernet0/0/0/6 is Active as part of Bundle-Ether1
RP/0/RSP0/CPU0:Jun 30 03:29:00.171 : ifmgr[243]: %PKT_INFRA-LINK-3-UPDOWN : Interface Bundle-Ether1, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.174 : ifmgr[243]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface Bundle-Ether1, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.178 : ifmgr[243]: %PKT_INFRA-LINK-3-UPDOWN : Interface Bundle-Ether1.35, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.178 : ifmgr[243]: %PKT_INFRA-LINK-3-UPDOWN : Interface Bundle-Ether1.32, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.179 : ifmgr[243]: %PKT_INFRA-LINK-3-UPDOWN : Interface Bundle-Ether1.31, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.179 : ifmgr[243]: %PKT_INFRA-LINK-3-UPDOWN : Interface Bundle-Ether1.15, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.180 : ifmgr[243]: %PKT_INFRA-LINK-3-UPDOWN : Interface Bundle-Ether1.307, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.180 : ifmgr[243]: %PKT_INFRA-LINK-3-UPDOWN : Interface Bundle-Ether1.34, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.181 : ifmgr[243]: %PKT_INFRA-LINK-3-UPDOWN : Interface Bundle-Ether1.33, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.181 : ifmgr[243]: %PKT_INFRA-LINK-3-UPDOWN : Interface Bundle-Ether1.360, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.182 : ifmgr[243]: %PKT_INFRA-LINK-3-UPDOWN : Interface Bundle-Ether1.49, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.182 : ifmgr[243]: %PKT_INFRA-LINK-3-UPDOWN : Interface Bundle-Ether1.305, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.183 : ifmgr[243]: %PKT_INFRA-LINK-3-UPDOWN : Interface Bundle-Ether1.40, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.183 : ifmgr[243]: %PKT_INFRA-LINK-3-UPDOWN : Interface Bundle-Ether1.312, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.183 : ifmgr[243]: %PKT_INFRA-LINK-3-UPDOWN : Interface Bundle-Ether1.311, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.184 : ifmgr[243]: %PKT_INFRA-LINK-3-UPDOWN : Interface Bundle-Ether1.16, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.184 : ifmgr[243]: %PKT_INFRA-LINK-3-UPDOWN : Interface Bundle-Ether1.29, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.184 : ifmgr[243]: %PKT_INFRA-LINK-3-UPDOWN : Interface Bundle-Ether1.27, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.185 : ifmgr[243]: %PKT_INFRA-LINK-3-UPDOWN : Interface Bundle-Ether1.709, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.192 : ifmgr[243]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface Bundle-Ether1.35, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.192 : ifmgr[243]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface Bundle-Ether1.32, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.192 : ifmgr[243]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface Bundle-Ether1.31, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.192 : ifmgr[243]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface Bundle-Ether1.15, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.193 : ifmgr[243]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface Bundle-Ether1.307, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.193 : ifmgr[243]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface Bundle-Ether1.34, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.193 : ifmgr[243]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface Bundle-Ether1.33, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.194 : ifmgr[243]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface Bundle-Ether1.360, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.194 : ifmgr[243]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface Bundle-Ether1.49, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.194 : ifmgr[243]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface Bundle-Ether1.305, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.195 : ifmgr[243]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface Bundle-Ether1.40, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.195 : ifmgr[243]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface Bundle-Ether1.312, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.195 : ifmgr[243]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface Bundle-Ether1.311, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.195 : ifmgr[243]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface Bundle-Ether1.16, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.195 : ifmgr[243]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface Bundle-Ether1.29, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.196 : ifmgr[243]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface Bundle-Ether1.27, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:00.196 : ifmgr[243]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface Bundle-Ether1.709, changed state to Up
LC/0/0/CPU0:Jun 30 03:29:03.825 : ifmgr[201]: %PKT_INFRA-LINK-3-UPDOWN : Interface GigabitEthernet0/0/0/0, changed state to Up
LC/0/0/CPU0:Jun 30 03:29:03.827 : ifmgr[201]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface GigabitEthernet0/0/0/0, changed state to Up
LC/0/0/CPU0:Jun 30 03:29:03.840 : ifmgr[201]: %PKT_INFRA-LINK-3-UPDOWN : Interface GigabitEthernet0/0/0/1, changed state to Up
LC/0/0/CPU0:Jun 30 03:29:03.841 : ifmgr[201]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface GigabitEthernet0/0/0/1, changed state to Up
LC/0/0/CPU0:Jun 30 03:29:03.855 : ifmgr[201]: %PKT_INFRA-LINK-3-UPDOWN : Interface GigabitEthernet0/0/0/2, changed state to Up
LC/0/0/CPU0:Jun 30 03:29:03.857 : ifmgr[201]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface GigabitEthernet0/0/0/2, changed state to Up
LC/0/0/CPU0:Jun 30 03:29:04.016 : ifmgr[201]: %PKT_INFRA-LINK-3-UPDOWN : Interface GigabitEthernet0/0/0/4, changed state to Up
LC/0/0/CPU0:Jun 30 03:29:04.018 : ifmgr[201]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface GigabitEthernet0/0/0/4, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:04.022 : BM-DISTRIB[1148]: %L2-BM-6-ACTIVE : GigabitEthernet0/0/0/4 is Active as part of Bundle-Ether1
RP/0/RSP0/CPU0:Jun 30 03:29:04.349 : ospf[1011]: %ROUTING-OSPF-5-ADJCHG : Process 200, Nbr 172.29.0.1 on GigabitEthernet0/0/0/0 in area 0 from LOADING to FULL, Loading Done,vrf default vrfid 0x60000000
LC/0/0/CPU0:Jun 30 03:29:05.811 : pfm_node_lc[291]: %PLATFORM-SFP-3-HIGH_RX_POWER_WARNING : Set|envmon_lc[143443]|SFP(0x1029001)|Port_0/01
LC/0/0/CPU0:Jun 30 03:29:05.811 : pfm_node_lc[291]: %PLATFORM-SFP-3-HIGH_RX_POWER_WARNING : Set|envmon_lc[143443]|SFP(0x1029002)|Port_0/02
RP/0/RSP0/CPU0:Jun 30 03:29:05.849 : BM-DISTRIB[1148]: %L2-BM-6-ACTIVE : GigabitEthernet0/0/0/1 is Active as part of Bundle-Ether2
RP/0/RSP0/CPU0:Jun 30 03:29:05.849 : BM-DISTRIB[1148]: %L2-BM-6-ACTIVE : GigabitEthernet0/0/0/2 is Active as part of Bundle-Ether2
RP/0/RSP0/CPU0:Jun 30 03:29:05.850 : ifmgr[243]: %PKT_INFRA-LINK-3-UPDOWN : Interface Bundle-Ether2, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:05.852 : ifmgr[243]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface Bundle-Ether2, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:05.877 : ospf[1011]: %ROUTING-OSPF-5-ADJCHG : Process 200, Nbr 172.29.0.8 on Bundle-Ether2 in area 0 from LOADING to FULL, Loading Done,vrf default vrfid 0x60000000
RP/0/RSP0/CPU0:Jun 30 03:29:05.928 : mpls_ldp[1043]: %ROUTING-LDP-5-NBR_CHANGE : Neighbor 172.29.0.8:0, UP
RP/0/RSP0/CPU0:Jun 30 03:29:06.730 : bgp[1048]: %ROUTING-BGP-5-ADJCHANGE : neighbor 172.29.0.1 Up (VRF: default)
LC/0/0/CPU0:Jun 30 03:29:06.914 : ifmgr[201]: %PKT_INFRA-LINK-3-UPDOWN : Interface GigabitEthernet0/0/0/3, changed state to Up
LC/0/0/CPU0:Jun 30 03:29:06.915 : ifmgr[201]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface GigabitEthernet0/0/0/3, changed state to Up
RP/0/RSP0/CPU0:Jun 30 03:29:06.984 : mpls_ldp[1043]: %ROUTING-LDP-5-NBR_CHANGE : Neighbor 172.29.0.11:0, UP
RP/0/RSP0/CPU0:Jun 30 03:29:07.079 : bgp[1048]: %ROUTING-BGP-5-ADJCHANGE : neighbor 172.29.0.12 Up (VRF: default)
RP/0/RSP0/CPU0:Jun 30 03:29:07.894 : mpls_ldp[1043]: %ROUTING-LDP-5-NBR_CHANGE : Neighbor 172.29.0.1:0, UP
RP/0/RSP0/CPU0:Jun 30 03:29:08.134 : bgp[1048]: %ROUTING-BGP-5-ADJCHANGE : neighbor 172.29.0.2 Up (VRF: default)
RP/0/RSP0/CPU0:Jun 30 03:29:08.924 : bgp[1048]: %ROUTING-BGP-5-ADJCHANGE : neighbor 172.29.0.11 Up (VRF: default)
RP/0/RSP0/CPU0:Jun 30 03:29:09.511 : bgp[1048]: %ROUTING-BGP-5-ADJCHANGE : neighbor 172.29.0.5 Up (VRF: default)
RP/0/RSP0/CPU0:Jun 30 03:29:10.033 : bgp[1048]: %ROUTING-BGP-5-ADJCHANGE : neighbor 172.29.0.6 Up (VRF: default)
RP/0/RSP0/CPU0:Jun 30 03:29:12.443 : ospf[1011]: %ROUTING-OSPF-5-ADJCHG : Process 200, Nbr 172.29.0.11 on GigabitEthernet0/0/0/3 in area 0 from LOADING to FULL, Loading Done,vrf default vrfid 0x60000000
RP/0/RSP0/CPU0:Jun 30 03:29:13.192 : bgp[1048]: %ROUTING-BGP-5-ADJCHANGE : neighbor 172.29.0.8 Up (VRF: default)
RP/0/RSP0/CPU0:Jun 30 03:29:21.056 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB32: Bundle-Ether1.32: state Speak -> Standby
RP/0/RSP0/CPU0:Jun 30 03:29:21.056 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB32: Bundle-Ether1.32: state Standby -> Active
RP/0/RSP0/CPU0:Jun 30 03:29:21.063 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB29: Bundle-Ether1.29: state Speak -> Standby
RP/0/RSP0/CPU0:Jun 30 03:29:21.064 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB29: Bundle-Ether1.29: state Standby -> Active
RP/0/RSP0/CPU0:Jun 30 03:29:21.074 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB79: Bundle-Ether1.709: state Speak -> Standby
RP/0/RSP0/CPU0:Jun 30 03:29:21.074 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB79: Bundle-Ether1.709: state Standby -> Active
RP/0/RSP0/CPU0:Jun 30 03:29:21.105 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB34: Bundle-Ether1.34: state Speak -> Standby
RP/0/RSP0/CPU0:Jun 30 03:29:21.105 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB34: Bundle-Ether1.34: state Standby -> Active
RP/0/RSP0/CPU0:Jun 30 03:29:21.123 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB12: Bundle-Ether1.312: state Speak -> Standby
RP/0/RSP0/CPU0:Jun 30 03:29:21.123 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB12: Bundle-Ether1.312: state Standby -> Active
RP/0/RSP0/CPU0:Jun 30 03:29:21.123 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB16: Bundle-Ether1.16: state Speak -> Standby
RP/0/RSP0/CPU0:Jun 30 03:29:21.124 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB16: Bundle-Ether1.16: state Standby -> Active
RP/0/RSP0/CPU0:Jun 30 03:29:21.139 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB31: Bundle-Ether1.31: state Speak -> Standby
RP/0/RSP0/CPU0:Jun 30 03:29:21.139 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB31: Bundle-Ether1.31: state Standby -> Active
RP/0/RSP0/CPU0:Jun 30 03:29:21.145 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB30: Bundle-Ether1.307: state Speak -> Standby
RP/0/RSP0/CPU0:Jun 30 03:29:21.146 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB30: Bundle-Ether1.307: state Standby -> Active
RP/0/RSP0/CPU0:Jun 30 03:29:21.152 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB27: Bundle-Ether1.27: state Speak -> Standby
RP/0/RSP0/CPU0:Jun 30 03:29:21.153 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB27: Bundle-Ether1.27: state Standby -> Active
RP/0/RSP0/CPU0:Jun 30 03:29:21.172 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB49: Bundle-Ether1.49: state Speak -> Standby
RP/0/RSP0/CPU0:Jun 30 03:29:21.172 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB49: Bundle-Ether1.49: state Standby -> Active
RP/0/RSP0/CPU0:Jun 30 03:29:21.172 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB11: Bundle-Ether1.311: state Speak -> Standby
RP/0/RSP0/CPU0:Jun 30 03:29:21.173 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB11: Bundle-Ether1.311: state Standby -> Active
RP/0/RSP0/CPU0:Jun 30 03:29:21.180 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB35: Bundle-Ether1.35: state Speak -> Standby
RP/0/RSP0/CPU0:Jun 30 03:29:21.180 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB35: Bundle-Ether1.35: state Standby -> Active
RP/0/RSP0/CPU0:Jun 30 03:29:21.187 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB35: Bundle-Ether1.305: state Speak -> Standby
RP/0/RSP0/CPU0:Jun 30 03:29:21.187 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB35: Bundle-Ether1.305: state Standby -> Active
RP/0/RSP0/CPU0:Jun 30 03:29:21.196 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB33: Bundle-Ether1.33: state Speak -> Standby
RP/0/RSP0/CPU0:Jun 30 03:29:21.196 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB33: Bundle-Ether1.33: state Standby -> Active
RP/0/RSP0/CPU0:Jun 30 03:29:21.201 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB15: Bundle-Ether1.15: state Speak -> Standby
RP/0/RSP0/CPU0:Jun 30 03:29:21.202 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB15: Bundle-Ether1.15: state Standby -> Active
RP/0/RSP0/CPU0:Jun 30 03:29:21.212 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB60: Bundle-Ether1.360: state Speak -> Standby
RP/0/RSP0/CPU0:Jun 30 03:29:21.212 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB60: Bundle-Ether1.360: state Standby -> Active
RP/0/RSP0/CPU0:Jun 30 03:29:37.354 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB35: Bundle-Ether1.305: state Active -> Speak
RP/0/RSP0/CPU0:Jun 30 03:29:37.452 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB31: Bundle-Ether1.31: state Active -> Speak
RP/0/RSP0/CPU0:Jun 30 03:29:37.606 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB79: Bundle-Ether1.709: state Active -> Speak
RP/0/RSP0/CPU0:Jun 30 03:29:37.631 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB33: Bundle-Ether1.33: state Active -> Speak
RP/0/RSP0/CPU0:Jun 30 03:29:37.727 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB27: Bundle-Ether1.27: state Active -> Speak
RP/0/RSP0/CPU0:Jun 30 03:29:37.815 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB11: Bundle-Ether1.311: state Active -> Speak
RP/0/RSP0/CPU0:Jun 30 03:29:37.862 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB60: Bundle-Ether1.360: state Active -> Speak
RP/0/RSP0/CPU0:Jun 30 03:29:38.037 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB15: Bundle-Ether1.15: state Active -> Speak
RP/0/RSP0/CPU0:Jun 30 03:29:38.349 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB35: Bundle-Ether1.35: state Active -> Speak
RP/0/RSP0/CPU0:Jun 30 03:29:47.355 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB35: Bundle-Ether1.305: state Speak -> Standby
RP/0/RSP0/CPU0:Jun 30 03:29:47.453 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB31: Bundle-Ether1.31: state Speak -> Standby
RP/0/RSP0/CPU0:Jun 30 03:29:47.607 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB79: Bundle-Ether1.709: state Speak -> Standby
RP/0/RSP0/CPU0:Jun 30 03:29:47.632 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB33: Bundle-Ether1.33: state Speak -> Standby
RP/0/RSP0/CPU0:Jun 30 03:29:47.728 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB27: Bundle-Ether1.27: state Speak -> Standby
RP/0/RSP0/CPU0:Jun 30 03:29:47.815 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB11: Bundle-Ether1.311: state Speak -> Standby
RP/0/RSP0/CPU0:Jun 30 03:29:47.863 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB60: Bundle-Ether1.360: state Speak -> Standby
RP/0/RSP0/CPU0:Jun 30 03:29:48.038 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB15: Bundle-Ether1.15: state Speak -> Standby
RP/0/RSP0/CPU0:Jun 30 03:29:48.350 : hsrp[1085]: %IP-STANDBY-6-INFO_STATECHANGE : SB35: Bundle-Ether1.35: state Speak -> Standby
07-02-2013 01:00 PM
Looks like the whole card went down..
LC/0/0/CPU0:Jun 30 03:27:41.274 : sysmgr[90]: %OS-SYSMGR-5-NOTICE : Card is COLD started
LC/0/0/CPU0:Jun 30 03:27:41.757 : init[65540]: %OS-INIT-7-INSTALL_READY : total time 51.189 seconds
LC/0/0/CPU0:Jun 30 03:27:42.383 : sysmgr[352]: %OS-SYSMGR-6-INFO : Backup system manager is ready
RP/0/RSP0/CPU0:Jun 30 03:27:51.664 : bgp[1048]: %ROUTING-BGP-5-ADJCHANGE : neighbor 172.29.0.5 Down - BGP Notification sent, hold time expired (VRF: default)
RP/0/RSP0/CPU0:Jun 30 03:27:55.337 : shelfmgr[395]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/0/0 A9K-MPA-20X1GE state:UNPOWERED
RP/0/RSP0/CPU0:Jun 30 03:27:55.449 : shelfmgr[395]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/0/0 A9K-MPA-20X1GE state:POWER UP
LC/0/0/CPU0 30 03:27:56.275 : lda_server[64]: %L2-SPA-5-STATE_CHANGE : SPA in bay 0 type A9K-MPA-20X1GE Initing
RP/0/RSP0/CPU0:Jun 30 03:27:57.309 : shelfmgr[395]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/0/0 A9K-MPA-20X1GE state:INITIALIZED
LC/0/0/CPU0:Jun 30 03:27:57.435 : kdscrape-lc[62]: KERNEL-DUMP: Copied LC1.130630-012501.e500 to /harddisk:/dumper
LC/0/0/CPU0:Jun 30 03:27:57.542 : kdscrape-lc[62]: KERNEL-DUMP: Copied LC1.130630-012501.crashinfo.by.p40x0mc to
get sh reboot history location o/x/cpu0 and check why the card went down. also try getting the crash files from hardisk to check why the LC went down
07-03-2013 06:00 AM
Hi Please have a look at this:
RP/0/RSP0/CPU0:HOSTNAME#sh reboot history location 0/0/CPU0
Wed Jul 3 22:49:17.379 gmt
No Time Cause Code Reason
--------------------------------------------------------------------------------
01 Tue Jul 2 14:56:09 2013 0x2c000015 Cause: Fatal Eror Condition
Process: p40x0mc
02 Tue Jul 2 14:56:09 2013 0x0c000015 Cause: Fatal Eror Condition
Process: p40x0mc
03 Tue Jul 2 07:59:57 2013 0x2c000015 Cause: Fatal Eror Condition
Process: p40x0mc
04 Tue Jul 2 07:59:57 2013 0x0c000015 Cause: Fatal Eror Condition
Process: p40x0mc
05 Sun Jun 30 12:20:38 2013 0x2c000015 Cause: Fatal Eror Condition
Process: p40x0mc
06 Sun Jun 30 12:20:38 2013 0x0c000015 Cause: Fatal Eror Condition
Process: p40x0mc
07 Sun Jun 30 03:24:57 2013 0x2c000015 Cause: Fatal Eror Condition
Process: p40x0mc
08 Sun Jun 30 03:24:57 2013 0x0c000015 Cause: Fatal Eror Condition
Process: p40x0mc
07-03-2013 06:14 AM
Hi Also the Crash dump looks like this:
RP/0/RSP0/CPU0:HOSTNAME#more LC1.130630-012501.crashinfo.by.p40x0mc
LC/0/0/CPU0:Jun 30 03:24:57.075 : p40x0mc[74]: ====== L3 CACHE ERROR #1 ======
LC/0/0/CPU0:Jun 30 03:24:57.080 : p40x0mc[74]: CPC Registers: OFF CPC1 CPC2
LC/0/0/CPU0:Jun 30 03:24:57.084 : p40x0mc[74]: CPCERRDET* e40 0x80000008 0x00000000
LC/0/0/CPU0:Jun 30 03:24:57.088 : p40x0mc[74]: CPCCAPTECC e28 0xc9000009 0x00000000
LC/0/0/CPU0:Jun 30 03:24:57.093 : p40x0mc[74]: CPCERRINTEN e48 0x000000ec 0x000000ec
LC/0/0/CPU0:Jun 30 03:24:57.097 : p40x0mc[74]: CPCERRATTR e4c 0x7082011d 0x00000000
LC/0/0/CPU0:Jun 30 03:24:57.102 : p40x0mc[74]: CPCERRADDR e54 0x0fef1210 0x00000000
LC/0/0/CPU0:Jun 30 03:24:57.106 : p40x0mc[74]: CPCCSR0 000 0xc0000000 0xc0000000
LC/0/0/CPU0:Jun 30 03:24:57.110 : p40x0mc[74]: CPCCFG0 008 0x50b7c010 0x50b7c010
LC/0/0/CPU0:Jun 30 03:24:57.115 : p40x0mc[74]: CPCHDBCR0 f00 0x015e0000 0x015e0000
LC/0/0/CPU0:Jun 30 03:24:57.119 : p40x0mc[74]: CPCCAPTDTHI e20 0x01001001 0x00000000
LC/0/0/CPU0:Jun 30 03:24:57.124 : p40x0mc[74]: CPCCAPTDTLO e24 0x79d54656 0x00000000
LC/0/0/CPU0:Jun 30 03:24:57.128 : p40x0mc[74]: CPCPIR0x0 200 0xffffffff 0xffffffff
LC/0/0/CPU0:Jun 30 03:24:57.132 : p40x0mc[74]: CPCPAR0x0 208 0x000003fb 0x000003fb
LC/0/0/CPU0:Jun 30 03:24:57.137 : p40x0mc[74]: CPCPWR0x0 20c 0xffffffff 0xffffffff
LC/0/0/CPU0:Jun 30 03:24:57.140 : p40x0mc[74]: ====== DDR CONTROLLER ERROR #1 ======
LC/0/0/CPU0:Jun 30 03:24:57.144 : p40x0mc[74]: DDR Registers: OFF DDR1 DDR2
LC/0/0/CPU0:Jun 30 03:24:57.149 : p40x0mc[74]: ERR_DETECT* e40 0x80000008 0x00000000
LC/0/0/CPU0:Jun 30 03:24:57.153 : p40x0mc[74]: ERR_INT_EN e48 0x0000018d 0x0000018d
LC/0/0/CPU0:Jun 30 03:24:57.158 : p40x0mc[74]: CAPTURE_ATTRIBUTES e4c 0x10832001 0x00000000
LC/0/0/CPU0:Jun 30 03:24:57.162 : p40x0mc[74]: CAPTURE_ADDRESS e50 0x0fef1200 0x00000000
LC/0/0/CPU0:Jun 30 03:24:57.167 : p40x0mc[74]: CAPTURE_DATA_HI e20 0x03001001 0x00000000
LC/0/0/CPU0:Jun 30 03:24:57.171 : p40x0mc[74]: CAPTURE_DATA_LO e24 0x77a7f528 0x00000000
LC/0/0/CPU0:Jun 30 03:24:57.175 : p40x0mc[74]: CAPTURE_DATA_ECC e28 0x21212121 0x00000000
LC/0/0/CPU0:Jun 30 03:24:57.317 : p40x0mc[74]: EISR0: 0x00000110 [DDR1|CPC1]
LC/0/0/CPU0:Jun 30 03:24:57.321 : p40x0mc[74]: DDR1 ERR_DETECT : 0x80000008 [MME|MBE] Count = 1
LC/0/0/CPU0:Jun 30 03:24:57.327 : p40x0mc[74]: CPC1 CPCERRDET : 0x80000008 [MULLERR|DMBECCERR] Count = 1
LC/0/0/CPU0:Jun 30 03:24:57.330 : p40x0mc[74]: Shutting down node due to fatal error
LC/0/0/CPU0:Jun 30 03:24:59.343 : p40x0mc[74]: reboot_internal: Incomplete graceful reboot cleanup (Connection timed out)
LC/0/0/CPU0:Jun 30 03:24:59.343 : p40x0mc[74]: Sun Jun 30 03:24:57 2013:sync start
LC/0/0/CPU0:Jun 30 03:24:59.343 : p40x0mc[74]: Sun Jun 30 03:24:57 2013:sync end
LC/0/0/CPU0:Jun 30 03:24:59.343 : p40x0mc[74]: Sun Jun 30 03:24:57 2013:platform_reboot_op start
RP/0/RSP0/CPU0:HOSTNAME#more LC1.130630-102042.crashinfo.by.p40x0mc
LC/0/0/CPU0:Jun 30 01:27:00.002 : insthelper[61]: %INSTALL-INSTHELPER-7-PKG_DOWNLOAD : MBI running; starting software download
LC/0/0/CPU0:Jun 30 01:27:41.274 : sysmgr[90]: %OS-SYSMGR-5-NOTICE : Card is COLD started
LC/0/0/CPU0:Jun 30 01:27:41.757 : init[65540]: %OS-INIT-7-INSTALL_READY : total time 51.189 seconds
LC/0/0/CPU0:Jun 30 01:27:42.383 : sysmgr[352]: %OS-SYSMGR-6-INFO : Backup system manager is ready
LC/0/0/CPU0:Jun 30 01:27:56.275 : lda_server[64]: %L2-SPA-5-STATE_CHANGE : SPA in bay 0 type A9K-MPA-20X1GE Initing
LC/0/0/CPU0:Jun 30 01:27:57.435 : kdscrape-lc[62]: KERNEL-DUMP: Copied LC1.130630-012501.e500 to /harddisk:/dumper
LC/0/0/CPU0:Jun 30 01:27:57.542 : kdscrape-lc[62]: KERNEL-DUMP: Copied LC1.130630-012501.crashinfo.by.p40x0mc to /harddisk:/dumper
LC/0/0/CPU0:Jun 30 01:28:01.269 : pfm_node_lc[291]: %FABRIC-FIA-1-DATA_SB_SERDES_1_FAIL_0 : Set|fialc[139329]|Fabric Interface(0x103b000)|Data SB Serdes Link 1 Failure on FIA 0
LC/0/0/CPU0:Jun 30 01:28:01.271 : pfm_node_lc[291]: %FABRIC-FIA-1-DATA_SB_SERDES_0_FAIL_0 : Set|fialc[139329]|Fabric Interface(0x103b000)|Data SB Serdes Link 0 Failure on FIA 0
LC/0/0/CPU0:Jun 30 01:28:09.559 : prm_server_ty[303]: %PLATFORM-QOSHAL-6-QOS_MODE_CHANGE : Line-card QoS mode set to child-shaping-enabled.
LC/0/0/CPU0:Jun 30 01:28:09.563 : prm_server_ty[303]: %PLATFORM-QOSHAL-6-QOS_MODE_CHANGE : Line-card QoS mode set to lowburst-disabled.
LC/0/0/CPU0:Jun 30 01:28:10.011 : sysmgr[90]: %OS-SYSMGR-7-DEBUG : sysmgr_admin_plane_check:SYSMGR_PLANE_ADMIN Notification sent.
LC/0/0/CPU0:Jun 30 01:28:13.005 : syslog_dev[88]: hsbi_fpd_agent[198]: FPD ltrace_file_name => fpd-agent/hsbi_fpd_agent
LC/0/0/CPU0:Jun 30 03:28:15.980 : rsi_agent[318]: %OS-RSI_AGENT-6-CARD_ROLE_CHANGE : Based on the card configuration/type, the AFI IPv4 role of the card has changed from Invalid to Customer Facing
LC/0/0/CPU0:Jun 30 03:28:15.984 : rsi_agent[318]: %OS-RSI_AGENT-6-CARD_ROLE_CHANGE : Based on the card configuration/type, the AFI IPv6 role of the card has changed from Invalid to Not Interested
LC/0/0/CPU0:Jun 30 03:28:34.330 : pfm_node_lc[291]: %FABRIC-FIA-1-DATA_SB_SERDES_1_FAIL_0 : Clear|fialc[139329]|Fabric Interface(0x103b000)|Data SB Serdes Link 1 Failure on FIA 0
LC/0/0/CPU0:Jun 30 03:28:34.331 : pfm_node_lc[291]: %FABRIC-FIA-1-DATA_SB_SERDES_0_FAIL_0 : Clear|fialc[139329]|Fabric Interface(0x103b000)|Data SB Serdes Link 0 Failure on FIA 0
LC/0/0/CPU0:Jun 30 03:28:44.631 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface TenGigE0/0/2/3, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:44.632 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface TenGigE0/0/2/2, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:44.633 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface TenGigE0/0/2/1, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:44.633 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface TenGigE0/0/2/0, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:44.637 : cfgmgr-lc[134]: %MGBL-CONFIG-6-OIR_RESTORE : Configuration for node '0/0/2' has been restored.
LC/0/0/CPU0:Jun 30 03:28:47.602 : rsi_agent[318]: %OS-RSI_AGENT-6-CARD_ROLE_CHANGE : Based on the card configuration/type, the AFI IPv4 role of the card has changed from Customer Facing to Core Facing
LC/0/0/CPU0:Jun 30 03:28:47.603 : rsi_agent[318]: %OS-RSI_AGENT-6-CARD_ROLE_CHANGE : Based on the card configuration/type, the AFI IPv6 role of the card has changed from Not Interested to Core Facing
LC/0/0/CPU0:Jun 30 03:28:51.022 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/19, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:51.022 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/18, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:51.022 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/17, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:51.023 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/16, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:51.023 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/15, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:51.024 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/14, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:51.024 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/13, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:51.024 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/12, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:51.025 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/11, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:51.025 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/10, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:51.025 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/9, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:51.026 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/8, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:51.026 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/7, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:51.030 : ifmgr[201]: %PKT_INFRA-LINK-3-UPDOWN : #ALMLOC#: Interface GigabitEthernet0/0/0/6, changed state to Down
LC/0/0/CPU0:Jun 30 03:28:51.031 : ifmgr[201]: %PKT_INFRA-LINEPROTO-5-UPDOWN : #ALMLOC#: Line protocol on Interface GigabitEthernet0/0/0/6, changed state to Down
LC/0/0/CPU0:Jun 30 03:28:51.031 : ifmgr[201]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/5, changed state to Administratively Down
LC/0/0/CPU0:Jun 30 03:28:51.032 : ifmgr[201]: %PKT_INFRA-LINK-3-UPDOWN : #ALMLOC#: Interface GigabitEthernet0/0/0/4, changed state to Down
LC/0/0/CPU0:Jun 30 03:28:51.032 : ifmgr[201]: %PKT_INFRA-LINEPROTO-5-UPDOWN : #ALMLOC#: Line protocol on Interface GigabitEthernet0/0/0/4, changed state to Down
LC/0/0/CPU0:Jun 30 03:28:51.032 : ifmgr[201]: %PKT_INFRA-LINK-3-UPDOWN : #ALMLOC#: Interface GigabitEthernet0/0/0/3, changed state to Down
LC/0/0/CPU0:Jun 30 03:28:51.032 : ifmgr[201]: %PKT_INFRA-LINEPROTO-5-UPDOWN : #ALMLOC#: Line protocol on Interface GigabitEthernet0/0/0/3, changed state to Down
LC/0/0/CPU0:Jun 30 03:28:51.033 : ifmgr[201]: %PKT_INFRA-LINK-3-UPDOWN : #ALMLOC#: Interface GigabitEthernet0/0/0/2, changed state to Down
LC/0/0/CPU0:Jun 30 03:28:51.034 : ifmgr[201]: %PKT_INFRA-LINEPROTO-5-UPDOWN : #ALMLOC#: Line protocol on Interface GigabitEthernet0/0/0/2, changed state to Down
LC/0/0/CPU0:Jun 30 03:28:51.034 : ifmgr[201]: %PKT_INFRA-LINK-3-UPDOWN : #ALMLOC#: Interface GigabitEthernet0/0/0/1, changed state to Down
LC/0/0/CPU0:Jun 30 03:28:51.034 : ifmgr[201]: %PKT_INFRA-LINEPROTO-5-UPDOWN : #ALMLOC#: Line protocol on Interface GigabitEthernet0/0/0/1, changed state to Down
LC/0/0/CPU0:Jun 30 03:28:51.035 : ifmgr[201]: %PKT_INFRA-LINK-3-UPDOWN : #ALMLOC#: Interface GigabitEthernet0/0/0/0, changed state to Down
LC/0/0/CPU0:Jun 30 03:28:51.035 : ifmgr[201]: %PKT_INFRA-LINEPROTO-5-UPDOWN : #ALMLOC#: Line protocol on Interface GigabitEthernet0/0/0/0, changed state to Down
LC/0/0/CPU0:Jun 30 03:28:51.043 : cfgmgr-lc[134]: %MGBL-CONFIG-6-OIR_RESTORE : Configuration for node '0/0/0' has been restored.
LC/0/0/CPU0:Jun 30 03:28:57.029 : rsi_agent[318]: %OS-RSI_AGENT-6-CARD_ROLE_CHANGE : Based on the card configuration/type, the AFI IPv4 role of the card has changed from Core Facing to Standard
LC/0/0/CPU0:Jun 30 03:28:57.030 : rsi_agent[318]: %OS-RSI_AGENT-6-CARD_ROLE_CHANGE : Based on the card configuration/type, the AFI IPv6 role of the card has changed from Core Facing to Standard
LC/0/0/CPU0:Jun 30 03:28:58.163 : ifmgr[201]: %PKT_INFRA-LINK-3-UPDOWN : #ALMLOC#: Interface GigabitEthernet0/0/0/6, changed state to Up
LC/0/0/CPU0:Jun 30 03:28:58.165 : ifmgr[201]: %PKT_INFRA-LINEPROTO-5-UPDOWN : #ALMLOC#: Line protocol on Interface GigabitEthernet0/0/0/6, changed state to Up
LC/0/0/CPU0:Jun 30 03:29:03.825 : ifmgr[201]: %PKT_INFRA-LINK-3-UPDOWN : #ALMLOC#: Interface GigabitEthernet0/0/0/0, changed state to Up
LC/0/0/CPU0:Jun 30 03:29:03.827 : ifmgr[201]: %PKT_INFRA-LINEPROTO-5-UPDOWN : #ALMLOC#: Line protocol on Interface GigabitEthernet0/0/0/0, changed state to Up
LC/0/0/CPU0:Jun 30 03:29:03.840 : ifmgr[201]: %PKT_INFRA-LINK-3-UPDOWN : #ALMLOC#: Interface GigabitEthernet0/0/0/1, changed state to Up
LC/0/0/CPU0:Jun 30 03:29:03.841 : ifmgr[201]: %PKT_INFRA-LINEPROTO-5-UPDOWN : #ALMLOC#: Line protocol on Interface GigabitEthernet0/0/0/1, changed state to Up
LC/0/0/CPU0:Jun 30 03:29:03.855 : ifmgr[201]: %PKT_INFRA-LINK-3-UPDOWN : #ALMLOC#: Interface GigabitEthernet0/0/0/2, changed state to Up
LC/0/0/CPU0:Jun 30 03:29:03.857 : ifmgr[201]: %PKT_INFRA-LINEPROTO-5-UPDOWN : #ALMLOC#: Line protocol on Interface GigabitEthernet0/0/0/2, changed state to Up
LC/0/0/CPU0:Jun 30 03:29:04.016 : ifmgr[201]: %PKT_INFRA-LINK-3-UPDOWN : #ALMLOC#: Interface GigabitEthernet0/0/0/4, changed state to Up
LC/0/0/CPU0:Jun 30 03:29:04.018 : ifmgr[201]: %PKT_INFRA-LINEPROTO-5-UPDOWN : #ALMLOC#: Line protocol on Interface GigabitEthernet0/0/0/4, changed state to Up
LC/0/0/CPU0:Jun 30 03:29:05.811 : pfm_node_lc[291]: %PLATFORM-SFP-3-HIGH_RX_POWER_WARNING : Set|envmon_lc[143443]|SFP(0x1029001)|Port_0/01
LC/0/0/CPU0:Jun 30 03:29:05.811 : pfm_node_lc[291]: %PLATFORM-SFP-3-HIGH_RX_POWER_WARNING : Set|envmon_lc[143443]|SFP(0x1029002)|Port_0/02
LC/0/0/CPU0:Jun 30 03:29:06.914 : ifmgr[201]: %PKT_INFRA-LINK-3-UPDOWN : #ALMLOC#: Interface GigabitEthernet0/0/0/3, changed state to Up
LC/0/0/CPU0:Jun 30 03:29:06.915 : ifmgr[201]: %PKT_INFRA-LINEPROTO-5-UPDOWN : #ALMLOC#: Line protocol on Interface GigabitEthernet0/0/0/3, changed state to Up
LC/0/0/CPU0:Jun 30 12:20:38.138 : p40x0mc[74]: ====== DDR CONTROLLER ERROR #1 ======
LC/0/0/CPU0:Jun 30 12:20:38.142 : p40x0mc[74]: DDR Registers: OFF DDR1 DDR2
LC/0/0/CPU0:Jun 30 12:20:38.146 : p40x0mc[74]: ERR_DETECT* e40 0x80000008 0x00000000
LC/0/0/CPU0:Jun 30 12:20:38.151 : p40x0mc[74]: ERR_INT_EN e48 0x0000018d 0x0000018d
LC/0/0/CPU0:Jun 30 12:20:38.155 : p40x0mc[74]: CAPTURE_ATTRIBUTES e4c 0x50852001 0x00000000
LC/0/0/CPU0:Jun 30 12:20:38.159 : p40x0mc[74]: CAPTURE_ADDRESS e50 0x0ec80810 0x00000000
LC/0/0/CPU0:Jun 30 12:20:38.164 : p40x0mc[74]: CAPTURE_DATA_HI e20 0x7ccf5002 0x00000000
LC/0/0/CPU0:Jun 30 12:20:38.168 : p40x0mc[74]: CAPTURE_DATA_LO e24 0x00000264 0x00000000
LC/0/0/CPU0:Jun 30 12:20:38.173 : p40x0mc[74]: CAPTURE_DATA_ECC e28 0x39393939 0x00000000
LC/0/0/CPU0:Jun 30 12:20:38.284 : p40x0mc[74]: EISR0: 0x00000100 [DDR1]
LC/0/0/CPU0:Jun 30 12:20:38.289 : p40x0mc[74]: DDR1 ERR_DETECT : 0x80000008 [MME|MBE] Count = 1
LC/0/0/CPU0:Jun 30 12:20:38.292 : p40x0mc[74]: Shutting down node due to fatal error
LC/0/0/CPU0:Jun 30 12:20:40.305 : p40x0mc[74]: reboot_internal: Incomplete graceful reboot cleanup (Connection timed out)
LC/0/0/CPU0:Jun 30 12:20:40.305 : p40x0mc[74]: Sun Jun 30 12:20:38 2013:sync start
LC/0/0/CPU0:Jun 30 12:20:40.305 : p40x0mc[74]: Sun Jun 30 12:20:38 2013:sync end
LC/0/0/CPU0:Jun 30 12:20:40.305 : p40x0mc[74]: Sun Jun 30 12:20:38 2013:platform_reboot_op start
I am getting cisco to help with replacing the card, but what are the odds of the issue coming back? can we say the LC is faulty?
07-03-2013 08:07 AM
This is a known issue...we even have a FN out for this...replace your line card for sure.
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: