Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements

Welcome to Cisco Support Community. We would love to have your feedback.

For an introduction to the new site, click here. And see here for current known issues.

New Member

ASR1004-ESP10 crashed

hi.

my ASR 1004 was crashed and I have loggs from the router.

please help me to know what happend and what could be the solution(software problem or herdware) ?

log :



Feb 17 22:26:34.708: %IOSXE-3-PLATFORM: F0: cpp_cdm: CPP crashed, core file /tmp/corelink/bb_ESP_0_cpp-mcplo-ucode_021812002634.core.gz
Feb 17 22:26:35.477: %ASR1000_OIR-6-OFFLINECARD: Card (fp) offline in slot F0
Feb 17 22:26:40.569: %CPPDRV-3-LOCKDOWN: F0: cpp_cp:  CPP10(0) CPP Driver LOCKDOWN due to fatal error.
Feb 17 22:26:47.044: %DUAL-5-NBRCHANGE: EIGRP-IPv4 1: Neighbor x.x.x.x (GigabitEthernet1/0/3) is down: holding time expired
Feb 17 22:26:48.034: %DUAL-5-NBRCHANGE: EIGRP-IPv4 1: Neighbor x.x.x.x (GigabitEthernet1/0/4) is down: holding time expired
Feb 17 22:27:04.451: %CPPHA-3-CDMDONE: F0: cpp_ha:  CPP 0 microcode crashdump creation completed.
Feb 17 22:27:04.454: %IOSXE-6-PLATFORM: F0: cpp_ha: Shutting down CPP MDM while client(s) still connected
Feb 17 22:27:04.454: %IOSXE-6-PLATFORM: F0: cpp_ha: Shutting down CPP CDM while client(s) still connected
Feb 17 22:27:04.491: %IOSXE-6-PLATFORM: F0: cpp_cdm: Shutting down CPP MDM while client(s) still connected
Feb 17 22:27:04.827: %PMAN-3-PROCHOLDDOWN: F0: pman.sh:  The process cpp_ha_top_level_server has been helddown (rc 69)
Feb 17 22:27:04.863: %PMAN-3-PROCHOLDDOWN: F0: pman.sh:  The process cpp_cdm_svr has been helddown (rc 69)
Feb 17 22:27:13.290: %OSPFv3-5-ADJCHG: Process 378, Nbr .x.x.x.x on GigabitEthernet1/0/2 from FULL to DOWN, Neighbor Down: Dead timer expired
Feb 17 22:27:13.291: %OSPF-5-ADJCHG: Process 378, Nbr x.x.x.x on GigabitEthernet1/0/2 from FULL to DOWN, Neighbor Down: Dead timer expired
Feb 17 22:27:13.835: %OSPF-5-ADJCHG: Process 378, Nbr x.x.x.x on GigabitEthernet1/0/1 from FULL to DOWN, Neighbor Down: Dead timer expired
Feb 17 22:27:14.258: %OSPFv3-5-ADJCHG: Process 378, Nbr x.x.x.x on GigabitEthernet1/0/1 from FULL to DOWN, Neighbor Down: Dead timer expired
Feb 17 22:27:34.644: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to down
Feb 17 22:27:55.678: %PIM-5-NBRCHG: VRF tzfat: neighbor x.x.x.x DOWN on interface GigabitEthernet1/0/4 non DR
Feb 17 22:28:06.678: %PIM-5-NBRCHG: neighbor x.x.x.x DOWN on interface GigabitEthernet1/0/1 non DR
Feb 17 22:28:11.678: %PIM-5-NBRCHG: VRF tzfat: neighbor x.x.x.x DOWN on interface GigabitEthernet1/0/3 DR
Feb 17 22:28:11.678: %PIM-5-DRCHG: VRF tzfat: DR change from neighbor x.x.x.x to x.x.x.x on interface GigabitEthernet1/0/3
Feb 17 22:28:11.678: %PIM-5-NBRCHG: neighbor x.x.x.x DOWN on interface GigabitEthernet1/0/2 non DR
Feb 17 22:28:36.107: %ASR1000_OIR-6-ONLINECARD: Card (fp) online in slot F0
Feb 17 22:28:53.739: %CPPHA-7-START: F0: cpp_ha:  CPP 0 preparing image /tmp/sw/fp/0/0/fp/mount/usr/cpp/bin/qfp-ucode-esp10
Feb 17 22:28:54.126: %CPPHA-7-START: F0: cpp_ha:  CPP 0 startup init image /tmp/sw/fp/0/0/fp/mount/usr/cpp/bin/qfp-ucode-esp10
Feb 17 22:28:59.086: %CPPHA-7-START: F0: cpp_ha:  CPP 0 running init image /tmp/sw/fp/0/0/fp/mount/usr/cpp/bin/qfp-ucode-esp10
Feb 17 22:28:59.348: %CPPHA-7-READY: F0: cpp_ha:  CPP 0 loading and initialization complete
Feb 17 22:29:00.854: %IOSXE-6-PLATFORM: F0: cpp_cp: Process CPP_PFILTER_EA_EVENT__API_CALL__REGISTER
Feb 17 22:29:02.032: %DUAL-5-NBRCHANGE: EIGRP-IPv4 1: Neighbor x.x.x.x (GigabitEthernet1/0/4) is up: new adjacency
Feb 17 22:29:02.457: %DUAL-5-NBRCHANGE: EIGRP-IPv4 1: Neighbor x.x.x.x (GigabitEthernet1/0/3) is up: new adjacency
Feb 17 22:29:03.366: %OSPFv3-5-ADJCHG: Process 378, Nbr x.x.x.x on GigabitEthernet1/0/2 from LOADING to FULL, Loading Done
Feb 17 22:29:03.367: %OSPF-5-ADJCHG: Process 378, Nbr x.x.x.x on GigabitEthernet1/0/2 from LOADING to FULL, Loading Done
Feb 17 22:29:03.936: %OSPF-5-ADJCHG: Process 378, Nbr x.x.x.x on GigabitEthernet1/0/1 from LOADING to FULL, Loading Done
Feb 17 22:29:04.635: %OSPFv3-5-ADJCHG: Process 378, Nbr x.x.x.x on GigabitEthernet1/0/1 from LOADING to FULL, Loading Done
Feb 17 22:29:06.059: %PIM-5-NBRCHG: neighbor x.x.x.x UP on interface GigabitEthernet1/0/1
Feb 17 22:29:06.853: %PIM-5-NBRCHG: VRF tzfat: neighbor x.x.x.x UP on interface GigabitEthernet1/0/4
Feb 17 22:29:09.118: %PIM-5-NBRCHG: VRF tzfat: neighbor x.x.x.x UP on interface GigabitEthernet1/0/3
Feb 17 22:29:09.120: %PIM-5-DRCHG: VRF tzfat: DR change from neighbor x.x.x.x to x.x.x.x on interface GigabitEthernet1/0/3
Feb 17 22:29:14.645: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to up

Feb 17 22:26:34.556: %CPPDRV-3-LOCKDOWN: F0: fman_fp_image:  CPP10(0) CPP Driver LOCKDOWN due to fatal error.

Feb 17 22:26:34.583: %CPPOSLIB-3-ERROR_NOTIFY: F0: /tmp/sw/fp/0/0/fp/mount/usr/cpp/bin/cpp_driver[6109]:  cpp_driver-0 encountered an error -Traceback= 1#18f9da19b6c01995eb8088a2798220bc   errmsg:F5A9000+2160 cpp_common_os:FBE1000+BA30 cpp_common_os:FBE1000+18DF8 cpp_drv_cmn:FD34000+6A3EC cpp_drv_cmn:FD34000+74C4C :10000000+CF04 cpp_drv_cmn:FD34000+6C2C4 cpp_drv_cmn:FD34000+60570 time:F26D000+5744 evlib:F8FF000+DABC evlib:F8FF000+FFC4 :10000000+623C c:EBCB000+1E938 c:EBCB000+1EAE0

Feb 17 22:26:34.588: %CPPHA-3-FAULT: F0: cpp_ha:  CPP:0 desc:DPE5_CPE_CPE_DPE_INT_SET_0_LEAF_INT_INT_PHY_ERROR det:DRVR(interrupt) class:OTHER sev:FATAL id:1493 cppstate:STOPPED res:UNKNOWN flags:0x7 cdmflags:0x1

Feb 17 22:26:34.588: %CPPHA-3-FAULTCRASH: F0: cpp_ha:  CPP 0 unresolved fault detected, initiating crash dump.

Feb 17 22:26:34.602: %CPPHA-3-FAULT: F0: cpp_ha:  CPP:0 desc:DPE5_CPE_CPE_DPE_DUI_LEAF_INT_INT_DUI_CHN0_DRAM_MBE det:DRVR(interrupt) class:MBE sev:FATAL id:1522 cppstate:STOPPED res:SUCCESS flags:0x7 cdmflags:0x1

Feb 17 22:26:34.602: %CPPHA-3-FAULTCRASH: F0: cpp_ha:  CPP 0 unresolved fault detected, initiating crash dump.

Feb 17 22:26:34.617: %CPPDRV-6-INTR: F0: /tmp/sw/fp/0/0/fp/mount/usr/cpp/bin/cpp_driver[6109]:  CPP10(0) Interrupt : Last 16 Interrupts Since Boot

Feb 17 22:26:34.617: %CPPDRV-6-INTR: F0: /tmp/sw/fp/0/0/fp/mount/usr/cpp/bin/cpp_driver[6109]:  CPP10(0) Interrupt : 

Feb 16 01:38:31.143161: :GAL_GAL_CSR_IPM_IF_GAL_IPM_IF_LEAF_INT_INT_IPM_ERR

Feb 17 22:26:34.617: %CPPDRV-6-INTR: F0: /tmp/sw/fp/0/0/fp/mount/usr/cpp/bin/cpp_driver[6109]:  CPP10(0) Interrupt :Â

Feb 17 22:34:34.451807: :DPE5_CPE_CPE_DPE_DUI_LEAF_INT_INT_DUI_CHN0_DRAM_SBE

Feb 17 22:26:34.617: %CPPDRV-6-INTR: F0: /tmp/sw/fp/0/0/fp/mount/usr/cpp/bin/cpp_driver[6109]:  CPP10(0) Interrupt :Â

Feb 17 22:34:34.457928: :DPE5_CPE_CPE_DPE_DUI_LEAF_INT_INT_DUI_CHN0_DRAM_SBE

Feb 17 22:26:34.617: %CPPDRV-6-INTR: F0: /tmp/sw/fp/0/0/fp/mount/usr/cpp/bin/cpp_driver[6109]:  CPP10(0) Interrupt :Â

Feb 17 22:56:58.451815: :DPE5_CPE_CPE_DPE_DUI_LEAF_INT_INT_DUI_CHN0_DRAM_SBE

Feb 17 22:26:34.617: %CPPDRV-6-INTR: F0: /tmp/sw/fp/0/0/fp/mount/usr/cpp/bin/cpp_driver[6109]:  CPP10(0) Interrupt :Â

Feb 17 22:56:58.457335: :DPE5_CPE_CPE_DPE_DUI_LEAF_INT_INT_DUI_CHN0_DRAM_SBE

Feb 17 22:26:34.617: %CPPDRV-6-INTR: F0: /tmp/sw/fp/0/0/fp/mount/usr/cpp/bin/cpp_driver[6109]:  CPP10(0) Interrupt :Â

Feb 17 23:19:22.451792: :DPE5_CPE_CPE_DPE_DUI_LEAF_INT_INT_DUI_CHN0_DRAM_SBE

Feb 17 22:26:34.617: %CPPDRV-6-INTR: F0: /tmp/sw/fp/0/0/fp/mount/usr/cpp/bin/cpp_driver[6109]:  CPP10(0) Interrupt : 

Feb 17 23:19:22.457337: :DPE5_CPE_CPE_DPE_DUI_LEAF_INT_INT_DUI_CHN0_DRAM_SBE

Feb 17 22:26:34.618: %CPPDRV-6-INTR: F0: /tmp/sw/fp/0/0/fp/mount/usr/cpp/bin/cpp_driver[6109]:  CPP10(0) Interrupt : 

Feb 18 00:26:34.587948: :HALT:DPE5_CPE_CPE_DPE_INT_SET_0_LEAF_INT_INT_PHY_ERROR

Feb 17 22:26:34.618: %CPPDRV-6-INTR: F0: /tmp/sw/fp/0/0/fp/mount/usr/cpp/bin/cpp_driver[6109]:  CPP10(0) Interrupt : 

Feb 18 00:26:34.587948: :HALT:DPE5_CPE_CPE_DPE_DUI_LEAF_INT_INT_DUI_CHN0_DRAM_MBE

Feb 17 22:26:34.618: %CPPDRV-6-INTR: F0: /tmp/sw/fp/0/0/fp/mount/usr/cpp/bin/cpp_driver[6109]:  CPP10(0) Interrupt : 

Feb 18 00:26:34.587948: :HEDP_HED_HALTED_IN_63_0_LEAF_INT_INT_HALTED10

Feb 17 22:26:34.618: %CPPDRV-6-INTR: F0: /tmp/sw/fp/0/0/fp/mount/usr/cpp/bin/cpp_driver[6109]:  CPP10(0) Interrupt : 

Feb 18 00:26:34.587948: :HOT:INFP_INF_IDMU_LEAF_INT_INT_TRANSACTION_TIMEOUT

Feb 17 22:26:34.708: %IOSXE-3-PLATFORM: F0: cpp_cdm: CPP crashed, core file /tmp/corelink/bb_ESP_0_cpp-mcplo-ucode_021812002634.core.gz

Feb 17 22:26:35.477: %ASR1000_OIR-6-OFFLINECARD: Card (fp) offline in slot F0

Feb 17 22:26:40.569: %CPPDRV-3-LOCKDOWN: F0: cpp_cp:  CPP10(0) CPP Driver LOCKDOWN due to fatal error.

Feb 17 22:26:47.044: %DUAL-5-NBRCHANGE: EIGRP-IPv4 1: Neighbor x.x.x.x (GigabitEthernet1/0/3) is down: holding time expired

Feb 17 22:26:48.034: %DUAL-5-NBRCHANGE: EIGRP-IPv4 1: Neighbor x.x.x.x (GigabitEthernet1/0/4) is down: holding time expired

Feb 17 22:27:04.451: %CPPHA-3-CDMDONE: F0: cpp_ha:  CPP 0 microcode crashdump creation completed.

Feb 17 22:27:04.454: %IOSXE-6-PLATFORM: F0: cpp_ha: Shutting down CPP MDM while client(s) still connected

Feb 17 22:27:04.454: %IOSXE-6-PLATFORM: F0: cpp_ha: Shutting down CPP CDM while client(s) still connected

Feb 17 22:27:04.491: %IOSXE-6-PLATFORM: F0: cpp_cdm: Shutting down CPP MDM while client(s) still connected

Feb 17 22:27:04.827: %PMAN-3-PROCHOLDDOWN: F0: pman.sh:  The process cpp_ha_top_level_server has been helddown (rc 69)

Feb 17 22:27:04.863: %PMAN-3-PROCHOLDDOWN: F0: pman.sh:  The process cpp_cdm_svr has been helddown (rc 69)

Feb 17 22:27:13.290: %OSPFv3-5-ADJCHG: Process 378, Nbr x.x.x.x on GigabitEthernet1/0/2 from FULL to DOWN, Neighbor Down: Dead timer expired

Feb 17 22:27:13.291: %OSPF-5-ADJCHG: Process 378, Nbr x.x.x.x on GigabitEthernet1/0/2 from FULL to DOWN, Neighbor Down: Dead timer expired

Feb 17 22:27:13.835: %OSPF-5-ADJCHG: Process 378, Nbr x.x.x.x on GigabitEthernet1/0/1 from FULL to DOWN, Neighbor Down: Dead timer expired

Feb 17 22:27:14.258: %OSPFv3-5-ADJCHG: Process 378, Nbr x.x.x.x on GigabitEthernet1/0/1 from FULL to DOWN, Neighbor Down: Dead timer expired

Feb 17 22:27:34.644: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to down

Feb 17 22:27:55.678: %PIM-5-NBRCHG: VRF tzfat: neighbor x.x.x.x DOWN on interface GigabitEthernet1/0/4 non DR

Feb 17 22:28:06.678: %PIM-5-NBRCHG: neighbor x.x.x.x DOWN on interface GigabitEthernet1/0/1 non DR

Feb 17 22:28:11.678: %PIM-5-NBRCHG: VRF tzfat: neighbor x.x.x.x DOWN on interface GigabitEthernet1/0/3 DR

Feb 17 22:28:11.678: %PIM-5-DRCHG: VRF tzfat: DR change from neighbor x.x.x.x to x.x.x.x on interface GigabitEthernet1/0/3

Feb 17 22:28:11.678: %PIM-5-NBRCHG: neighbor x.x.x.x DOWN on interface GigabitEthernet1/0/2 non DR

Feb 17 22:28:36.107: %ASR1000_OIR-6-ONLINECARD: Card (fp) online in slot F0

Feb 17 22:28:53.739: %CPPHA-7-START: F0: cpp_ha:  CPP 0 preparing image /tmp/sw/fp/0/0/fp/mount/usr/cpp/bin/qfp-ucode-esp10

Feb 17 22:28:54.126: %CPPHA-7-START: F0: cpp_ha:  CPP 0 startup init image /tmp/sw/fp/0/0/fp/mount/usr/cpp/bin/qfp-ucode-esp10

Feb 17 22:28:59.086: %CPPHA-7-START: F0: cpp_ha:  CPP 0 running init image /tmp/sw/fp/0/0/fp/mount/usr/cpp/bin/qfp-ucode-esp10

Feb 17 22:28:59.348: %CPPHA-7-READY: F0: cpp_ha:  CPP 0 loading and initialization complete

Feb 17 22:29:00.854: %IOSXE-6-PLATFORM: F0: cpp_cp: Process CPP_PFILTER_EA_EVENT__API_CALL__REGISTER

Feb 17 22:29:02.032: %DUAL-5-NBRCHANGE: EIGRP-IPv4 1: Neighbor x.x.x.x (GigabitEthernet1/0/4) is up: new adjacency

Feb 17 22:29:02.457: %DUAL-5-NBRCHANGE: EIGRP-IPv4 1: Neighbor x.x.x.x (GigabitEthernet1/0/3) is up: new adjacency

Feb 17 22:29:03.366: %OSPFv3-5-ADJCHG: Process 378, Nbr x.x.x.x on GigabitEthernet1/0/2 from LOADING to FULL, Loading Done

Feb 17 22:29:03.367: %OSPF-5-ADJCHG: Process 378, Nbr x.x.x.x on GigabitEthernet1/0/2 from LOADING to FULL, Loading Done

Feb 17 22:29:03.936: %OSPF-5-ADJCHG: Process 378, Nbr x.x.x.x on GigabitEthernet1/0/1 from LOADING to FULL, Loading Done

Feb 17 22:29:04.635: %OSPFv3-5-ADJCHG: Process 378, Nbr x.x.x.x on GigabitEthernet1/0/1 from LOADING to FULL, Loading Done

Feb 17 22:29:06.059: %PIM-5-NBRCHG: neighbor x.x.x.x UP on interface GigabitEthernet1/0/1

Feb 17 22:29:06.853: %PIM-5-NBRCHG: VRF tzfat: neighbor x.x.x.x UP on interface GigabitEthernet1/0/4

Feb 17 22:29:09.118: %PIM-5-NBRCHG: VRF tzfat: neighbor x.x.x.x UP on interface GigabitEthernet1/0/3

Feb 17 22:29:09.120: %PIM-5-DRCHG: VRF tzfat: DR change from neighbor x.x.x.x to x.x.x.x on interface GigabitEthernet1/0/3

Feb 17 22:29:14.645: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to up

Feb 17 22:29:21.908: %PIM-5-NBRCHG: neighbor x.x.x.x UP on interface GigabitEthernet1/0/2Â

2 REPLIES

ASR1004-ESP10 crashed

hi,

we had a similar issue just yesterday. kindly post your show platform output.

we had to reboot the router for a temporarily fix and then RMAd the ESP card.

#sh platform

Chassis type: ASR1004           

Slot      Type                State                 Insert time (ago)

--------- ------------------- --------------------- -----------------

0         ASR1000-SIP10       ok                    50w2d       

0/0      SPA-5X1GE-V2        ok                    50w2d       

0/1      SPA-2X1GE-V2        ok                    50w2d       

R0        ASR1000-RP1         ok, active            50w2d       

F0        ASR1000-ESP10       booting               50w2d       

P0        ASR1004-PWR-AC      ok                    50w2d       

P1        ASR1004-PWR-AC      ok                    50w2d    

#sh plat

Chassis type: ASR1004            

Slot      Type                State                 Insert time (ago)

--------- ------------------- --------------------- -----------------

0         ASR1000-SIP10       ok                    17:51:09     

0/0      SPA-5X1GE-V2        ok                    17:49:16     

0/1      SPA-2X1GE-V2        ok                    17:49:16     

R0        ASR1000-RP1         ok, active            17:51:09     

F0        ASR1000-ESP10       ok, active            17:51:09   

P0        ASR1004-PWR-AC      ok                    17:50:21     

P1        ASR1004-PWR-AC      ok                    17:50:20     

Slot      CPLD Version        Firmware Version                       

--------- ------------------- ---------------------------------------

0         07091401            12.2(33r)XN2                       

R0        07062111            12.2(33r)XN2                       

F0        07091401            12.2(33r)XN2 

New Member

ASR1004-ESP10 crashed

Hello!

We've got a similar ESP-crash on our asr1004 with log:

CPPDRV-3-LOCKDOWN: F0: cpp_ha:  CPP10(0) CPP Driver LOCKDOWN due to fatal error

And also it created a core-dump. We assume that this is bug CSCte19641 (http://tools.cisco.com/Support/BugToolKit/search/getBugDetails.do?method=fetchBugDetails&bugId=CSCte19641), but there is no solution and no workaround. That's why I ask you, why did you do an RMA?

3225
Views
0
Helpful
2
Replies
CreatePlease login to create content