I did add below, and still get card bouncing status.tar file used is ASR9K-iosxr-px-4.2.1.tar

interface ServiceInfra1

ipv4 address 3.3.3.3 255.255.255.252

service-location 0/2/CPU0

commit

!

hw-module service cgn location 0/2/CPU0

commit

!

Then reload the card:

hw-mod location 0/2/cpu0 reload

/2/CPU0        A9K-ISM-100(LCP)          IOS XR PREP      PWR,NSHUT,MON

RP/0/RSP0/CPU0:CGN_PE#

RP/0/RSP0/CPU0:CGN_PE#sh platform

Thu Sep 27 13:51:00.145 CETDST

Node            Type                      State            Config State

-----------------------------------------------------------------------------

0/RSP0/CPU0     A9K-RSP440-SE(Active)     IOS XR RUN       PWR,NSHUT,MON

0/0/CPU0        A9K-MOD80-TR              IOS XR RUN       PWR,NSHUT,MON

0/0/0           A9K-MPA-4X10GE            OK               PWR,NSHUT,MON

0/2/CPU0        A9K-ISM-100(LCP)          MBI-RUNNING      PWR,NSHUT,MON

Extra log:

RP/0/RSP0/CPU0:Sep 27 13:50:40.406 : shelfmgr[386]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/2/CPU0 A9K-ISM-100 state:MBI-RUNNING

RP/0/RSP0/CPU0:Sep 27 13:53:31.862 : shelfmgr[386]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/2/CPU0 A9K-ISM-100 state:IOS XR RUN

RP/0/RSP0/CPU0:Sep 27 13:53:35.904 : shelfmgr[386]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/2/CPU0 A9K-ISM-100 state:BRINGDOWN

RP/0/RSP0/CPU0:Sep 27 13:53:41.988 : shelfmgr[386]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/2/CPU0 A9K-ISM-100 state:ROMMON

RP/0/RSP0/CPU0:Sep 27 13:54:13.486 : shelfmgr[386]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/2/CPU0 A9K-ISM-100 state:MBI-BOOTING

shelfmgr[386]: %PLATFORM-SHELFMGR-0-MAX_RESET_BRINGDOWN : Can not boot node 0/2/CPU0 A9K-ISM-100 due to multiple resets, putting it IN_RESET state. The probable cause is an unexpected event on the node or a failure in communication with the node. Please refer to the Cisco ASR 9000 System Error Message Reference Guide for further information if needed.

Probably the most valuable clue so far, will read up about this now.

show asic-errors all location 0/2/CPU0

Thu Sep 27 14:07:57.591 CETDST

List Asics operation failed: 0x40812A00, 'sysdb' detected the 'warning' condition 'Failed to send a request to a remote node, node doesn't exist or isn't available yet'

collect an output from "admin show ins ac sum"

attach to the ISM card

    term mon

    run attachCon 0/2/cpu0

from another session reboot this ISM "hw-module location 0/2/CPU0 reload"

collect all the logs from the first session during the module boot sequence.

Thanks again !

see attached log.

RP/0/RSP0/CPU0:CGN_PE#run attachCon 0/2/cpu0

Thu Sep 27 14:22:41.363 CETDST

attachCon: Starting console session to node 0/2/cpu0

attachCon: To quit console session type 'detach'

Current Baud 115200

Setting Baud to 9600

RP/0/RSP0/CPU0:Sep 27 14:23:33.893 : exec[65861]: %SECURITY-login-6-AUTHEN_SUCCESS : Successfully authenticated user 'sme' from '213.167.97.41' on 'vty1'

RP/0/RSP0/CPU0:Sep 27 14:23:52.409 : shelfmgr[386]: %PLATFORM-SHELFMGR-6-USER_RESET : Node 0/2/CPU0 is reset due to user reload request

RP/0/RSP0/CPU0:Sep 27 14:23:52.410 : shelfmgr[386]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/2/CPU0 A9K-ISM-100 state:IOS XR FAILURE

?RP/0/RSP0/CPU0:Sep 27 14:23:58.866 : canb-server[150]: %PLATFORM-CANB_SERVER-7-CBC_PRE_RESET_NOTIFICATION : Node 0/2/CPU0, reason : 0x5 0x0 0x0 0x0 

RP/0/RSP0/CPU0:Sep 27 14:23:58.867 : shelfmgr[386]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/2/CPU0 A9K-ISM-100 state:BRINGDOWN

RP/0/RSP0/CPU0:Sep 27 14:23:58.870 : invmgr[256]: %PLATFORM-INV-6-NODE_STATE_CHANGE : Node: 0/2/CPU0, state: BRINGDOWN

?RP/0/RSP0/CPU0:Sep 27 14:24:05.321 : canb-server[150]: %PLATFORM-CANB_SERVER-7-CBC_POST_RESET_NOTIFICATION : Node 0/2/CPU0, reason : 0x5 0x0 0x0 0x0 

RP/0/RSP0/CPU0:Sep 27 14:24:05.322 : shelfmgr[386]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/2/CPU0 A9K-ISM-100 state:ROMMON

Selecting ROMMON Image... B

DDR in Interleaved mode

POST 1 : PASSED : code 0 : DDR2 Memory

System Bootstrap, Version 1.2(20091201:235620) [ASR9K ROMMON],

Copyright (c) 1994-2009 by Cisco Systems, Inc.

Compiled Fri 08-Jan-10 16:18 by ashok

CPU Reset Reason = 0x0005

PPC 8641D (partnum 0x0003), Revision 0.2, (Core Version 2.20136)

M8641 CLKIN:   66 Mhz

Core Clock: 1333 Mhz

  MPX Clock:  533 Mhz

  LBC Clock:   33 Mhz

Board Type: 1000305

POST 2 : PASSED : code 0 : FPGA Flash Images

POST 3 : PASSED : code 0 : Slot ID/Board Type

Loading Field Programmable Devices:

FPGA 0-B IMG_INVALID : image: 0xfd800000 - 0xfe7fffff

FPGA 0-A PROGRAMMED  : image: 0xfe800028 - 0xff373c44, et: 558ms

FPGA 1-B IMG_INVALID : image: 0xfd800000 - 0xfe7fffff

FPGA 1-A PROGRAMMED  : image: 0xfe800028 - 0xff373c44, et: 558ms

passed fpga_prog_devices

Pass reseting devices

Main Board: 0x1000305, rev 0x3

      PLD: 0.1

  Bridge0: 0.25

  Bridge1: 0.25

      CBC: 18.5

Passed init Phy

PCI-E1: Ready as Root Complex

PCI-E2: Ready as Root Complex

Cannot detect the presence of RSP1.

ASR9K (8641D PPC) platform with 4032 Mb of main memory

    CARD_SLOT_NUMBER: 4

        CPU_INSTANCE: 1

MBI Validation starts ...RP/0/RSP0/CPU0:Sep 27 14:24:35.079 : ce_switch_srv[53]: %PLATFORM-CE_SWITCH-6-UPDN : Interface 6 (LC_Slot_2) is up

tsec_init_hw: configuring TSEC (port 0) for: 1GB, Full Duplex

tsec_init_interface: hardware initialization completed

Interface link changed state to UPRP/0/RSP0/CPU0:Sep 27 14:24:36.223 : ce_switch_srv[53]: %PLATFORM-CE_SWITCH-6-UPDN : Interface 6 (LC_Slot_2) is down

.

Interface link state up.

MBI validation sending request.

HIT CTRL-C to abort

RP/0/RSP0/CPU0:Sep 27 14:24:36.511 : ce_switch_srv[53]: %PLATFORM-CE_SWITCH-6-UPDN : Interface 6 (LC_Slot_2) is up

.RP/0/RSP0/CPU0:Sep 27 14:24:36.814 : shelfmgr[386]: %PLATFORM-SHELFMGR_HAL-6-BOOT_REQ_RECEIVED : Boot Request from 0/2/CPU0, RomMon Version: 1.2

mbi_val_prRP/0/RSP0/CPU0:Sep 27 14:24:36.815 : shelfmgr[386]: %PLATFORM-MBIMGR-7-IMAGE_VALIDATED : Remote location 0/2/CPU0: : MBI tftp:/disk0/asr9k-os-mbi-4.2.1/lc/mbiasr9k-lc.vm validated

ocess_packeRP/0/RSP0/CPU0:Sep 27 14:24:36.816 : shelfmgr[386]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/2/CPU0 A9K-ISM-100 state:MBI-BOOTING

t: received repsonse

Remote image to boot : tftp:/disk0/asr9k-os-mbi-4.2.1/lc/mbiasr9k-lc.vm

          IP_ADDRESS: 127.0.1.4

      IP_SUBNET_MASK: 255.255.0.0

     DEFAULT_GATEWAY: 127.0.1.0

         TFTP_SERVER: 127.0.1.0

           TFTP_FILE: /disk0/asr9k-os-mbi-4.2.1/lc/mbiasr9k-lc.vm

Performing tftpdnld

tsec_init_hw: configuring TSEC (port 0) for: 1GB, Full Duplex

tsec_init_interface: hardware initialization completed

tftp_process_packet: received OACK.

tftp_process_packet: Change blksize to 1400.

Receiving /disk0/asr9k-os-mbi-4.2.1/lc/mbiasr9k-lc.vm from 127.0.1.0 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

tftp_process_packet: last packet, block=12544, size=1148.

File reception completed.

Image size=17561348, TFTP download size=17561348

...................................................................

program load complete, entry point: 0x202148, size: 0x10be804

Attempting to start second CPU

Config = SMP, Running = SMP

Board type: 0x01000305

Card Capability = 0xffffffff

###########################################################################################

BSP: Board type : LC-ISM

tracelogger: starting tracing in background ring mode

tracelogger running with args: -startring -F 1 -F 2

              Restricted Rights Legend

Use, duplication, or disclosure by the Government is

subject to restrictions as set forth in subparagraph

(c) of the Commercial Computer Software - Restricted

Rights clause at FAR sec. 52.227-19 and subparagraph

(c) (1) (ii) of the Rights in Technical Data and Computer

Software clause at DFARS sec. 252.227-7013.

           cisco Systems, Inc.

           170 West Tasman Drive

           San Jose, California 95134-1706

Cisco IOS XR Software for the Cisco XR ASR9K, Version 4.2.1

Copyright (c) 2012 by Cisco Systems, Inc.

# RP/0/RSP0/CPU0:Sep 27 14:25:35.555 : shelfmgr[386]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/2/CPU0 A9K-ISM-100 state:MBI-RUNNING

FPD ltrace_file_name => fpd-agent/fialc

LC/0/2/CPU0:23: init[65540]: %OS-INIT-7-MBI_STARTED : total time 15.093 seconds

LC/0/2/CPU0:Sep 27 14:25:38.930 : insthelper[61]: %INSTALL-INSTHELPER-7-PKG_DOWNLOAD : MBI running; starting software download

LC/0/2/CPU0:Sep 27 14:25:56.800 : sysmgr[90]: %OS-SYSMGR-5-NOTICE : Card is COLD started 

LC/0/2/CPU0:Sep 27 14:25:57.056 : init[65540]: %OS-INIT-7-INSTALL_READY : total time 37.500 seconds

LC/0/2/CPU0:Sep 27 14:25:57.938 : sysmgr[328]: %OS-SYSMGR-6-INFO : Backup system manager is ready 

LC/0/2/CPU0:Sep 27 14:25:58.701 : spp[86]: %PKT_INFRA-spp-3-ERR : Node 'ptp_off_per_node' disposition 0 ('tsec2/tx') not found, using 'drop' instead 

LC/0/2/CPU0:Sep 27 14:25:58.707 : spp[86]: %PKT_INFRA-spp-3-ERR : Node 'ptp_off_rx_node' disposition 0 ('tsec2/tx') not found, using 'drop' instead 

LC/0/2/CPU0:Sep 27 14:26:01.021 : cgv6_services[141]: %PLATFORM-CGV6_SERVICES-7-GENERIC_INFO : Info: Starting Cgv6 Services Driver on location 0/2/CPU0

LC/0/2/CPU0:Sep 27 14:26:01.318 : sis[305]: Initializing APIV modules ...

LC/0/2/CPU0:Sep 27 14:26:01.481 : sis[305]: Registering for SysDB notifications ...

LC/0/2/CPU0:Sep 27 14:26:01.576 : sis[305]: Registering for SysDB SE READY notifications ...

LC/0/2/CPU0:Sep 27 14:26:01.877 : linux_maint_fpd_agent[245]: update_linux_maint_sw_version stat returned not ok

LC/0/2/CPU0:Sep 27 14:26:02.219 : linux_maint_fpd_agent[245]: %PLATFORM-UPGRADE_FPD-4-DOWN_REV : fpga2 instance 0 is down-rev (V0.01), upgrade to (V2.10). Use the "upgrade hw-module fpd" CLI in admin mode.

chipid:0 subid:0 vqi 0x10 vqivid 0x0 pt 0x0

chipid:0 subid:1 vqi 0x11 vqivid 0x0 pt 0x0

chipid:1 subid:0 vqi 0x12 vqivid 0x0 pt 0x0

chipid:1 subid:1 vqi 0x13 vqivid 0x0 pt 0x0

LC/0/2/CPU0:Sep 27 14:26:11.331 : fialc[170]: %PLATFORM-UPGRADE_FPD-4-DOWN_REV : fpga1 instance 0 is down-rev (V0.25), upgrade to (V0.30). Use the "upgrade hw-module fpd" CLI in admin mode.

LC/0/2/CPU0:Sep 27 14:26:11.331 : fialc[170]: %PLATFORM-UPGRADE_FPD-4-DOWN_REV : fpga1 instance 1 is down-rev (V0.25), upgrade to (V0.30). Use the "upgrade hw-module fpd" CLI in admin mode.

LC/0/2/CPU0:Sep 27 14:26:22.126 : umass-enum[91]: USB: /dev/lcdisk00: device not available, continuing

LC/0/2/CPU0:Sep 27 14:26:32.842 : sis[305]: Received SYSDB_EVENT_SET Notification (state 4) ...

LC/0/2/CPU0:Sep 27 14:26:32.843 : sysmgr[90]: %OS-SYSMGR-7-DEBUG : sysmgr_admin_plane_check:SYSMGR_PLANE_ADMIN Notification sent.  

LC/0/2/CPU0:Sep 27 14:26:33.159 : sync_agent[323]: %PLATFORM-FSYNC-6-UNKNOWN_BOARD_TYPE : Frequency Synchronization feature does not recognize hardware board type 0x1000305

LC/0/2/CPU0:Sep 27 14:26:33.202 : sync_agent[323]: %PLATFORM-FSYNC-6-NOT_SUPPORTED : Frequency Synchronization is not supported on this line card.

LC/0/2/CPU0:Sep 27 14:26:33.831 : syslog_dev[88]: hsbi_fpd_agent[193]: FPD ltrace_file_name => fpd-agent/hsbi_fpd_agent

LC/0/2/CPU0:Sep 27 14:26:34.560 : rsi_agent[294]: is_vrf_scale_configured: vrf-scale is not configured. Will continue with default scale

LC/0/2/CPU0:Sep 27 14:26:34.672 : BM-ADJ[128]: Registering for SysDB notifications ...

LC/0/2/CPU0:Sep 27 14:26:34.788 : rommon_fpd_agent[293]: fpga  instance 0 has image A programmed. Upgrade the possibly corrupt image B using "upgrade hw-module fpd" CLI in admin mode.

LC/0/2/CPU0:Sep 27 14:26:34.791 : rommon_fpd_agent[293]: fpga  instance 1 has image A programmed. Upgrade the possibly corrupt image B using "upgrade hw-module fpd" CLI in admin mode.

LC/0/2/CPU0:Sep 27 14:26:34.909 : svc_pm[342]: vkg_fib_ism_ral_funcs_init: Invalid Role - 0

LC/0/2/CPU0:Sep 27 14:26:34.913 : fib_mgr[171]: vkg_fib_ism_ral_funcs_init: Invalid Role - 0

LC/0/2/CPU0:Sep 27 14:26:35.623 : ipv6_io[233]: vkg_fib_ism_ral_funcs_init: Invalid Role - 0

LC/0/2/CPU0:Sep 27 14:26:35.638 : ipv4_io[230]: vkg_fib_ism_ral_funcs_init: Invalid Role - 0

LC/0/2/CPU0:Sep 27 14:26:35.768 : rsi_agent[294]: %OS-RSI_AGENT-6-CARD_ROLE_CHANGE : Based on the card configuration/type, the AFI IPv4 role of the card has changed from Invalid to Customer Facing

LC/0/2/CPU0:Sep 27 14:26:35.769 : rsi_agent[294]: %OS-RSI_AGENT-6-CARD_ROLE_CHANGE : Based on the card configuration/type, the AFI IPv6 role of the card has changed from Invalid to Not Interested

LC/0/2/CPU0:Sep 27 14:26:44.940 : svc_pm[342]: vkg_fib_ism_ral_funcs_init: Invalid Role - 0

LC/0/2/CPU0:Sep 27 14:26:44.942 : fib_mgr[171]: vkg_fib_ism_ral_funcs_init: Invalid Role - 0

LC/0/2/CPU0:Sep 27 14:26:45.636 : ipv6_io[233]: vkg_fib_ism_ral_funcs_init: Invalid Role - 0

LC/0/2/CPU0:Sep 27 14:26:45.650 : ipv4_io[230]: vkg_fib_ism_ral_funcs_init: Invalid Role - 0

LC/0/2/CPU0:Sep 27 14:26:50.031 : ipv4_io[230]: Initializing APIV modules ...

LC/0/2/CPU0:Sep 27 14:26:50.035 : ipv4_io[230]: Registering for SysDB notifications ...

LC/0/2/CPU0:Sep 27 14:26:54.953 : svc_pm[342]: vkg_fib_ism_ral_funcs_init: Invalid Role - 0

LC/0/2/CPU0:Sep 27 14:26:54.960 : fib_mgr[171]: vkg_fib_ism_ral_funcs_init: Invalid Role - 0

LC/0/2/CPU0:Sep 27 14:26:55.642 : ipv6_io[233]: vkg_fib_ism_ral_funcs_init: Invalid Role - 0

LC/0/2/CPU0:Sep 27 14:26:55.656 : ipv4_io[230]: vkg_fib_ism_ral_funcs_init: Invalid Role - 0

LC/0/2/CPU0:Sep 27 14:27:05.295 : svc_pm[342]: Initializing APIV modules ...

LC/0/2/CPU0:Sep 27 14:27:05.300 : svc_pm[342]: Registering for SysDB notifications ...

LC/0/2/CPU0:Sep 27 14:27:05.319 : fib_mgr[171]: Initializing APIV modules ...

LC/0/2/CPU0:Sep 27 14:27:05.325 : fib_mgr[171]: Registering for SysDB notifications ...

LC/0/2/CPU0:Sep 27 14:27:05.835 : netio[261]: vkg_fib_ism_ral_funcs_init: Invalid Role - 0

LC/0/2/CPU0:Sep 27 14:27:07.098 : vkg_l2fib_mac_cache[338]: Initializing L2 FIB MAC CAHE APIV modules ...

LC/0/2/CPU0:Sep 27 14:27:07.119 : sis[305]: Received SYSDB_EVENT_SET Notification (state 5) ...

LC/0/2/CPU0:Sep 27 14:27:07.156 : vkg_l2fib_mac_cache[338]:  L2fib Registering for SysDB notifications ...

LC/0/2/CPU0:Sep 27 14:27:08.830 : canb_upg_agt[133]: %PLATFORM-UPGRADE_FPD-4-DOWN_REV : cbc instance 0 is down-rev (V18.05), upgrade to (V18.06). Use the "upgrade hw-module fpd" CLI in admin mode.

LC/0/2/CPU0:Sep 27 14:27:09.359 : l2fib[238]: Initializing L2 FIB APIV modules ...

LC/0/2/CPU0:Sep 27 14:27:09.402 : l2fib[238]:  L2fib Registering for SysDB notifications ...

LC/0/2/CPU0:Sep 27 14:27:13.476 : vkg_l2fib_mac_cache[338]: Starting of L2FIB MAC CACHE APIV version negotiation ...

LC/0/2/CPU0:Sep 27 14:27:13.476 : fib_mgr[171]: Starting of APIV version negotiation ...

LC/0/2/CPU0:Sep 27 14:27:13.476 : ipv4_io[230]: Starting of APIV version negotiation ...

LC/0/2/CPU0:Sep 27 14:27:13.477 : BM-ADJ[128]: Starting of APIV version negotiation ...

LC/0/2/CPU0:Sep 27 14:27:13.477 : svc_pm[342]: Starting of APIV version negotiation ...

LC/0/2/CPU0:Sep 27 14:27:13.477 : sis[305]: Starting of APIV version negotiation (CREATE) ...

LC/0/2/CPU0:Sep 27 14:27:13.630 : ipv4_io[230]: %PLATFORM-PLAT_FIB_EA-3-ERR_STR : avsm fib ea apiv event func: Failed to bind with apiv ipc for tx: FAILURE (4110)  : pkg/bin/ipv4_io : (PID=147556) :  -Traceback= 4d8f9218 4c80b744 4c80d81c 4c261224 4c25efb4 4d9025c4 4c2c83c4

LC/0/2/CPU0:Sep 27 14:27:14.184 : inv_agent[204]: %PLATFORM-INV_AGENT-7-INFO : Sending SIM x86 inventory info 

LC/0/2/CPU0:Sep 27 14:27:15.843 : netio[261]: vkg_fib_ism_ral_funcs_init: Invalid Role - 0

LC/0/2/CPU0:Sep 27 14:27:18.760 : apiv_srvr[116]: %PLATFORM-APIV-5-API_INCOMPATIBLE : Message 8 in APIV module ISM_L2FIB_APIV is incompatible (reason: Not supported on the remote side)

LC/0/2/CPU0:Sep 27 14:27:18.761 : apiv_srvr[116]: %PLATFORM-APIV-3-MODULE_INCOMPATIBLE : APIV module ISM_L2FIB_APIV is incompatible

LC/0/2/CPU0:Sep 27 14:27:18.859 : apiv_srvr[116]: %PLATFORM-APIV-5-API_INCOMPATIBLE : Message 24 in APIV module APIV_SI is incompatible (reason: Not supported on the remote side)

LC/0/2/CPU0:Sep 27 14:27:18.859 : apiv_srvr[116]: %PLATFORM-APIV-3-MODULE_INCOMPATIBLE : APIV module APIV_SI is incompatible

LC/0/2/CPU0:Sep 27 14:27:19.006 : avsm_agent[125]: %APIV-3-API_INCOMPATIBLE : Message 24 in APIV module APIV_SI is incompatible(Not supported on local)

LC/0/2/CPU0:Sep 27 14:27:19.007 : avsm_agent[125]: %APIV-3-MODULE_INCOMPATIBLE : APIV_SI APIV module failed to negotiate versions due to incompatible version

LC/0/2/CPU0:Sep 27 14:27:19.761 : avsm_agent[125]: %ISM_L2_UFA-3-ERR :  L2UFA: apiv/ipcp init failed

LC/0/2/CPU0:Sep 27 14:27:22.010 : sis[305]: Received SYSDB_EVENT_SET Notification (state 8) ...

LC/0/2/CPU0:Sep 27 14:27:22.012 : sis[305]: Received SYSDB_EVENT_SET Notification (state 2) ...

LC/0/2/CPU0:Sep 27 14:27:22.015 : avsm_se_sm[126]: %PLATFORM-SIM_SE-3-SW_ERROR : SE_SM: An error (code: 15) occurred, recovery action 'shutdown service'  : pkg/bin/avsm_se_sm : (PID=135236) :  -Traceback= 40007484 4000304c 4c261224 4c25efb4 4000233c 4000f550

LC/0/2/CPU0:Sep 27 14:27:22.017 : avsm_se_sm[126]: %PLATFORM-SIM_SE-3-APIV_FAILURE : API Versioning Negotiation failure detected, SE will be out of service state.

LC/0/2/CPU0:Sep 27 14:27:22.017 : avsm_se_sm[126]: %PLATFORM-SIM_SE-5-STATE_CHANGE : Service Engine is in out of service state.

LC/0/2/CPU0:Sep 27 14:27:22.018 : sis[305]: Received SYSDB_EVENT_SET Notification (state 11) ...

LC/0/2/CPU0:Sep 27 14:27:23.886 : fib_mgr[171]: L3 FIB, FRR, LOOKUP & MISC APIV is up

LC/0/2/CPU0:Sep 27 14:27:23.889 : fib_mgr[171]: Initialized FIB State Machine!!!

LC/0/2/CPU0:Sep 27 14:27:24.077 : avsm_agent[125]: %PROCMGR-3-PROCESS_CRASH : process avsm_sia crashed, restarting it...

LC/0/2/CPU0:Sep 27 14:27:24.135 : apiv_srvr[116]: %PLATFORM-APIV-5-API_INCOMPATIBLE : Message 24 in APIV module APIV_SI is incompatible (reason: Not supported on the remote side)

LC/0/2/CPU0:Sep 27 14:27:24.135 : apiv_srvr[116]: %PLATFORM-APIV-3-MODULE_INCOMPATIBLE : APIV module APIV_SI is incompatible

FIB Init SM: state FIB_STATE_INIT event FIB_INIT_EVENT_NULL

FIB Init SM: state FIB_STATE_WAIT_FOR_INIT_ACK event FIB_INIT_EVENT_INIT

FIB Init SM: state FIB_STATE_SEND_INIT_ACK event FIB_INIT_EVENT_NULL

LC/0/2/CPU0:Sep 27 14:27:25.850 : netio[261]: vkg_fib_ism_ral_funcs_init: Invalid Role - 0

LC/0/2/CPU0:Sep 27 14:27:29.217 : avsm_agent[125]: %PROCMGR-3-PROCESS_CRASH : process avsm_sia crashed, restarting it...

LC/0/2/CPU0:Sep 27 14:27:29.275 : apiv_srvr[116]: %PLATFORM-APIV-5-API_INCOMPATIBLE : Message 24 in APIV module APIV_SI is incompatible (reason: Not supported on the remote side)

LC/0/2/CPU0:Sep 27 14:27:29.275 : apiv_srvr[116]: %PLATFORM-APIV-3-MODULE_INCOMPATIBLE : APIV module APIV_SI is incompatible

LC/0/2/CPU0:Sep 27 14:27:34.356 : avsm_agent[125]: %PROCMGR-3-PROCESS_CRASH : process avsm_sia crashed, restarting it...

LC/0/2/CPU0:Sep 27 14:27:34.415 : apiv_srvr[116]: %PLATFORM-APIV-5-API_INCOMPATIBLE : Message 24 in APIV module APIV_SI is incompatible (reason: Not supported on the remote side)

LC/0/2/CPU0:Sep 27 14:27:34.415 : apiv_srvr[116]: %PLATFORM-APIV-3-MODULE_INCOMPATIBLE : APIV module APIV_SI is incompatible

LC/0/2/CPU0:Sep 27 14:27:37.156 : arp[117]: vkg_fib_ism_ral_funcs_init: Invalid Role - 0

LC/0/2/CPU0:Sep 27 14:27:37.284 : l2fib[238]: vkg_fib_ism_ral_funcs_init: Invalid Role - 0

LC/0/2/CPU0:Sep 27 14:27:37.735 : mpls_io[256]: vkg_fib_ism_ral_funcs_init: Invalid Role - 0

LC/0/2/CPU0:Sep 27 14:27:39.497 : avsm_agent[125]: %PROCMGR-3-PROCESS_CRASH : process avsm_sia crashed, restarting it...

LC/0/2/CPU0:Sep 27 14:27:39.554 : apiv_srvr[116]: %PLATFORM-APIV-5-API_INCOMPATIBLE : Message 24 in APIV module APIV_SI is incompatible (reason: Not supported on the remote side)

LC/0/2/CPU0:Sep 27 14:27:39.554 : apiv_srvr[116]: %PLATFORM-APIV-3-MODULE_INCOMPATIBLE : APIV module APIV_SI is incompatible

LC/0/2/CPU0:Sep 27 14:27:44.634 : avsm_agent[125]: %PROCMGR-3-PROCESS_CRASH : process avsm_sia crashed, restarting it...

LC/0/2/CPU0:Sep 27 14:27:44.691 : apiv_srvr[116]: %PLATFORM-APIV-5-API_INCOMPATIBLE : Message 24 in APIV module APIV_SI is incompatible (reason: Not supported on the remote side)

LC/0/2/CPU0:Sep 27 14:27:44.691 : apiv_srvr[116]: %PLATFORM-APIV-3-MODULE_INCOMPATIBLE : APIV module APIV_SI is incompatible

LC/0/2/CPU0:Sep 27 14:27:47.182 : arp[117]: vkg_fib_ism_ral_funcs_init: Invalid Role - 0

LC/0/2/CPU0:Sep 27 14:27:47.299 : l2fib[238]: vkg_fib_ism_ral_funcs_init: Invalid Role - 0

LC/0/2/CPU0:Sep 27 14:27:47.754 : mpls_io[256]: vkg_fib_ism_ral_funcs_init: Invalid Role - 0

LC/0/2/CPU0:Sep 27 14:27:49.772 : avsm_agent[125]: %PROCMGR-3-PROCESS_CRASH : process avsm_sia crashed, restarting it...

LC/0/2/CPU0:Sep 27 14:27:49.773 : avsm_agent[125]: %PROCMGR-2-SE_RESTART : process avsm_sia restart too many times, Service Engine is going to reboot.

LC/0/2/CPU0:Sep 27 14:27:49.834 : apiv_srvr[116]: %PLATFORM-APIV-5-API_INCOMPATIBLE : Message 24 in APIV module APIV_SI is incompatible (reason: Not supported on the remote side)

LC/0/2/CPU0:Sep 27 14:27:49.834 : apiv_srvr[116]: %PLATFORM-APIV-3-MODULE_INCOMPATIBLE : APIV module APIV_SI is incompatible

LC/0/2/CPU0:Sep 27 14:27:54.182 : avsm_se_sm[126]: %PLATFORM-SIM_SE-5-STATE_CHANGE : Service Engine is in shutting down state.

LC/0/2/CPU0:Sep 27 14:27:54.183 : sis[305]: Received SYSDB_EVENT_SET Notification (state 9) ...

LC/0/2/CPU0:Sep 27 14:27:54.184 : vkg_l2fib_mac_cache[338]: Deactivating APIV module MOD_ID_AVSM_L2FIB...

LC/0/2/CPU0:Sep 27 14:27:54.185 : sis[305]: Starting of SI APIV Deletion ...

LC/0/2/CPU0:Sep 27 14:27:54.914 : avsm_agent[125]: %PROCMGR-3-PROCESS_CRASH : process avsm_sia crashed, restarting it...

LC/0/2/CPU0:Sep 27 14:27:54.972 : apiv_srvr[116]: %PLATFORM-APIV-5-API_INCOMPATIBLE : Message 24 in APIV module APIV_SI is incompatible (reason: Not supported on the remote side)

LC/0/2/CPU0:Sep 27 14:27:54.973 : apiv_srvr[116]: %PLATFORM-APIV-3-MODULE_INCOMPATIBLE : APIV module APIV_SI is incompatible

LC/0/2/CPU0:Sep 27 14:27:57.189 : arp[117]: vkg_fib_ism_ral_funcs_init: Invalid Role - 0

LC/0/2/CPU0:Sep 27 14:27:57.305 : l2fib[238]: vkg_fib_ism_ral_funcs_init: Invalid Role - 0

LC/0/2/CPU0:Sep 27 14:27:57.762 : mpls_io[256]: vkg_fib_ism_ral_funcs_init: Invalid Role - 0

LC/0/2/CPU0:Sep 27 14:28:09.664 : l2fib[238]: Starting of APIV version negotiation ...

LC/0/2/CPU0:Sep 27 14:28:12.714 : l2fib[238]: Deactivating APIV module MOD_ID_AVSM_L2FIB...

RP/0/RSP0/CPU0:Sep 27 14:28:25.633 : shelfmgr[386]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/2/CPU0 A9K-ISM-100 state:IOS XR RUN

LC/0/2/CPU0:Sep 27 14:28:28.479 : sis[305]: Received SYSDB_EVENT_SET Notification (state 10) ...

LC/0/2/CPU0:Sep 27 14:28:28.790 : avsm_se_sm[126]: %PLATFORM-SIM_SE-5-STATE_CHANGE : Service Engine is in reset(Service Engine Software) state.

rebooting

RP/0/RSP0/CPU0:Sep 27 14:28:28.842 : canb-server[150]: %PLATFORM-CANB_SERVER-7-CBC_PRE_RESET_NOTIFICATION : Node 0/2/CPU0, reason : 0x5 0x0 0x0 0x0 

RP/0/RSP0/CPU0:Sep 27 14:28:28.842 : canb-server[150]: %PLATFORM-CANB_SERVER-7-CBC_PRE_RESET_NOTIFICATION : Node 0/2/CPU0, reason : 0xb 0x0 0x0 0x0 

RP/0/RSP0/CPU0:Sep 27 14:28:28.842 : shelfmgr[386]: %PLATFORM-SHELFMGR-6-NODE_CPU_RESET : Node 0/2/CPU0 CPU reset detected.

RP/0/RSP0/CPU0:Sep 27 14:28:28.843 : shelfmgr[386]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/2/CPU0 A9K-ISM-100 state:BRINGDOWN

RP/0/RSP0/CPU0:Sep 27 14:28:28.995 : ce_switch_srv[53]: %PLATFORM-CE_SWITCH-6-UPDN : Interface 6 (LC_Slot_2) is down

RP/0/RSP0/CPU0:Sep 27 14:28:35.201 : canb-server[150]: %PLATFORM-CANB_SERVER-7-CBC_POST_RESET_NOTIFICATION : Node 0/2/CPU0, reason : 0xb 0x0 0x0 0x0 

RP/0/RSP0/CPU0:Sep 27 14:28:35.203 : shelfmgr[386]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/2/CPU0 A9K-ISM-100 state:ROMMON

Selecting ROMMON Image... B

DDR in Interleaved mode

POST 1 : PASSED : code 0 : DDR2 Memory

System Bootstrap, Version 1.2(20091201:235620) [ASR9K ROMMON],

Copyright (c) 1994-2009 by Cisco Systems, Inc.

Compiled Fri 08-Jan-10 16:18 by ashok

CPU Reset Reason = 0x000b

PPC 8641D (partnum 0x0003), Revision 0.2, (Core Version 2.20136)

M8641 CLKIN:   66 Mhz

Core Clock: 1333 Mhz

  MPX Clock:  533 Mhz

  LBC Clock:   33 Mhz

Board Type: 1000305

POST 2 : PASSED : code 0 : FPGA Flash Images

POST 3 : PASSED : code 0 : Slot ID/Board Type

Loading Field Programmable Devices:

FPGA 0-B IMG_INVALID : image: 0xfd800000 - 0xfe7fffff

FPGA 0-A PROGRAMMED  : image: 0xfe800028 - 0xff373c44, et: 558ms

FPGA 1-B IMG_INVALID : image: 0xfd800000 - 0xfe7fffff

FPGA 1-A PROGRAMMED  : image: 0xfe800028 - 0xff373c44, et: 558ms

passed fpga_prog_devices

Pass reseting devices

Main Board: 0x1000305, rev 0x3

      PLD: 0.1

  Bridge0: 0.25

  Bridge1: 0.25

      CBC: 18.5

Passed init Phy

PCI-E1: Ready as Root Complex

PCI-E2: Ready as Root Complex

Cannot detect the presence of RSP1.

ASR9K (8641D PPC) platform with 4032 Mb of main memory

    CARD_SLOT_NUMBER: 4

        CPU_INSTANCE: 1

MBI Validation starts ...RP/0/RSP0/CPU0:Sep 27 14:29:05.027 : ce_switch_srv[53]: %PLATFORM-CE_SWITCH-6-UPDN : Interface 6 (LC_Slot_2) is up

tsec_init_hw: configuring TSEC (port 0) for: 1GB, Full Duplex

tsec_init_interface: hardware initialization completed

Interface link changed state to UP.

Interface link state up.

MBI validation sending request.

HIT CRP/0/RSP0/CPU0:Sep 27 14:29:06.172 : ce_switch_srv[53]: %PLATFORM-CE_SWITCH-6-UPDN : Interface 6 (LC_Slot_2) is down

TRL-C to abort

RP/0/RSP0/CPU0:Sep 27 14:29:06.461 : ce_switch_srv[53]: %PLATFORM-CE_SWITCH-6-UPDN : Interface 6 (LC_Slot_2) is up

.RP/0/RSP0/CPU0:Sep 27 14:29:06.692 : shelfmgr[386]: %PLATFORM-SHELFMGR_HAL-6-BOOT_REQ_RECEIVED : Boot Request from 0/2/CPU0, RomMon Version: 1.2

mbi_val_pRP/0/RSP0/CPU0:Sep 27 14:29:06.693 : shelfmgr[386]: %PLATFORM-MBIMGR-7-IMAGE_VALIDATED : Remote location 0/2/CPU0: : MBI tftp:/disk0/asr9k-os-mbi-4.2.1/lc/mbiasr9k-lc.vm validated

rocess_packeRP/0/RSP0/CPU0:Sep 27 14:29:06.696 : shelfmgr[386]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/2/CPU0 A9K-ISM-100 state:MBI-BOOTING

t: received repsonse

Remote image to boot : tftp:/disk0/asr9k-os-mbi-4.2.1/lc/mbiasr9k-lc.vm

          IP_ADDRESS: 127.0.1.4

      IP_SUBNET_MASK: 255.255.0.0

     DEFAULT_GATEWAY: 127.0.1.0

         TFTP_SERVER: 127.0.1.0

           TFTP_FILE: /disk0/asr9k-os-mbi-4.2.1/lc/mbiasr9k-lc.vm

Performing tftpdnld

tsec_init_hw: configuring TSEC (port 0) for: 1GB, Full Duplex

tsec_init_interface: hardware initialization completed

tftp_process_packet: received OACK.

tftp_process_packet: Change blksize to 1400.

Receiving /disk0/asr9k-os-mbi-4.2.1/lc/mbiasr9k-lc.vm from 127.0.1.0 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

tftp_process_packet: last packet, block=12544, size=1148.

File reception completed.

Image size=17561348, TFTP download size=17561348

...................................................................

program load complete, entry point: 0x202148, size: 0x10be804

Attempting to start second CPU

Config = SMP, Running = SMP

Board type: 0x01000305

Card Capability = 0xffffffff

###########################################################################################

BSP: Board type : LC-ISM

tracelogger: starting tracing in background ring mode

tracelogger running with args: -startring -F 1 -F 2

              Restricted Rights Legend

Use, duplication, or disclosure by the Government is

subject to restrictions as set forth in subparagraph

(c) of the Commercial Computer Software - Restricted

Rights clause at FAR sec. 52.227-19 and subparagraph

(c) (1) (ii) of the Rights in Technical Data and Computer

Software clause at DFARS sec. 252.227-7013.

           cisco Systems, Inc.

           170 West Tasman Drive

           San Jose, California 95134-1706

Cisco IOS XR Software for the Cisco XR ASR9K, Version 4.2.1

Copyright (c) 2012 by Cisco Systems, Inc.

# RP/0/RSP0/CPU0:Sep 27 14:30:05.597 : shelfmgr[386]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/2/CPU0 A9K-ISM-100 state:MBI-RUNNING

RP/0/RSP0/CPU0:Sep 27 14:30:05.651 : sysdb_svr_admin[411]: %SYSDB-SYSDB-6-TIMEOUT_EDM : EDM request for 'admin/oper/inventory/rack/0/entity/' from 'lrd' (jid 316, node 0/RSP0/CPU0). No response from 'invmgr' (jid 256, node 0/RSP0/CPU0) within the timeout period (100 seconds)

RP/0/RSP0/CPU0:Sep 27 14:30:05.651 : sysdb_shared_nc[408]: %SYSDB-SYSDB-6-TIMEOUT_EDM : EDM request for 'oper/inventory/gl/rack/0/entity/' from 'pld_upg_d' (jid 359, node 0/RSP0/CPU0). No response from 'lrd' (jid 316, node 0/RSP0/CPU0) within the timeout period (100 seconds)

ok, we’d need to use the insatll-kit (and procedure) from my first message:

http://www.cisco.com/en/US/docs/routers/asr9000/software/asr9k_r4.2/general/release/notes/reln_a9k_421.html#reference_A752312FAE4F48F7B672999F81A327E2

If you’d have difficulties to do it, you may open a service request so we can do it together using WebEx.

Regards,

Alex.

Thanks again !

This has ot come up in my documentation but I will give it a go, I have A9K-RSP440-SE. what should right image be for ISM card. bare in mind, I am not sure admin install to 0/2/cpu0 will be permitted as it is none existent !

BR

/Samir

Hmmm, still cannot see how it would help though; we only need CGN at this point.

Cisco IOS XR Software Release 4.2.1 introduces  support for the ISM  single hardware PID solution on the ISM (Integrated  Service Module) line card. This feature provides role based  installation of the following different service applications:

• CDS TV
• CDS IS
• CGv6

How can I check if fpd is corrupted ?

Again 0/2/CPU0 does not appear.

I have had the card re seated.

RP/0/RSP0/CPU0:CGN_PE(admin)#sh install active

Thu Sep 27 15:30:40.440 CETDST

Secure Domain Router: Owner

  Node 0/RSP0/CPU0 [RP] [SDR: Owner]

    Boot Device: disk0:

    Boot Image: /disk0/asr9k-os-mbi-4.2.1/0x100305/mbiasr9k-rsp3.vm

    Active Packages:

      disk0:asr9k-services-p-px-4.2.1

      disk0:asr9k-mini-px-4.2.1

      disk0:asr9k-k9sec-px-4.2.1

      disk0:asr9k-mpls-px-4.2.1

      disk0:asr9k-mgbl-px-4.2.1

      disk0:asr9k-fpd-px-4.2.1

  Node 0/0/CPU0 [LC] [SDR: Owner]

    Boot Device: mem:

    Boot Image: /disk0/asr9k-os-mbi-4.2.1/lc/mbiasr9k-lc.vm

    Active Packages:

      disk0:asr9k-services-p-px-4.2.1

      disk0:asr9k-mini-px-4.2.1

      disk0:asr9k-mpls-px-4.2.1

It looks like the ISM can not pick up (or does not have) the correct image on the blade itself. That is why we need to run the insatll-kit. Note, it is not loaded with the main image and should be downloaded and installed separately as described in the procedure.

Regards,

/A

Unfortunately I have no access to this link, and also noticed "asr9k-ism-cgv6-". I will be doing NAT44 only.

I tried  forcing fpd upgrade  :upgrade hw-module fpd fpga force location 0/2/CPU0, but log shows

RP/0/RSP0/CPU0:Sep 27 15:52:17.480 : upgrade_fpd_cli[65856]: %PLATFORM-UPGRADE_FPD-6-STATUS : 0/2/CPU0 does not exist for upgrade

NAT44 is part of the package.

Opent TAC Service Request tomorrow between 8:00 and 16:00 CET time and put my name in the description. We'd be able to publish the kit for you and do the upgrade together.

Regads,

/A