Steven,
See this:
There is primarily a communication error (not a hardware failure,at least at first glance) with IOM attached to FI-B. Look at the difference between both of the IOMs, where the IOM to FI-A is affected but not that much (I made it a little more understandable):
DC-UCS-FI-A# connect iom 1
fex-1# show platform soft cmc showi2c
segment 1 chassis
norxack 5514 <<<<< means a problem if you have this much
unfinished 1
lostarbitration 1
fixup 2
segment 2 blade
segment 3 fan
segment 4 psu
pca9541postio2 12
gilroy.error.pca9541_control_state 77
gilroy.error.do_reserve_pca9541_control 77
bus_lost_counter: 991
error_pca9541_per_device:
c.ms 77 <<<< This is a problem with the chassis management selector which is located on both IOMs for mgnt
c.gpio0 1
# I2C Device Statistics
c.seeprom={SUCCESS=262839}
f.fm0.fru={SUCCESS=1} <<<<<< All fans are talking fine with the IOM that is supposed to manage their behavior
f.fm1.fru={SUCCESS=1}
f.fm2.fru={SUCCESS=1}
f.fm3.fru={SUCCESS=1}
f.fm4.fru={SUCCESS=1}
f.fm5.fru={SUCCESS=1}
f.fm6.fru={SUCCESS=1}
f.fm7.fru={SUCCESS=1}
--------------------------------------------------------------------------------------------------
DC-UCS-FI-B# connect iom 1
fex-1# show platform soft cmc showi2c
segment 1 chassis
norxack 2795 <<<<<<<
wait_gt_deadline 207743
segment 2 blade
segment 3 fan
norxack 108550 <<<<<< Is evident the problem when talking to the other IOM
timeout 27815 <<<<<<
unfinished 14
fixup 55614
pca9541clrerrprs 7
pca9541seterr 2
pca9541postio3 2
wait_gt_deadline 9319746
hub_sw_mbb 27785
hub_sw_mbb_to 27785
segment 4 psu
pca9541postio2 12
wait_gt_deadline 6664338
gilroy.error.pca9541_control_state 99
gilroy.error.do_reserve_pca9541_control 99
bus_lost_counter: 535
error_pca9541_per_device:
c.ms 99 <<<<< c.ms again
f.fm0.fc 3 << and the fans showing up here
f.fm1.ms 1
f.fm2.ms 1
f.fm3.fc 2
f.fm3.ms 1
f.fm4.ms 1
f.fm5.ms 1
f.fm6.ms 1
f.fm7.ms 1
# I2C Device Statistics
c.seeprom={SUCCESS=263768}
f.fm0.fc={SUCCESS=4410402,EIO=2,EBUSY=3477} << this means an efforts to communicate but finds the channel busy
f.fm1.fc={SUCCESS=4407270,ENXIO=2,EBUSY=3478}
f.fm2.fc={SUCCESS=4406286,EBUSY=3478}
f.fm3.fc={SUCCESS=4406492,ENXIO=2,EBUSY=3476}
f.fm4.fc={SUCCESS=4406621,EBUSY=3476}
f.fm5.fc={SUCCESS=4407430,EIO=1,ENXIO=1,EBUSY=3476}
f.fm6.fc={SUCCESS=4407772,EBUSY=3476}
f.fm7.fc={SUCCESS=4407739,ENXIO=1,EBUSY=3480}
I would recommend to reseat the IOM connected to FI-B during a maintenance (take out for about 5 minutes to drain all the power), same as the fans (take the fans out for about 3 minutes, same reason as the IOM) and be sure you are running the follwoing versions or above, according to the firmware you are running: 2.0.5b or later OR 2.1.1f or later OR any version of 2.2. IF you are already running one of those versions, then just do the reseat.
I hope this helps.
Rate ALL helpful answers.
-Kenny
