Messages errors ( 4232L3 and 4000 )

DWAM_2 · ‎01-09-2003

Hello,

Equipement : Cisco 4000

Follow a show module of Cisco 4000

Mod Slot Ports Module-Type Model Sub Status

--- ---- ----- ------------------------- ------------------- --- --------

1 1 2 1000BaseX Supervisor WS-X4013 no ok

2 2 48 10/100BaseTx Ethernet WS-X4148-RJ no ok

3 3 6 1000BaseX Ethernet WS-X4306 no ok

4 4 24 100BaseFX Ethernet WS-X4124-FX-MT no ok

5 5 34 Router Switch Card WS-X4232-L3 no ok

6 6 24 10/100/1000 Ethernet WS-X4424-GB-RJ45 no ok

Mod MAC-Address(es) Hw Fw Sw

--- -------------------------------------- ------ ---------- -----------------

1 00-02-fc-30-5c-00 to 00-02-fc-30-5f-ff 1.1 5.4(1) 6.3(4)

2 00-02-4b-c3-e3-30 to 00-02-4b-c3-e3-5f 2.3

3 00-04-4d-fb-0d-7e to 00-04-4d-fb-0d-83 2.0

4 00-d0-06-01-71-78 to 00-d0-06-01-71-8f 1.0

5 00-06-28-43-ef-70 to 00-06-28-43-ef-91 1.7 12.0(7)W5( 12.0(18)W5(22b)

6 00-07-eb-87-f8-b8 to 00-07-eb-87-f8-cf 1.5

Version of 4232 : cat4232-in-mz.120-18.W5.22b.bin.

Version of CatOS : 6.3.4.

Yesterday i have change the 4232 in the Cisco 4000 with an upgrade of IOS for the 4232 (because it was a too old version of IOS)

After the change all are working.

But after 4H of good working, all the 4000 can not used the routing Layer.

We decided to reboot the Cisco 4000 without to take trace.

After the reboot all communications are OK, routing works good.

But after 5H later the last problem, all the 4000 can not used again the routing Layer.

Here are the symptoms that i have on this equipement

When i am on the layer two : i can not access to the layer 3

Switch> (enable) session 5

Trying IntlgLineCard-5...

session: Unable to tunnel to IntlgLineCard-5 (57)

Switch> (enable)

the only access to join 4232 is to connect with console port on 4232.

I do a sh tech and i have nothing interesting on the sh logging.

But when i do a sh tech i got the follow messages inside the sh tech at the part sh controllers:

.Jan 8 15:02:59: xpif_port_read_mac: Timeout while waiting for response message on GigabitEthernet1GMAC Status: 0x1C GBIC Status: 0x3

MII registers of External GMAC:

.Jan 8 15:03:06: xpif_port_read_mac: Timeout while waiting for response message on GigabitEthernet1Control Register (0x00): 0xFFFF

.Jan 8 15:03:13: xpif_port_read_mii: Timeout for response message for GigabitEthernet1Rx Uinfo Registerter-GMAC (0xC006): 0xFFFF

.Jan 8 15:03:19: xpif_port_write_mii: Timeout for response message GigabitEthernet1Line 2326 xpif_port_indirect_read_mii : Write MII failed for GigabitEthernet1

.Jan 8 15:03:27: xpif_port_read_mac: Timeout while waiting for response message on GigabitEthernet2GMAC Status: 0x1C GBIC Status: 0x3

MII registers of External GMAC:

.Jan 8 15:03:33: xpif_port_read_mac: Timeout while waiting for response message on GigabitEthernet2Control Register (0x00): 0xFFFF

.Jan 8 15:03:40: xpif_port_read_mii: Timeout for response message for GigabitEthernet2Rx Uinfo Registerter-GMAC (0xC006): 0xFFFF

.Jan 8 15:03:46: xpif_port_write_mii: Timeout for response message GigabitEthernet2Line 2326 xpif_port_indirect_read_mii : Write MII failed for GigabitEthernet2

Control Register (0x00): 0xFFFF

.Jan 8 15:03:52: xpif_port_read_mii: Timeout for response message for GigabitEthernet2Rx Uinfo Registerter-GMAC (0xC006): 0xFFFF

.Jan 8 15:03:59: xpif_port_write_mii: Timeout for response message GigabitEthernet2Line 2326 xpif_port_indirect_read_mii : Write MII failed for GigabitEthernet2

.Jan 8 15:04:07: xpif_port_read_mac: Timeout while waiting for response message on GigabitEthernet3GMAC Status: 0x1C GBIC Status: 0x3

MII registers of External GMAC:

.Jan 8 15:04:13: xpif_port_read_mac: Timeout while waiting for response message on GigabitEthernet3Control Register (0x00): 0xFFFF

.Jan 8 15:04:19: xpif_port_read_mii: Timeout for response message for GigabitEthernet3Rx Uinfo Registerter-GMAC (0xC006): 0xFFFF

MAC registers:

.Jan 8 15:04:26: xpif_port_write_mii: Timeout for response message GigabitEthernet3Line 2326 xpif_port_indirect_read_mii : Write MII failed for GigabitEthernet3

.Jan 8 15:04:34: xpif_port_read_mac: Timeout while waiting for response message on GigabitEthernet4GMAC Status: 0x1C GBIC Status: 0x3

MII registers of External GMAC:

.Jan 8 15:04:40: xpif_port_read_mac: Timeout while waiting for response message on GigabitEthernet4Control Register (0x00): 0xFFFF

.Jan 8 15:04:46: xpif_port_read_mii: Timeout for response message for GigabitEthernet4Rx Uinfo Registerter-GMAC (0xC006): 0xFFFF

And

------------------ show epc status ------------------

Status of GigabitEthernet1: not OK - aborting this command

-----------

The sh stack and the sh proc cpu give no interesting information.

I just want to add that this 4232 was used during 6 months inside another 4000 without any problem.

All ideas are welcome and Thanks you for your help..

efrahim · ‎01-09-2003

Hi,

Try to reseat the module.. Seems like the internal GigE are stuck or not communicating with the sup. properly. How much was the cpu on the 4232 when you checked.

DWAM_2 · ‎01-09-2003

Thanks you for your answer Erum Frahim

Follow the sh cpu of 4232 during the check :

------------------ show process cpu ------------------

CPU utilization for five seconds: 1%/0%; one minute: 1%; five minutes: 1%

--------------------------------------------------------------

Actually it's not really possible to reset acually the module because we are in big production environnement.

So we change the 4232 and 4000 (but not the sup) this night and we hope that this operation will resolve all ours problems.

I think You are right about the wrong communication between the (Sup and the backplane) or (4232 and backplane) or a problem in one of this element.

How to be sure of that ?!

efrahim · ‎01-09-2003

Communication of the 4232 /sup involves sup, 4232 module and the chassis

so if it is bad hardware, then it could be anything ( most probable the 4232) because this type of bahaviour sometimes sees with the high traffic, or may be the module was just not seated properly,,, so it is very hard to say at this point if u already replaced the 4232 and the chassis..

DWAM_2 · ‎01-09-2003

That's right.

I will update this post tomorrow about the result of the replace.

Thanks You very much for your help.