08-24-2015 03:24 AM
LC/1/1/CPU0:Aug 24 11:00:35.617 : ifmgr[177]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface TenGigE1/1/5/0, changed state to Down
LC/1/1/CPU0:Aug 24 11:00:35.639 : ifmgr[177]: %PKT_INFRA-LINK-3-UPDOWN : Interface TenGigE1/1/5/0, changed state to Up
LC/1/1/CPU0:Aug 24 11:00:35.643 : ifmgr[177]: %PKT_INFRA-LINK-3-UPDOWN : Interface TenGigE1/1/5/0, changed state to Down
LC/1/1/CPU0:Aug 24 11:00:35.652 : ifmgr[177]: %PKT_INFRA-LINK-3-UPDOWN : Interface TenGigE1/1/5/0, changed state to Up
LC/1/1/CPU0:Aug 24 11:00:35.664 : ifmgr[177]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface TenGigE1/1/5/0, changed state to Up
LC/1/6/CPU0:Aug 24 11:00:36.364 : ifmgr[177]: %PKT_INFRA-LINK-3-UPDOWN : Interface TenGigE1/6/3/0, changed state to Up
LC/1/6/CPU0:Aug 24 11:00:36.366 : ifmgr[177]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface TenGigE1/6/3/0, changed state to Up
LC/1/5/CPU0:Aug 24 11:00:39.258 : spa_xge_v2(2)[301]: %L2-ether_spa_plugin-6-LINK_EVENT : Interface TenGige 1/5/2/0, Rx Fault
LC/1/5/CPU0:Aug 24 11:00:39.258 : ifmgr[177]: %PKT_INFRA-LINK-3-UPDOWN : Interface TenGigE1/5/2/0, changed state to Down
LC/1/5/CPU0:Aug 24 11:00:39.260 : ifmgr[177]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface TenGigE1/5/2/0, changed state to Down
LC/1/5/CPU0:Aug 24 11:00:39.273 : ifmgr[177]: %PKT_INFRA-LINK-3-UPDOWN : Interface TenGigE1/5/2/0, changed state to Up
LC/1/5/CPU0:Aug 24 11:00:39.276 : ifmgr[177]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface TenGigE1/5/2/0, changed state to Up
LC/1/0/CPU0:Aug 24 11:00:39.960 : ifmgr[177]: %PKT_INFRA-LINK-3-UPDOWN : Interface TenGigE1/0/3/0, changed state to Down
LC/1/0/CPU0:Aug 24 11:00:39.961 : ifmgr[177]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface TenGigE1/0/3/0, changed state to Down
LC/1/0/CPU0:Aug 24 11:00:39.968 : ifmgr[177]: %PKT_INFRA-LINK-3-UPDOWN : Interface TenGigE1/0/3/0, changed state to Up
LC/1/0/CPU0:Aug 24 11:00:39.976 : ifmgr[177]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface TenGigE1/0/3/0, changed state to Up
LC/1/9/CPU0:Aug 24 11:00:43.504 : ifmgr[177]: %PKT_INFRA-LINK-3-UPDOWN : Interface TenGigE1/9/2/0, changed state to Down
LC/1/9/CPU0:Aug 24 11:00:43.505 : ifmgr[177]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface TenGigE1/9/2/0, changed state to Down
LC/1/9/CPU0:Aug 24 11:00:43.512 : ifmgr[177]: %PKT_INFRA-LINK-3-UPDOWN : Interface TenGigE1/9/2/0, changed state to Up
LC/1/9/CPU0:Aug 24 11:00:43.520 : ifmgr[177]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface TenGigE1/9/2/0, changed state to Up
LC/1/6/CPU0:Aug 24 11:00:47.726 : ifmgr[177]: %PKT_INFRA-LINK-3-UPDOWN : Interface TenGigE1/6/5/0, changed state to Down
LC/1/6/CPU0:Aug 24 11:00:47.726 : ifmgr[177]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface TenGigE1/6/5/0, changed state to Down
LC/1/6/CPU0:Aug 24 11:00:47.734 : ifmgr[177]: %PKT_INFRA-LINK-3-UPDOWN : Interface TenGigE1/6/5/0, changed state to Up
LC/1/6/CPU0:Aug 24 11:00:47.744 : ifmgr[177]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface TenGigE1/6/5/0, changed state to Up
LC/0/10/CPU0:Aug 24 11:00:48.240 : ifmgr[177]: %PKT_INFRA-LINK-3-UPDOWN : Interface TenGigE0/10/5/0, changed state to Down
LC/0/10/CPU0:Aug 24 11:00:48.241 : ifmgr[177]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface TenGigE0/10/5/0, changed state to Down
LC/0/10/CPU0:Aug 24 11:00:48.248 : ifmgr[177]: %PKT_INFRA-LINK-3-UPDOWN : Interface TenGigE0/10/5/0, changed state to Up
LC/0/10/CPU0:Aug 24 11:00:48.257 : ifmgr[177]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface TenGigE0/10/5/0, changed state to Up
LC/0/10/CPU0:Aug 24 11:00:48.367 : spa_xge_v2(5)[302]: %L2-ether_spa_plugin-6-LINK_EVENT : Interface TenGige 0/10/5/0, Rx Fault
LC/0/10/CPU0:Aug 24 11:00:48.367 : ifmgr[177]: %PKT_INFRA-LINK-3-UPDOWN : Interface TenGigE0/10/5/0, changed state to Down
LC/0/10/CPU0:Aug 24 11:00:48.369 : ifmgr[177]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface TenGigE0/10/5/0, changed state to Down
LC/0/10/CPU0:Aug 24 11:00:48.381 : ifmgr[177]: %PKT_INFRA-LINK-3-UPDOWN : Interface TenGigE0/10/5/0, changed state to Up
LC/0/10/CPU0:Aug 24 11:00:48.385 : ifmgr[177]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface TenGigE0/10/5/0, changed state to Up
LC/0/11/CPU0:Aug 24 11:00:50.238 : spa_xge_v2(5)[301]: %L2-ether_spa_plugin-6-LINK_EVENT : Interface TenGige 0/11/5/0, Rx Fault
[K --More-- [KLC/0/11/CPU0:Aug 24 11:00:50.239 : ifmgr[177]: %PKT_INFRA-LINK-3-UPDOWN : Interface TenGigE0/11/5/0, changed state to Down
LC/0/11/CPU0:Aug 24 11:00:50.240 : ifmgr[177]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface TenGigE0/11/5/0, changed state to Down
LC/0/11/CPU0:Aug 24 11:00:50.253 : ifmgr[177]: %PKT_INFRA-LINK-3-UPDOWN : Interface TenGigE0/11/5/0, changed state to Up
LC/0/11/CPU0:Aug 24 11:00:50.257 : ifmgr[177]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface TenGigE0/11/5/0, changed state to Up
LC/0/11/CPU0:Aug 24 11:00:50.375 : ifmgr[177]: %PKT_INFRA-LINK-3-UPDOWN : Interface TenGigE0/11/5/0, changed state to Down
LC/0/11/CPU0:Aug 24 11:00:50.376 : ifmgr[177]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface TenGigE0/11/5/0, changed state to Down
LC/0/11/CPU0:Aug 24 11:00:50.385 : ifmgr[177]: %PKT_INFRA-LINK-3-UPDOWN : Interface TenGigE0/11/5/0, changed state to Up
LC/0/11/CPU0:Aug 24 11:00:50.396 : ifmgr[177]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface TenGigE0/11/5/0, changed state to Up
LC/1/3/CPU0:Aug 24 11:00:51.118 : spa_xge_v2(5)[303]: %L2-ether_spa_plugin-6-LINK_EVENT : Interface TenGige 1/3/5/0, Rx Fault
LC/1/3/CPU0:Aug 24 11:00:51.119 : ifmgr[177]: %PKT_INFRA-LINK-3-UPDOWN : Interface TenGigE1/3/5/0, changed state to Down
LC/1/3/CPU0:Aug 24 11:00:51.120 : ifmgr[177]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface TenGigE1/3/5/0, changed state to Down
LC/1/3/CPU0:Aug 24 11:00:51.133 : ifmgr[177]: %PKT_INFRA-LINK-3-UPDOWN : Interface TenGigE1/3/5/0, changed state to Up
LC/1/3/CPU0:Aug 24 11:00:51.136 : ifmgr[177]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface TenGigE1/3/5/0, changed state to Up
LC/1/6/CPU0:Aug 24 11:00:52.262 : spa_xge_v2(3)[301]: %L2-ether_spa_plugin-6-LINK_EVENT : Interface TenGige 1/6/3/0, Rx Fault
LC/1/6/CPU0:Aug 24 11:00:52.262 : ifmgr[177]: %PKT_INFRA-LINK-3-UPDOWN : Interface TenGigE1/6/3/0, changed state to Down
LC/1/6/CPU0:Aug 24 11:00:52.264 : ifmgr[177]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface TenGigE1/6/3/0, changed state to Down
LC/1/6/CPU0:Aug 24 11:00:52.276 : ifmgr[177]: %PKT_INFRA-LINK-3-UPDOWN : Interface TenGigE1/6/3/0, changed state to Up
LC/1/6/CPU0:Aug 24 11:00:52.280 : ifmgr[177]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface TenGigE1/6/3/0, changed state to Up
LC/1/1/CPU0:Aug 24 11:00:55.927 : spa_xge_v2(5)[303]: %L2-ether_spa_plugin-6-LINK_EVENT : Interface TenGige 1/1/5/0, Rx Fault
LC/1/1/CPU0:Aug 24 11:00:55.927 : ifmgr[177]: %PKT_INFRA-LINK-3-UPDOWN : Interface TenGigE1/1/5/0, changed state to Down
LC/1/1/CPU0:Aug 24 11:00:55.929 : ifmgr[177]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface TenGigE1/1/5/0, changed state to Down
RP/1/RP0/CPU0:Aug 24 11:00:55.936 : ospf[1009]: %ROUTING-OSPF-5-ADJCHG : Process 100, Nbr 122.192.52.1 on TenGigE1/1/5/0 in area 0 from 2WAY to DOWN, Neighbor Down: interface down or detached,vrf default vrfid 0x60000000
LC/1/1/CPU0:Aug 24 11:00:55.943 : ifmgr[177]: %PKT_INFRA-LINK-3-UPDOWN : Interface TenGigE1/1/5/0, changed state to Up
LC/1/1/CPU0:Aug 24 11:00:55.947 : ifmgr[177]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface TenGigE1/1/5/0, changed state to Up
RP/1/RP0/CPU0:Aug 24 11:01:37.720 : ospf[1009]: %ROUTING-OSPF-5-ADJCHG : Process 100, Nbr 122.192.52.1 on TenGigE1/1/5/0 in a
08-24-2015 10:02 AM
Please open a TAC case for us so that we can troubleshoot this. Rx fault indicates some issue on the remote end so it might be a good idea to identify if there might be any transport issues/L1 problems on the link.
08-24-2015 06:00 PM
Hi,osman,
Thank you!
How about Tx Fault.
I saw few Tx Fault in the log. but just only few times. does that mean XFP hardware problem?
Best
Tom
08-24-2015 06:40 PM
Hi Tom
Can you please share the complete syslog for this event? I can review to see if there might be any correlation. Best bet is to attach the info to a TAC SR.
08-24-2015 07:54 PM
Hi,osman,
Thank you!
I check show controllers tenGigE 1/6/3/0 phy
it shows Link Stage down, but i can't understand output like below?
I attach log file ,
thank you!
Tom
Link flapped at Mon Aug 24 11:00:52 2015
Link State changed to down
1.1 FPGA Address : 0x60a84204 Value : 0x12070000
1.2 FPGA Address : 0x60a84204 Value : 0x12070000
2.1 FPGA Address : 0x60a84200 Value : 0xe
2.2 FPGA Address : 0x60a84200 Value : 0x8
3.1. Serdes PCS status Register1 (0x230001): 0x 82
3.2. Serdes PCS status Register1 (0x230001): 0x 82
4.1. Serdes PCS status Register2 (0x230008): 0x 8405
4.2. Serdes PCS status Register2 (0x230008): 0x 8405
5.1.Frequency status status Register (0x238126): 0x a002
5.2.Frequency status status Register (0x238126): 0x a002
6.1.10G Serdes status Register (0x238127): 0x 20
6.2.10G Serdes status Register (0x238127): 0x 0
7.1. Reg (0x238300): 0x 2301
7.2. Reg (0x238300): 0x 2301
8.1. Reg (0x23fe26): 0x 99d6
8.2. Reg (0x23fe26): 0x 99d6
9.1. PHY XGXS status 1 reg(0x240001) : 0x 86
9.2. PHY XGXS status 1 reg(0x240001) : 0x 86
10.1. PHY XGXS status 2 reg (0x240008): 0x 8400
10.2. PHY XGXS status 2 reg (0x240008): 0x 8400
11.1.PHY XGXS Lane status reg (0x240018) : 0x 1c0f
11.2.PHY XGXS Lane status reg (0x240018) : 0x 1c0f
12.1. reg (0x248009) : 0x 400
12.2. reg (0x248009) : 0x 400
13.1. reg (0x230020) : 0x 4
13.2. reg (0x230020) : 0x 4
14.1. reg (0x230021) : 0x 4000
14.2. reg (0x230021) : 0x 0
15.1. reg (0x23820A) : 0x 400
15.2. reg (0x23820A) : 0x 400
Link flapped at Mon Aug 24 11:00:52 2015
08-24-2015 08:01 PM
this problem not happened once, It happened for serval months, sometimes few interface were flapping , but this time many interfaces flapping at same time. including different slot, and some interface are connected directly by fiber, some inefaces are connected by DWDM
it confuse me.
08-27-2015 01:41 AM
Hi Osman
we check line in customer site, found all optical line connected to a device, and found problem happened on this device.
thank you!
but I have a question, from XFP mas specification, Tx fault Rx fault signal come from transceiver, IN nr there 3 signal , laser fault, Rx lock , TX lock, I don't understand what is Tx fault means, why outside optical line quality can make crs-1 transceiver generate Tx fault
Thank you
Tom
08-27-2015 08:54 AM
Hi Tom
We expect to see a remote fault in case of issues on the remote end. There is an option to print "Tx Fault" instead of "Detected Remote Fault" in the syslog but it has to be configured using a CLI using hw-module logging CLI.
Detected Remote Fault is the correct behaviour in this case.
08-27-2015 06:59 PM
Thank you!
08-25-2015 07:57 AM
tomorrow I will check other side routers, because there many routers , and different vendor. If these routers report similar log, I think it was transmission problem. but transmission engineer said their device was ok.
thank you for your help!
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: