05-07-2015 01:29 AM
Dear Team
CRS-x 16 , 5.1.3 has been configured with load balancing "cef load-balancing fields L4."
CEF algo being responsible to load balance the traffic on bundle ether links. An issue was observed that after link flaps, there was abnormal load balancing ( logs below) . Load balancing algorithm didn’t trigger upto the expectations after the links flapped in the bundle in the beginning. No error message or crash/tracebacks observed at the logs . Customer reset tenGigE 0/2/0/5 which was carrying 3% OUT traffic and load-balancing normalized.
Please share your inputs and advise if you need more information.
++++++++++ Before the interface was shut : -
RP/0/RP0/CPU0:FJN-EMIX-RC#show bundle bundle-ether 5
Mon Apr 20 23:47:04.912 Asia/AbuDhabi
Bundle-Ether5
Status: Up
Local links <active/standby/configured>: 6 / 0 / 6
Local bandwidth <effective/available>: 60000000 (60000000) kbps
MAC address (source): f44e.05f4.4e00 (Chassis pool)
Inter-chassis link: No
Minimum active links / bandwidth: 1 / 1 kbps
Maximum active links: 64
Wait while timer: 2000 ms
Load balancing: Default
LACP: Not operational
Flap suppression timer: Off
Cisco extensions: Disabled
mLACP: Not configured
IPv4 BFD: Not configured
Port Device State Port ID B/W, kbps
-------------------- --------------- ----------- -------------- ----------
Te0/2/0/5 Local Active 0x8000, 0x002e 10000000
Link is Active
Te0/3/0/5 Local Active 0x8000, 0x0006 10000000
Link is Active
Te0/4/0/5 Local Active 0x8000, 0x000c 10000000
Link is Active
Te0/5/0/5 Local Active 0x8000, 0x0029 10000000
Link is Active
Te0/11/0/5 Local Active 0x8000, 0x0018 10000000
Link is Active
Te0/12/0/5 Local Active 0x8000, 0x0021 10000000
Link is Active
RP/0/RP0/CPU0:FJN-EMIX-RC#monitor interface Te0/2/0/5 Te0/3/0/5 Te0/4/0/5 Te0/5/0/5 Te0/11/0/5 Te0/12/0/5
Mon Apr 20 23:53:43.734 Asia/AbuDhabi
FJN-EMIX-RC Monitor Time: 00:00:01 SysUptime: 834:23:11
Protocol:General
Interface In(bps) Out(bps) InBytes/Delta OutBytes/Delta
Te0/2/0/5 689.2M/ 6% 383.3M/ 3% 235.7T/0 1337T/0
Te0/3/0/5 682.2M/ 6% 388.8M/ 3% 233.8T/0 1337T/0
Te0/4/0/5 725.3M/ 7% 386.2M/ 3% 237.6T/0 1338T/0
Te0/5/0/5 781.3M/ 7% 10.3G/103% 228.6T/0 1395T/0 <<<<<<<<<<<<<<
Te0/11/0/5 764.3M/ 7% 379.4M/ 3% 237.4T/0 1337T/0
Te0/12/0/5 742.1M/ 7% 10.3G/103% 234.3T/0 1396T/0 <<<<<<<<<<<<<<
++++++++++++++++++
RP/0/RP0/CPU0:FJN-EMIX-RC(config)#interface tenGigE 0/2/0/5 <<<<<<<<<< ( has 3% traffic )
RP/0/RP0/CPU0:FJN-EMIX-RC(config-if)#shut
RP/0/RP0/CPU0:FJN-EMIX-RC(config-if)#commit
Tue Apr 21 00:42:30.540 Asia/AbuDhabi
LC/0/12/CPU0:Apr 21 00:42:30.579 Asia/AbuDhabi: BM-ADJ[128]: %L2-BM_PD-6-INFO_MSG : BFC: Trigger for Bundle-IFH 0x800f0 with Members 1 batch 0/1
LC/0/7/CPU0:Apr 21 00:42:30.580 Asia/AbuDhabi: BM-ADJ[128]: %L2-BM_PD-6-INFO_MSG : BFC: Trigger for Bundle-IFH 0x800f0 with Members 1 batch 0/1
LC/0/15/CPU0:Apr 21 00:42:30.579 Asia/AbuDhabi: BM-ADJ[128]: %L2-BM_PD-6-INFO_MSG : BFC: Trigger for Bundle-IFH 0x800f0 with Members 1 batch 0/1
LC/0/5/CPU0:Apr 21 00:42:30.580 Asia/AbuDhabi: BM-ADJ[128]: %L2-BM_PD-6-INFO_MSG : BFC: Trigger for Bundle-IFH 0x800f0 with Members 1 batch 0/1
LC/0/8/CPU0:Apr 21 00:42:30.579 Asia/AbuDhabi: BM-ADJ[128]: %L2-BM_PD-6-INFO_MSG : BFC: Trigger for Bundle-IFH 0x800f0 with Members 1 batch 0/1
LC/0/11/CPU0:Apr 21 00:42:30.578 Asia/AbuDhabi: BM-ADJ[128]: %L2-BM_PD-6-INFO_MSG : BFC: Trigger for Bundle-IFH 0x800f0 with Members 1 batch 0/1
LC/0/13/CPU0:Apr 21 00:42:30.579 Asia/AbuDhabi: BM-ADJ[128]: %L2-BM_PD-6-INFO_MSG : BFC: Trigger for Bundle-IFH 0x800f0 with Members 1 batch 0/1
LC/0/2/CPU0:Apr 21 00:42:30.580 Asia/AbuDhabi: ifmgr[194]: %PKT_INFRA-LINK-5-CHANGED : Interface TenGigE0/2/0/5, changed state to Administratively Down
RP/0/RP0/CPU0:Apr 21 00:42:30.584 Asia/AbuDhabi: BM-DISTRIB[1148]: %L2-BM-6-ACTIVE : TenGigE0/2/0/5 is no longer Active as part of Bundle-Ether5 (Link is shutdown)
LC/0/2/CPU0:Apr 21 00:42:30.581 Asia/AbuDhabi: ethernet_link_oam_daemon[164]: %L2-ELO-6-EFD_CHANGE : Interface TenGigE0/2/0/5 shut down using EFD due to session down
RP/0/RP0/CPU0:Apr 21 00:42:30.775 Asia/AbuDhabi: config[65876]: %MGBL-CONFIG-6-DB_COMMIT : Configuration committed by user 'sachin'. Use 'show configuration commit changes 1000000466' to view the changes.
RP/0/RP0/CPU0:FJN-EMIX-RC(config-if)#
RP/0/RP0/CPU0:FJN-EMIX-RC(config-if)#
RP/0/RP0/CPU0:FJN-EMIX-RC(config-if)#
RP/0/RP0/CPU0:FJN-EMIX-RC(config-if)#
RP/0/RP0/CPU0:Apr 21 00:42:34.835 Asia/AbuDhabi: config[65876]: %MGBL-SYS-5-CONFIG_I : Configured from console by sachin on vty0 (194.170.14.37)
RP/0/RP0/CPU0:FJN-EMIX-RC#
RP/0/RP0/CPU0:FJN-EMIX-RC#
RP/0/RP0/CPU0:FJN-EMIX-RC#
RP/0/RP0/CPU0:FJN-EMIX-RC#monitor interface Te0/2/0/5 Te0/3/0/5 Te0/4/0/5 Te0/5/0/5 Te0/11/0/5 Te0/12/0/5
Tue Apr 21 00:42:37.522 Asia/AbuDhabi
FJN-EMIX-RC Monitor Time: 00:00:00 SysUptime: 835:12:04
Protocol:General
Interface In(bps) Out(bps) InBytes/Delta OutBytes/Delta
Te0/2/0/5 456.8M/ 4% 271.3M/ 2% 236.0T/0 1337T/0
Te0/3/0/5 654.2M/ 6% 377.2M/ 3% 234.0T/0 1337T/0
Te0/4/0/5 626.8M/ 6% 389.2M/ 3% 237.8T/0 1338T/0
Te0/5/0/5 659.0M/ 6% 7.9G/ 79% 228.8T/0 1398T/0
Te0/11/0/5 667.4M/ 6% 427.8M/ 4% 237.6T/0 1337T/0
Te0/12/0/5 640.8M/ 6% 10.2G/102% 234.6T/0 1399T/0
Protocol:General
Interface In(bps) Out(bps) InBytes/Delta OutBytes/Delta
Te0/2/0/5 28.7M/ 0% 17.1M/ 0% 236.0T/0 1337T/0 <<<<<<<<<<<<< SHUT
Te0/3/0/5 744.5M/ 7% 8.4G/ 84% 234.0T/182.0M 1337T/2.0G
Te0/4/0/5 748.2M/ 7% 8.5G/ 85% 237.8T/182.5M 1338T/2.0G
Te0/5/0/5 718.6M/ 7% 8.8G/ 88% 228.8T/180.8M 1398T/2.1G
Te0/11/0/5 762.5M/ 7% 8.4G/ 84% 237.6T/189.6M 1338T/2.1G
Te0/12/0/5 742.6M/ 7% 9.0G/ 90% 234.6T/186.9M 1399T/2.0G
Quit='q', Clear='c', Freeze='f', Thaw='t',
Next set='n', Prev set='p', Bytes='y', Packets='k'
(General='g', IPv4 Uni='4u', IPv4 Multi='4m', IPv6 Uni='6u', IPv6 Multi='6m')
RP/0/RP0/CPU0:FJN-EMIX-RC#
RP/0/RP0/CPU0:FJN-EMIX-RC#
RP/0/RP0/CPU0:FJN-EMIX-RC#
RP/0/RP0/CPU0:FJN-EMIX-RC#conf t
Tue Apr 21 00:44:00.703 Asia/AbuDhabi
RP/0/RP0/CPU0:FJN-EMIX-RC(config)#
RP/0/RP0/CPU0:FJN-EMIX-RC(config)#interface tenGigE 0/2/0/5
RP/0/RP0/CPU0:FJN-EMIX-RC(config-if)#no shut
RP/0/RP0/CPU0:FJN-EMIX-RC(config-if)#commit
Tue Apr 21 00:44:06.475 Asia/AbuDhabi
LC/0/2/CPU0:Apr 21 00:44:06.551 Asia/AbuDhabi: ifmgr[194]: %PKT_INFRA-LINK-3-UPDOWN : Interface TenGigE0/2/0/5, changed state to Down
LC/0/2/CPU0:Apr 21 00:44:06.551 Asia/AbuDhabi: ifmgr[194]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface TenGigE0/2/0/5, changed state to Down
RP/0/RP0/CPU0:Apr 21 00:44:06.789 Asia/AbuDhabi: config[65876]: %MGBL-CONFIG-6-DB_COMMIT : Configuration committed by user 'sachin'. Use 'show configuration commit changes 1000000467' to view the changes.
RP/0/RP0/CPU0:FJN-EMIX-RC(config-if)#
RP/0/RP0/CPU0:Apr 21 00:44:07.100 Asia/AbuDhabi: config[65876]: %MGBL-SYS-5-CONFIG_I : Configured from console by sachin on vty0 (194.170.14.37)
RP/0/RP0/CPU0:FJN-EMIX-RC#
RP/0/RP0/CPU0:FJN-EMIX-RC#
RP/0/RP0/CPU0:FJN-EMIX-RC#
RP/0/RP0/CPU0:FJN-EMIX-RC#LC/0/2/CPU0:Apr 21 00:44:07.886 Asia/AbuDhabi: ifmgr[194]: %PKT_INFRA-LINK-3-UPDOWN : Interface TenGigE0/2/0/5, changed state to Up
LC/0/2/CPU0:Apr 21 00:44:07.891 Asia/AbuDhabi: ethernet_link_oam_daemon[164]: %L2-ELO-6-EFD_CHANGE : Interface TenGigE0/2/0/5 brought up using EFD due to all faults being cleared
RP/0/RP0/CPU0:Apr 21 00:44:07.895 Asia/AbuDhabi: BM-DISTRIB[1148]: %L2-BM-6-ACTIVE : TenGigE0/2/0/5 is Active as part of Bundle-Ether5
LC/0/2/CPU0:Apr 21 00:44:07.891 Asia/AbuDhabi: ethernet_link_oam_daemon[164]: %L2-ELO-6-INTF_SESSION_UP : TenGigE0/2/0/5: OAM session has come up
LC/0/2/CPU0:Apr 21 00:44:07.891 Asia/AbuDhabi: ifmgr[194]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface TenGigE0/2/0/5, changed state to Up
monitor interface Te0/2/0/5 Te0/3/0/5 Te0/4/0/5 Te0/5/0/5 Te0/11/0/5 Te0/12/0/5
Tue Apr 21 00:44:09.859 Asia/AbuDhabi
FJN-EMIX-RC Monitor Time: 00:00:01 SysUptime: 835:13:38
Protocol:General
Interface In(bps) Out(bps) InBytes/Delta OutBytes/Delta
Te0/2/0/5 59.9M/ 0% 10.7M/ 0% 236.0T/0 1337T/0
Te0/3/0/5 741.9M/ 7% 8.4G/ 84% 234.0T/0 1337T/0
Te0/4/0/5 742.9M/ 7% 8.3G/ 83% 237.8T/0 1338T/0
Te0/5/0/5 726.3M/ 7% 8.6G/ 86% 228.9T/0 1398T/0
Te0/11/0/5 757.5M/ 7% 8.3G/ 83% 237.6T/0 1338T/0
Te0/12/0/5 742.7M/ 7% 8.6G/ 86% 234.6T/0 1399T/0
Quit='q', Clear='c', Freeze='f', Thaw='t',
Next set='n', Prev set='p', Bytes='y', Packets='k'
(General='g', IPv4 Uni='4u', IPv4 Multi='4m', IPv6 Uni='6u', IPv6 Multi='6m')
FJN-EMIX-RC Monitor Time: 00:00:02 SysUptime: 835:13:38
********************** After some time , traffic load balanced
Protocol:General
Interface In(bps) Out(bps) InBytes/Delta OutBytes/Delta
Te0/2/0/5 591.2M/ 5% 4.6G/ 46% 236.0T/148.6M 1337T/1.1G
Te0/3/0/5 612.5M/ 6% 4.7G/ 47% 234.0T/153.0M 1338T/1.1G
Te0/4/0/5 590.9M/ 5% 4.8G/ 48% 237.8T/151.4M 1339T/1.2G
Te0/5/0/5 621.5M/ 6% 4.9G/ 49% 228.9T/156.5M 1398T/1.2G
Te0/11/0/5 595.7M/ 5% 4.9G/ 49% 237.6T/148.9M 1338T/1.2G
Te0/12/0/5 590.5M/ 5% 4.8G/ 48% 234.6T/45.8M 1399T/373.8M
Quit='q', Clear='c', Freeze='f', Thaw='t',
Next set='n', Prev set='p', Bytes='y', Packets='k'
(General='g', IPv4 Uni='4u', IPv4 Multi='4m', IPv6 Uni='6u', IPv6 Multi='6m')
RP/0/RP0/CPU0:FJN-EMIX-RC#
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: