Cross-Stack EtherChannel Between C3750 and C2960

Answered Question
Aug 20th, 2010

Hello experts,

Can cross-stack Etherchannel cannot be implemented between Cat3750 and Cat2960?

I recently have Forwarding loops issue in my customer's network.

The  setup is 3 Cat3750G-12S stack together using stackwise. they are  connected to 6 Cat2960G-24TC-L via cross-stack etherchannel. And I do  not know what happen, loops are being created. Can I confirm with you  once again that this setup is wrong? If its wrong, what should the  solution be to connect 6 Cat2960 to 3 Cat3750G stack switches?

Thanks a million in advance!!!

Regards

Ryan

I have this problem too.
0 votes
Correct Answer by Nagaraja Thanthry about 6 years 3 months ago

Hello,

Yes, I have seen such issues and mostly it was due to physical loops. If

there is a server with dual NICs connected to two different switches, or if

there is a wireless client trying to associate with different AP's, we

witness such MAC flaps. In your case, I do not see any reason why a MAC be

flapping between PO4 and PO7 if the two switches are on different floors

unless there is some kind of physical loop created between these two floors.

Also, some of the error messages indicate issues between gi 1/0/5 and PO7

(but on a different vlan). So, yes, changing the ports to access mode could

help. But more than that, I would suggest you investigating the physical

loop part.

Regards,

NT

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (1 ratings)
Loading.
Nagaraja Thanthry Fri, 08/20/2010 - 08:38

Hello,

When you say that you are using cross-stack etherchannel, are you referring

to one etherchannel per 2960 switch or one link per switch with etherchannel

on 3750 side?

Regards,

NT

Jayakrishna Mada Fri, 08/20/2010 - 09:00

Hi Ryan,

As NT mentioned can you describe the topology a little better.

1. Are the 6 2960 in a cluster or they individual ?

2. Cross stack from 3750 stack to an individual switches is fine, each in their own port-channel from both 3750 and 2960 sides, cannot be part of one single port-channel. And should be either MODE ON or LACP.

3. Do you see any spanning-tree discripancies ?

JayaKrishna

ryanhoibm Fri, 08/20/2010 - 09:17

Hello NT and JayaKrisna

Here is the Network Diagram. I got the error as shown below after implementation I really not sure what went wrong.

2602: Aug 14 16:41:46: %SW_MATM-4-MACFLAP_NOTIF: Host 001e.37d2.7259 in vlan 24 is flapping between port Po7 and port Po4
012603: Aug 14 16:41:47: %SW_MATM-4-MACFLAP_NOTIF: Host 0015.60b9.0044 in vlan 24 is flapping between port Po4 and port Po7
012604: Aug 14 16:41:47: %SW_MATM-4-MACFLAP_NOTIF: Host 00e0.0033.8d85 in vlan 25 is flapping between port Po7 and port Gi1/0/5
012605: Aug 14 16:41:47: %SW_MATM-4-MACFLAP_NOTIF: Host 0014.5e3f.4066 in vlan 1 is flapping between port Po10 and port Po7
012606: Aug 14 16:41:48: %SW_MATM-4-MACFLAP_NOTIF: Host 000c.2919.3585 in vlan 25 is flapping between port Gi1/0/5 and port Po7
012607: Aug 14 16:41:50: %SW_MATM-4-MACFLAP_NOTIF: Host 001d.096d.cf30 in vlan 1 is flapping between port Po2 and port Po7
012608: Aug 14 16:41:50: %SW_MATM-4-MACFLAP_NOTIF: Host 0015.5d3d.2f01 in vlan 1 is flapping between port Po7 and port Po1
012609: Aug 14 16:41:51: %SW_MATM-4-MACFLAP_NOTIF: Host 00e0.0033.c5aa in vlan 24 is flapping between port Po7 and port Po4
012610: Aug 14 16:41:51: %SW_MATM-4-MACFLAP_NOTIF: Host 001e.4f4d.dac6 in vlan 25 is flapping between port Po7 and port Gi1/0/5
012611: Aug 14 16:41:51: %SW_MATM-4-MACFLAP_NOTIF: Host 0021.5e2c.f1be in vlan 1 is flapping between port Po7 and port Po10
012612: Aug 14 16:41:56: %SW_MATM-4-MACFLAP_NOTIF: Host 000c.2919.3585 in vlan 25 is flapping between port Po7 and port Gi1/0/5
012613: Aug 14 16:41:56: %SW_MATM-4-MACFLAP_NOTIF: Host 0014.5e3f.3f5c in vlan 1 is flapping between port Po7 and port Po10
012614: Aug 14 16:41:57: %SW_MATM-4-MACFLAP_NOTIF: Host 0014.5e3f.3f5d in vlan 1 is flapping between port Po7 and port Po2
012615: Aug 14 16:41:57: %SW_MATM-4-MACFLAP_NOTIF: Host 0014.5e3f.3f46 in vlan 1 is flapping between port Po7 and port Po10
012616: Aug 14 16:41:58: %SW_MATM-4-MACFLAP_NOTIF: Host 0011.4337.cf12 in vlan 1 is flapping between port Po7 and port Po1
012617: Aug 14 16:41:58: %SW_MATM-4-MACFLAP_NOTIF: Host 0011.4337.cf11 in vlan 1 is flapping between port Po7 and port Po2
012618: Aug 14 16:41:58: %SW_MATM-4-MACFLAP_NOTIF: Host 0024.e891.5fc0 in vlan 24 is flapping between port Po4 and port Po7
012619: Aug 14 16:41:59: %SW_MATM-4-MACFLAP_NOTIF: Host 0014.5e3f.3f47 in vlan 1 is flapping between port Po2 and port Po7
012620: Aug 14 16:41:59: %SW_MATM-4-MACFLAP_NOTIF: Host 0015.60b9.1068 in vlan 25 is flapping between port Po7 and port Gi1/0/5
012621: Aug 14 16:42:00: %SW_MATM-4-MACFLAP_NOTIF: Host 001d.096d.cf30 in vlan 1 is flapping between port Po7 and port Po2
012622: Aug 14 16:42:00: %SW_MATM-4-MACFLAP_NOTIF: Host 001e.3739.46f8 in vlan 24 is flapping between port Po4 and port Po7
012623: Aug 14 16:42:01: %SW_MATM-4-MACFLAP_NOTIF: Host 705a.b696.d0c2 in vlan 24 is flapping between port Po4 and port Po7
012624: Aug 14 16:42:01: %SW_MATM-4-MACFLAP_NOTIF: Host 001e.37d2.7259 in vlan 24 is flapping between port Po4 and port Po7
012625: Aug 14 16:42:01: %SW_MATM-4-MACFLAP_NOTIF: Host 0015.60b9.0044 in vlan 24 is flapping between port Po4 and port Po7
012626: Aug 14 16:42:01: %SW_MATM-4-MACFLAP_NOTIF: Host 0023.32d5.b0ca in vlan 24 is flapping between port Po4 and port Po7
012627: Aug 14 16:42:02: %SW_MATM-4-MACFLAP_NOTIF: Host 00e0.0033.8d85 in vlan 25 is flapping between port Gi1/0/5 and port Po7
012628: Aug 14 16:42:05: %LINK-5-CHANGED: Interface GigabitEthernet2/0/5, changed state to administratively down
012629: Aug 14 16:42:05: %LINK-5-CHANGED: Interface GigabitEthernet3/0/7, changed state to administratively down
012630: Aug 14 16:42:05: %LINK-5-CHANGED: Interface Port-channel7, changed state to administratively down
012631: Aug 14 16:42:06: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet2/0/5, changed state to down
012632: Aug 14 16:42:06: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet3/0/7, changed state to down
012633: Aug 14 16:42:06: %LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel7, changed state to down
012634: Aug 14 16:42:15: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (180.2.24.2)
012635: Aug 14 17:04:53: %SYS-5-CONFIG_I: Configured from console by vty0 (127.0.0.5)
KW-L24-CSW01#$                                                            

Regards

Ryan

Jayakrishna Mada Fri, 08/20/2010 - 09:32

Ryan,

What you are doing is correct and supported.

The issue seems to be something related to PO7. It could be possible that on PO7 one side of the port-channel is bundled correctly and the other side its not.

Do you see any error messages or logs on the 2960 switch ?

Verify the configuration on both sides of po7.

JayaKrishna

Edison Ortiz Fri, 08/20/2010 - 09:25

The 3750 will be seen as one virtual switch due to its stacking capabilities while the 2960 is seen as one individual switch.

The src and dst on a bundle must be between the same virtual/logical switches.

If you bundle 2 links from the 3750 stack, the destination ports must be connected to the same physical 2960 ports.

Regards,

Edison

ryanhoibm Fri, 08/20/2010 - 09:36

Hi Edison,

The 3750 will be seen as one virtual switch due to its stacking capabilities while the 2960 is seen as one individual switch.

The src and dst on a bundle must be between the same virtual/logical switches.

Que: src and dst on a bundle must be between the same virtual/logical switches? does it mean that the etherchannel should be form between just ONE switch out of the 3 stack switches to a single 2960 instead of 1 port from switch 1 and 1 port from switch 2 in the stack connects to one 2960?

If you bundle 2 links from the 3750 stack, the destination ports must be connected to the same physical 2960 ports.

bundle 2 links from 3750 stack meaning cross stack right? yes the dst ports are connect to one physical 2960 ports.

example of my connection

2960 G0/1----------------- Stack Switch 1 Port G1/0/1

         G0/2------------------Stack Switch 2 Port G2/0/1

is this setup correct ?

Regards

Ryan

Nagaraja Thanthry Fri, 08/20/2010 - 09:46

Hello,

That setup is correct. Since 3750 Stack is virtually a single switch, you

can connect Gi 1/0/1 and Gi 2/0/1 to gi 0/1 and gi 0/2 of one 2960.

Regards,

NT

Edison Ortiz Fri, 08/20/2010 - 09:47

Que: src and dst on a bundle must be between the same virtual/logical switches? does it mean that the etherchannel should be form between just ONE switch out of the 3 stack switches to a single 2960 instead of 1 port from switch 1 and 1 port from switch 2 in the stack connects to one 2960?

It's seen as one virtual switch so from the 3750 you can form the bundle from any 3750 physical switch.

The limitation is at the 2960 side where the destination must be on the same switch.

example of my connection

2960 G0/1----------------- Stack Switch 1 Port G1/0/1

         G0/2------------------Stack Switch 2 Port G2/0/1





is this setup correct ?

Yes

ryanhoibm Fri, 08/20/2010 - 10:02

Thanks NT and Edison.

One more question, as you can see there are 6 C2960 in the diagram. From top to bottom, first 2 are from the same floor.

so from the same floor, there are 2 c2960 with 2 port channels to the stack switches. both port channels are in forwarding mode since both are from separate physical C2960 switches even though they are from the same floor. Will that cause spanning tree issues?

If 2 port channels between one physical 2960 to C3750, I am certain one of the port channel will be in blocking/standby state but since its from 2 physical switches so I am concern if it will cause STP issues.

Any advise and recommendations?

Regards

Thanks

Ryan

Nagaraja Thanthry Fri, 08/20/2010 - 10:24

Hello,

Thanks for the diagram. One question I have for you is the MAC addresses

that are flapping, are they by any chance wireless clients? Do you have

multiple access points in close vicinity?

Regards,

NT

ryanhoibm Fri, 08/20/2010 - 16:57

Hi NT,

Nope there aren't any wireless setup. Those MAC addresses are servers which resides at the server farm switches above the stack switches not shown in the diagram.

regards

Ryan

Nagaraja Thanthry Fri, 08/20/2010 - 17:05

Hello,

I see that PO4 and PO7 go to two different switches. Are they connected to

each other?

Regards,

NT

ryanhoibm Fri, 08/20/2010 - 17:13

Hi NT,

Nope PO4 is at 24th flr and PO7 is at 26th flr if you look at the diagram which indicates all the POs.

I am thinking of implementing BPDU Guard on every user ports and Loop Guard globally. And I will convert the trunk and Port Channel links to access ports. Since its per floor Vlan, I will configure all uplinks on 24th flr to be on access vlan 24. Hope that will solve all these strange MAC flaps. And I will double check if the uplinks are not connected correctly to the patch panel.

Have you ever encounter this from your customer???

Regards

Ryan

ryanhoibm Fri, 08/20/2010 - 17:36

Hi Manish,

Thanks for the link. I have seen it before I post this discussion

Regards

Ryan

Correct Answer
Nagaraja Thanthry Sat, 08/21/2010 - 00:23

Hello,

Yes, I have seen such issues and mostly it was due to physical loops. If

there is a server with dual NICs connected to two different switches, or if

there is a wireless client trying to associate with different AP's, we

witness such MAC flaps. In your case, I do not see any reason why a MAC be

flapping between PO4 and PO7 if the two switches are on different floors

unless there is some kind of physical loop created between these two floors.

Also, some of the error messages indicate issues between gi 1/0/5 and PO7

(but on a different vlan). So, yes, changing the ports to access mode could

help. But more than that, I would suggest you investigating the physical

loop part.

Regards,

NT

ryanhoibm Sat, 08/21/2010 - 05:24

Hi NT,

I agreed. I will need to investigate on the physical connection. Thanks alot!!!

Regards

Ryan Ho

ryanhoibm Mon, 08/23/2010 - 03:14

Hi NT,

I have a question.

If

there is a server with dual NICs connected to two different switches, or if

there is a wireless client trying to associate with different AP's, we

witness such MAC flaps

Servers with Dual NIC connecting to 2 different switches are for redundancy purposes. So what should be the correct way to connect servers with dual NIC? I thought Dual NIC has 2 different MAC addresses and since both the MAC addresses are different, why will it cause MAC flaps?

Regards

Ryan

Nagaraja Thanthry Tue, 08/24/2010 - 09:36

Hello,

While it is true that each NIC will have its own MAC address, typically, when you configure NIC teaming, you will configure a virtual MAC that is shared by both NIC cards. Also, one another thing that people generally use is Bridging. If you are doing either one of these two and if the teaming broke, then there will be a physical loop. Since the MAC address is shared by both NIC cards (for redundancy), you will see it flapping between the interfaces.

Regards,

NT

Actions

This Discussion