Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements

Welcome to Cisco Support Community. We would love to have your feedback.

For an introduction to the new site, click here. And see here for current known issues.

New Member

QinQ & MPLS Layer 2

dear All,

i have come accross this following network scenario and got kind of stuck.

qinq-mpls-l2.jpg

The Service Provider is connecting two remote sites of its customer with customer's head office.

Requirement of the customer is to provide them Trunk Links to all the remote sites as well as at the head office so that they can carry their own VLAN information. Basically they have only switched network.

Customer's internal setup is as follows:

VLAN: 1 - PC/Printers etc.

VLAN: 97 - MGT IP of Switches

VLAN: 99 - IP Phone [this is configured as voice VLAN]

VLAN: 100 - for Cisco Call Manager

All of customer's switches are either catalyst 3560 or 3750. at head office there is one switch acting as a core switch [catalyst 3750 / 3 stacked switches] to provide all the trunk links over Fiber Optic links as well as acting as Access Gateway. also all these switches are in VTP TRANSPARENT Mode and all the above mentioned VLANs are manually created in each switch prior to deployment. all the switches are using VLAN 97 as their MGT access but native vlan is 1 as default.

it is also been confirmed that there is no physical loop in between the  switches at head office end or any of the remote sites of customer. as for connecting with SP network customer has to use two switch ports [cofigured as trunk] for connecting two of their remote sites.

at the SP end switches are catalyst 3550 and routers used are 7206VXR-NPE-G1.

for each remote sites SP has configured a QinQ tunnel with a unique VLAN ID and carried that tunnel through MPLS L2 circuits to the remote site where the switch ports are also configured as QinQ tunnel ports with corresponding VLAN ids. these remote ports are then connected with customers LAN switches [trunk ports].

all trunk port settings of Customer switches are as follows:

int g0/XXX

switchport trunk encapsulation dot1q

switchport mode trunk

switchport trunk allowed vlan all

Configuration details for SP end Routers/Switches are as follows:

please note that vlan 641 and 648 has been removed from spanning-tree at respective SP sites using:

"no spanning-tree vlan 641/no spanning-tree vlan 648"

native vlan tagging is not enabled in any switches of SP or Customer.

Switches at SP Site:

SP Head Office end configuration:

-------------------------------------------------

interface FastEthernet0/30

description To-Location-1

switchport access vlan 641

switchport mode dot1q-tunnel

l2protocol-tunnel cdp

l2protocol-tunnel stp

l2protocol-tunnel vtp

no cdp enable

spanning-tree bpdufilter enable

end

interface FastEthernet0/31

description To-Location-2

switchport access vlan 648

switchport mode dot1q-tunnel

l2protocol-tunnel cdp

l2protocol-tunnel stp

l2protocol-tunnel vtp

no cdp enable

spanning-tree bpdufilter enable

end

SP Remote end configuration - Location 1:

-------------------------------------------------------------

interface FastEthernet0/10

description To-Location-1-LAN

switchport access vlan 641

switchport mode dot1q-tunnel

l2protocol-tunnel cdp

l2protocol-tunnel stp

l2protocol-tunnel vtp

no cdp enable

spanning-tree bpdufilter enable

end

SP Remote end configuration - Location 2:

-------------------------------------------------------------

interface FastEthernet0/8

description To-Location-2-LAN

switchport access vlan 648

switchport mode dot1q-tunnel

l2protocol-tunnel cdp

l2protocol-tunnel stp

l2protocol-tunnel vtp

no cdp enable

spanning-tree bpdufilter enable

end

Routers at SP Sites:

SP Head Office:

-----------------------

interface FastEthernet1/0.641

description Location-1

encapsulation dot1Q 641

no snmp trap link-status

no cdp enable

mpls l2transport route X.X.127.231 641

end

interface FastEthernet1/0.648

description Location-2

encapsulation dot1Q 648

no snmp trap link-status

no cdp enable

mpls l2transport route X.X.250.221 648

end

SP Remote Site: Location - 1

------------------------------------------

interface FastEthernet0/0.641

description Location-1

encapsulation dot1Q 641

no snmp trap link-status

no cdp enable

mpls l2transport route X.X.127.204 641

end

SP Remote Site: Location - 2

------------------------------------------

interface FastEthernet2/0.648

description Location-2

encapsulation dot1Q 648

no snmp trap link-status

no cdp enable

mpls l2transport route X.X.127.204 648

end

With these configurations customer is facing the following issues:

1. if any one site is connected with head office then all services are running smoothly. IP phone calls are established, LAN Pcs are being able to browse properly. switches are accessible.

2. if both sites are connected then the following issues are raised:

  • no calls can be established using IP Phones but rings can be heard. if the call is received then there is no voice.
  • icmp is available in random manner. i mean not all the equiments which are available via icmp if one link is connected can be reached if both the links are alive.
  • PCs can not reach Internet. and sometimes they have problem reaching gateway as well.
  • most of the times DHCP doesn't work.

all these are resolved instantly when either one of the locations are shutdown.

Dynamic MAC Address Table in SP switches shows that MAC is propagating accross the provider network from each remote sites to head office and the way back.

however i have noticed a problem in the MAC flow. during the time when both the links were alive.

MAC address of Remote Switch at Location 1:      xxxx.xxxx.d0e1

this MAC was found at Port Fa0/31 of SP Head Office end Switch instead of Fa0/30 [ show mac address-table dynamic | include d0e1 ] and this switch was not accessible from HO.

Till now I have come accross all these and could not resolve.

Please suggest what need to be done to overcome the situation when no network service is working if both the remote locations are connected. Also please feel free to knock for any further queries ....

Looking forward for suggestions as well as Thank You in Advance.

Tahmeed

4 REPLIES
New Member

QinQ & MPLS Layer 2

If everything works OK when only one remote site is conneted to the HQ, and both sites are confirmed as work when the other is disconnected, then that points to HQ being the problem site here.

Do you have redunadant links within HQ and no spanning tree running so there is a loop? High CPU usage on the devices there? Also why not use different voice and data VLANs for the different remotes sites? Broadcasts from Site1 are going through the SP network to the HQ, and back out to site 2? This isn't a very efficient use of the links (if I have understood your topology correctly).

I would focus on the HQ site from what you have described. Can you verify there are no loops? Also can you follow the MAC address of a client who is performing a DHCP request for example, comming in and out of every switch between the client and HQ site when both sites are connected to see how far the request gets? If it makes it all the way to the DHCP server in the HQ site (assuming that's where it is), can you follow the reply back to the requesting client in the remote site again?

New Member

Re: QinQ & MPLS Layer 2

hi! jwbensley

i haven't been able to trace DHCP requests through out each hop in the network but will do that. and yes the servers are at HQ. and i am sure that there is no loop in any of customer's sites. however STP is running at each of the Customer switches. but none of the switches connected with SP network shows any blocking information.

as for using different vlans for voice/data won't be possible due to organizations internal & international policy.

also, my googling came up with an information about MAC Based Forwarding in switches. but i couldn't find if this is a configurable parameter in catalyst 3550/3560/3750 switches. could you pls suggest.

Thanks for your feedback.

Tahmeed

Message was edited by: Q S Tahmeed Edited/Added: forgot to mention that when both the locations are active "show cdp neighbor" at HQ switch shows each of the switches at remote sites. 

New Member

Re: QinQ & MPLS Layer 2

dear All,

it was later found that the issue arised due to low quality Fibre Optic media converter at Head Office end which simply couldn't handle the load. after replacing it with a better pair the issue is resolved.

Thank you all for you time and effort.

Tahmeed

Wow, I was starting to worry

Wow, I was starting to worry about the design problem and starting to suspect that headend SP interface is supposed to only handle 1 pseudowire per physical interface. Good to know that the design is OK. Time for me to lab up this scenario in VIRL.

3507
Views
0
Helpful
4
Replies
CreatePlease login to create content