Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
New Member

Multicast Dilemmas. Help!

Hi all, we've been experiencing some multicast problems and I'm hopeful that the experts out there can give me a hand figuring them out before I decide to throw in the towel and put in place GRE tunneling.

Some facts about our network:

  • Each of our PE has at least 3 uplinks to at least 2 different Pops.
  • Each PE receives about  90k BGP routes from our 4 route-reflectors with a default route (0/0) for any missing specific routes.
  • Each PE receives about 3k  multicast routes from our 4 route-reflectors with a default route (0/0) for any missing specific routes.
  • ISISI is our IGP (L1 only, multitopology -- IPv6 unicast/multicast is running as well on all PE with the full ipv6 table (18k))
  • RFP check is running on all backbone line in loose mode (ip verify unicast source reachable-via any)
  • PIM sparse mode is active on all backbone links.
  • There are multiple paths with equal cost metrics on all backbone links and inbound/outbound traffic is not symmetrical (...yes I know)
  • There are 2 anycast RP (one in the north and the second in the south).  IGP metrics determine which is used.

 

The problem...

One of our clients it trying to join a group but he's having problems doing so.  The configuration of his interface is correct.

From the PE he is connected to as well as within our lab, I can join the group that he is interested in on the router using igmp:

---------------

LAB-NOC-7200-GATEWAY(config-subif)#ip igmp join-group 233.29.151.44

LAB-NOC-7200-GATEWAY#sh ip igmp groups

IGMP Connected Group Membership

Group Address    Interface                Uptime    Expires   Last Reporter   Group Accounted

233.29.151.44    GigabitEthernet0/2.431   00:00:10  00:02:49  193.51.177.0

224.0.1.40       GigabitEthernet0/2.431   1w1d      00:02:58  193.51.177.0

--------------------------------------------------------

The MROUTE results are coherent:

-------------------------------------------------------

LAB-NOC-7200-GATEWAY#sh ip mroute

IP Multicast Routing Table

Flags: D - Dense, S - Sparse, B - Bidir Group, s - SSM Group, C - Connected,

       L - Local, P - Pruned, R - RP-bit set, F - Register flag,

       T - SPT-bit set, J - Join SPT, M - MSDP created entry,

       X - Proxy Join Timer Running, A - Candidate for MSDP Advertisement,

       U - URD, I - Received Source Specific Host Report,

       Z - Multicast Tunnel, z - MDT-data group sender,

       Y - Joined MDT-data group, y - Sending to MDT-data group,

       V - RD & Vector, v - Vector

Outgoing interface flags: H - Hardware switched, A - Assert winner

Timers: Uptime/Expires

Interface state: Interface, Next-Hop or VCD, State/Mode

 

(*, 233.29.151.44), 00:02:48/stopped, RP 193.51.178.254, flags: SJPL

  Incoming interface: Tunnel650580, RPF nbr 193.51.177.6

  Outgoing interface list: Null

(202.158.196.68, 233.29.151.44), 00:01:39/00:02:57, flags: PLT

  Incoming interface: Tunnel650580, RPF nbr 193.51.177.6, Mbgp

  Outgoing interface list: Null

(202.158.195.196, 233.29.151.44), 00:01:41/00:02:58, flags: PLT

  Incoming interface: Tunnel650580, RPF nbr 193.51.177.6, Mbgp

  Outgoing interface list: Null

(129.78.221.17, 233.29.151.44), 00:01:43/00:02:59, flags: PLT

  Incoming interface: Tunnel650580, RPF nbr 193.51.177.6, Mbgp

  Outgoing interface list: Null

(202.6.112.4, 233.29.151.44), 00:01:46/00:02:57, flags: PLT

  Incoming interface: Tunnel650580, RPF nbr 193.51.177.6, Mbgp

  Outgoing interface list: Null

(202.158.196.134, 233.29.151.44), 00:01:46/00:02:57, flags: PLT

  Incoming interface: Tunnel650580, RPF nbr 193.51.177.6, Mbgp

  Outgoing interface list: Null

(202.158.195.132, 233.29.151.44), 00:01:48/00:02:55, flags: PLT

  Incoming interface: Tunnel650580, RPF nbr 193.51.177.6, Mbgp

  Outgoing interface list: Null

(202.158.196.4, 233.29.151.44), 00:01:48/00:02:58, flags: PLT

  Incoming interface: Tunnel650580, RPF nbr 193.51.177.6, Mbgp

  Outgoing interface list: Null

(202.158.195.68, 233.29.151.44), 00:01:48/00:02:53, flags: PLT

  Incoming interface: Tunnel650580, RPF nbr 193.51.177.6, Mbgp

  Outgoing interface list: Null

(138.77.139.71, 233.29.151.44), 00:01:48/00:02:53, flags: PLT

  Incoming interface: Tunnel650580, RPF nbr 193.51.177.6, Mbgp

  Outgoing interface list: Null

(138.77.155.72, 233.29.151.44), 00:01:48/00:02:58, flags: PLT

  Incoming interface: Tunnel650580, RPF nbr 193.51.177.6, Mbgp

  Outgoing interface list: Null

(131.172.135.210, 233.29.151.44), 00:01:49/00:02:58, flags: PLT

  Incoming interface: Tunnel650580, RPF nbr 193.51.177.6, Mbgp

  Outgoing interface list: Null

(203.27.221.247, 233.29.151.44), 00:01:49/00:02:59, flags: PLT

  Incoming interface: Tunnel650580, RPF nbr 193.51.177.6, Mbgp

  Outgoing interface list: Null

(131.172.8.214, 233.29.151.44), 00:01:49/00:02:56, flags: PLT

  Incoming interface: Tunnel650580, RPF nbr 193.51.177.6, Mbgp

  Outgoing interface list: Null

(202.158.195.4, 233.29.151.44), 00:01:51/00:02:58, flags: PLT

  Incoming interface: Tunnel650580, RPF nbr 193.51.177.6, Mbgp

  Outgoing interface list: Null

(195.220.53.138, 233.29.151.44), 00:01:51/00:02:59, flags: PLT

  Incoming interface: Tunnel650580, RPF nbr 193.51.177.6, Mbgp

  Outgoing interface list: Null

(134.129.95.122, 233.29.151.44), 00:01:51/00:02:58, flags: PLT

  Incoming interface: Tunnel650580, RPF nbr 193.51.177.6, Mbgp

  Outgoing interface list: Null

(138.77.147.71, 233.29.151.44), 00:01:51/00:02:59, flags: PLT

  Incoming interface: Tunnel650580, RPF nbr 193.51.177.6, Mbgp

  Outgoing interface list: Null

(*, 224.0.1.40), 1w1d/00:02:17, RP 0.0.0.0, flags: DPL

  Incoming interface: Null, RPF nbr 0.0.0.0

  Outgoing interface list: Null

------------------------------------------------

The mulitcast packets arrive on our lab 7200 and packets received increase.

------------------------------------------------------

LAB-NOC-7200-GATEWAY#sh ip mroute count

IP Multicast Statistics

19 routes using 6536 bytes of memory

2 groups, 8.50 average sources per group

Forwarding Counts: Pkt Count/Pkts(neg(-) = Drops) per second/Avg Pkt Size/Kilobits per second

Other counts: Total/RPF failed/Other drops(OIF-null, rate-limit etc)

 

Group: 233.29.151.44, Source count: 17, Packets forwarded: 0, Packets received: 8248

  RP-tree: Forwarding: 0/0/0/0, Other: 0/0/0

  Source: 202.158.195.132/32, Forwarding: 0/-1/0/0, Other: 398/0/398

  Source: 202.158.195.4/32, Forwarding: 0/-2/0/0, Other: 499/0/499

  Source: 138.77.147.71/32, Forwarding: 0/-1/0/0, Other: 630/0/630

  Source: 129.78.221.17/32, Forwarding: 0/-1/0/0, Other: 450/0/450

  Source: 202.6.112.4/32, Forwarding: 0/-2/0/0, Other: 492/0/492

  Source: 202.158.195.196/32, Forwarding: 0/-3/0/0, Other: 468/0/468

  Source: 202.158.195.68/32, Forwarding: 0/-1/0/0, Other: 397/0/397

  Source: 195.220.53.138/32, Forwarding: 0/-4/0/0, Other: 539/0/539

  Source: 131.172.135.210/32, Forwarding: 0/-2/0/0, Other: 458/0/458

  Source: 203.27.221.247/32, Forwarding: 0/-1/0/0, Other: 398/0/398

  Source: 138.77.139.71/32, Forwarding: 0/-1/0/0, Other: 398/0/398

  Source: 131.172.8.214/32, Forwarding: 0/-1/0/0, Other: 537/0/537

  Source: 134.129.95.122/32, Forwarding: 0/-2/0/0, Other: 748/0/748

  Source: 138.77.155.72/32, Forwarding: 0/-2/0/0, Other: 443/0/443

  Source: 202.158.196.134/32, Forwarding: 0/-1/0/0, Other: 473/0/473

  Source: 202.158.196.4/32, Forwarding: 0/-1/0/0, Other: 458/0/458

  Source: 202.158.196.68/32, Forwarding: 0/-2/0/0, Other: 462/0/462

Group: 224.0.1.40, Source count: 0, Packets forwarded: 0, Packets received: 0

 

------------------------------------------------------DEBUG-----------------------------------

 

LAB-NOC-7200-GATEWAY#debug ip mpacket

IP multicast packets debugging is on

LAB-NOC-7200-GATEWAY#

LAB-NOC-7200-GATEWAY#

LAB-NOC-7200-GATEWAY#term

LAB-NOC-7200-GATEWAY#terminal moni

LAB-NOC-7200-GATEWAY#terminal monitor

LAB-NOC-7200-GATEWAY# (Tunnel650580) d=233.29.151.44 id=0, ttl=113, prot=17, len=40(40), mroute olist null

Nov  6 11:15:13: IP(0): s=203.27.221.247 (Tunnel650580) d=233.29.151.44 id=0, ttl=109, prot=17, len=547(547), mroute olist null

Nov  6 11:15:13: IP(0): s=202.158.196.134 (Tunnel650580) d=233.29.151.44 id=0, ttl=113, prot=17, len=40(40), mroute olist null

Nov  6 11:15:13: IP(0): s=202.158.196.68 (Tunnel650580) d=233.29.151.44 id=0, ttl=111, prot=17, len=170(170), mroute olist null

Nov  6 11:15:13: IP(0): s=195.220.53.138 (Tunnel650580) d=233.29.151.44 id=1682, ttl=116, prot=17, len=81(81), mroute olist null

Nov  6 11:15:13: IP(0): s=131.172.135.210 (Tunnel650580) d=233.29.151.44 id=2118, ttl=109, prot=17, len=550(550), mroute olist null

Nov  6 11:15:13: IP(0): s=134.129.95.122 (Tunnel650580) d=233.29.151.44 id=43998, ttl=112, prot=17, len=40(40), mroute olist null

Nov  6 11:15:13: IP(0): s=129.78.221.17 (Tunnel650580) d=233.29.151.44 id=0, ttl=106, prot=17, len=97(97), mroute olist null

Nov  6 11:15:13: IP(0): s=134.129.95.122 (Tunnel650580) d=233.29.151.44 id=110, ttl=112, prot=17, len=40(40), mroute olist null

Nov  6 11:15:13: IP(0): s=134.129.95.122 (Tunnel650580) d=233.29.151.44 id=19326, ttl=112, prot=17, len=40(40), mroute olist null

Nov  6 11:15:13: IP(0): s=134.129.95.122 (Tunnel650580) d=233.29.151.44 id=16680, ttl=112, prot=17, len=62(62), mroute olist null

 

---------------------------------------------------------------END DEBUG-------------------------------------------------------------------

 

However, if I mtrace the destination I get stuck on our  paris1 router.

 

------------------------------------------------------MTRACE-------------------------------------

LAB-NOC-7200-GATEWAY#mtrace 202.6.112.4 193.51.178.198 233.29.151.44

Type escape sequence to abort.

Mtrace from 202.6.112.4 to 193.51.178.198 via group 233.29.151.44

From source (?) to destination (?)

Querying full reverse path... * switching to hop-by-hop:

0  193.51.178.198

-1  193.51.178.198 PIM/MBGP  [202.6.112.0/24]

-2  tu650580-paris1-rtr-032.noc.renater.fr (193.51.177.6) PIM  [202.6.112.0/24]

-3  * * * gi8-3-paris1-rtr-021.noc.renater.fr (193.51.177.154)

 

----------------------------------------------

On paris1 the routing is fine, or so it would seem

 

-----------------------------------------------Paris1 routing-----------------------------------------------------------------------

 

paris1-rtr-021#sh ip mroute 233.29.151.44

IP Multicast Routing Table

Flags: D - Dense, S - Sparse, B - Bidir Group, s - SSM Group, C - Connected,

       L - Local, P - Pruned, R - RP-bit set, F - Register flag,

       T - SPT-bit set, J - Join SPT, M - MSDP created entry, E - Extranet,

       X - Proxy Join Timer Running, A - Candidate for MSDP Advertisement,

       U - URD, I - Received Source Specific Host Report,

       Z - Multicast Tunnel, z - MDT-data group sender,

       Y - Joined MDT-data group, y - Sending to MDT-data group,

       V - RD & Vector, v - Vector

Outgoing interface flags: H - Hardware switched, A - Assert winner

Timers: Uptime/Expires

Interface state: Interface, Next-Hop or VCD, State/Mode

 

(*, 233.29.151.44), 00:08:37/00:02:43, RP 193.51.178.254, flags: S

  Incoming interface: GigabitEthernet8/8, RPF nbr 193.51.177.159

  Outgoing interface list:

    GigabitEthernet8/3, Forward/Sparse, 00:08:37/00:02:43

(202.158.206.38, 233.29.151.44), 00:00:44/00:02:45, flags:

  Incoming interface: TenGigabitEthernet2/1, RPF nbr 193.51.177.26, Mbgp

  Outgoing interface list:

    GigabitEthernet8/8, Forward/Sparse, 00:00:44/00:02:45

    GigabitEthernet8/3, Forward/Sparse, 00:00:44/00:02:15

(137.219.15.25, 233.29.151.44), 00:00:45/00:02:44, flags:

  Incoming interface: TenGigabitEthernet2/1, RPF nbr 193.51.177.26, Mbgp

  Outgoing interface list:

    GigabitEthernet8/8, Forward/Sparse, 00:00:45/00:02:44

    GigabitEthernet8/3, Forward/Sparse, 00:00:45/00:02:14

(203.27.221.247, 233.29.151.44), 00:08:37/00:02:40, flags: T

  Incoming interface: TenGigabitEthernet2/1, RPF nbr 193.51.177.26, Mbgp

  Outgoing interface list:

    GigabitEthernet8/3, Forward/Sparse, 00:08:37/00:02:43

(202.158.196.134, 233.29.151.44), 00:08:37/00:02:53, flags: T

  Incoming interface: TenGigabitEthernet2/1, RPF nbr 193.51.177.26, Mbgp

  Outgoing interface list:

    GigabitEthernet8/3, Forward/Sparse, 00:08:37/00:02:43

(202.158.196.68, 233.29.151.44), 00:08:37/00:02:50, flags: T

  Incoming interface: TenGigabitEthernet2/1, RPF nbr 193.51.177.26, Mbgp

  Outgoing interface list:

    GigabitEthernet8/3, Forward/Sparse, 00:08:37/00:02:43

 

(202.158.196.4, 233.29.151.44), 00:08:37/00:02:43, flags: T

  Incoming interface: TenGigabitEthernet2/1, RPF nbr 193.51.177.26, Mbgp

  Outgoing interface list:

    GigabitEthernet8/3, Forward/Sparse, 00:08:37/00:02:43

(202.158.195.196, 233.29.151.44), 00:08:37/00:02:48, flags: T

  Incoming interface: TenGigabitEthernet2/1, RPF nbr 193.51.177.26, Mbgp

  Outgoing interface list:

    GigabitEthernet8/3, Forward/Sparse, 00:08:37/00:02:43

 

(202.158.195.132, 233.29.151.44), 00:08:37/00:02:42, flags: T

  Incoming interface: TenGigabitEthernet2/1, RPF nbr 193.51.177.26, Mbgp

  Outgoing interface list:

    GigabitEthernet8/3, Forward/Sparse, 00:08:37/00:02:43

(202.158.195.68, 233.29.151.44), 00:08:39/00:02:37, flags: T

  Incoming interface: TenGigabitEthernet2/1, RPF nbr 193.51.177.26, Mbgp

  Outgoing interface list:

    GigabitEthernet8/3, Forward/Sparse, 00:08:39/00:02:42

(202.158.195.4, 233.29.151.44), 00:08:39/00:02:41, flags: T

  Incoming interface: TenGigabitEthernet2/1, RPF nbr 193.51.177.26, Mbgp

  Outgoing interface list:

    GigabitEthernet8/3, Forward/Sparse, 00:08:39/00:02:42

(202.6.112.4, 233.29.151.44), 00:08:39/00:02:50, flags: T

  Incoming interface: TenGigabitEthernet2/1, RPF nbr 193.51.177.26, Mbgp

  Outgoing interface list:

    GigabitEthernet8/3, Forward/Sparse, 00:08:39/00:02:42

(195.220.53.138, 233.29.151.44), 00:08:39/00:02:38, flags: T

  Incoming interface: TenGigabitEthernet2/1, RPF nbr 193.51.177.26, Mbgp

  Outgoing interface list:

    GigabitEthernet8/3, Forward/Sparse, 00:08:39/00:02:42

(138.77.155.72, 233.29.151.44), 00:08:39/00:02:42, flags: T

  Incoming interface: TenGigabitEthernet2/1, RPF nbr 193.51.177.26, Mbgp

  Outgoing interface list:

    GigabitEthernet8/3, Forward/Sparse, 00:08:39/00:02:42

 

(138.77.147.71, 233.29.151.44), 00:08:39/00:02:41, flags: T

  Incoming interface: TenGigabitEthernet2/1, RPF nbr 193.51.177.26, Mbgp

  Outgoing interface list:

    GigabitEthernet8/3, Forward/Sparse, 00:08:39/00:02:42

(138.77.139.71, 233.29.151.44), 00:08:39/00:02:37, flags: T

  Incoming interface: TenGigabitEthernet2/1, RPF nbr 193.51.177.26, Mbgp

  Outgoing interface list:

    GigabitEthernet8/3, Forward/Sparse, 00:08:39/00:02:42

(134.129.95.122, 233.29.151.44), 00:08:39/00:02:42, flags: T

  Incoming interface: TenGigabitEthernet2/1, RPF nbr 193.51.177.26, Mbgp

  Outgoing interface list:

    GigabitEthernet8/3, Forward/Sparse, 00:08:39/00:02:42

 

(131.172.135.210, 233.29.151.44), 00:08:39/00:02:38, flags: T

  Incoming interface: TenGigabitEthernet2/1, RPF nbr 193.51.177.26, Mbgp

  Outgoing interface list:

    GigabitEthernet8/3, Forward/Sparse, 00:08:39/00:02:42

(131.172.8.214, 233.29.151.44), 00:08:39/00:02:41, flags: T

  Incoming interface: TenGigabitEthernet2/1, RPF nbr 193.51.177.26, Mbgp

  Outgoing interface list:

    GigabitEthernet8/3, Forward/Sparse, 00:08:39/00:02:42

(129.78.221.17, 233.29.151.44), 00:08:39/00:02:51, flags: T

  Incoming interface: TenGigabitEthernet2/1, RPF nbr 193.51.177.26, Mbgp

  Outgoing interface list:

    GigabitEthernet8/3, Forward/Sparse, 00:08:39/00:02:42

 

MTRACES on all of our PE seem to providing similar results and messages --

 

Timed out receiving responses

Perhaps no local router has a route for source,

the receiver is not a member of the multicast

group or the multicast ttl is too low.

 

In short, it's not possible to mtrace on any of our PE.   Anyone have an idea what could be the problem and what would be a possible fix for this problem.  

Everyone's tags (1)
134
Views
0
Helpful
0
Replies
CreatePlease to create content