Protocol Independent Multicast (PIM) mode is the most commonly used protocol for IP multicasting within an Autonomous System (AS). PIM can use the route learned from any unicast routing protocol to perform the Reverse Path Forwarding (RPF) check and forward the multicast packets that a router receives from a source. When multicasting is implemented across different ASs (also known as Inter-AS multicasting), PIM can be used inside the individual domains for building data distribution trees. But it has to rely on the unicast route information learned through Border Gateway Protocol (BGP) to perform the RPF check.
There may be cases where the unicast traffic has to flow through a particular path between two ASs and a different path has to be used for multicast traffic between the same two ASs. In such a case, the RPF check fails if a multicast source is learned through the interface meant for unicast traffic. This is a common case in Network Access Points (NAPs), where multiple service providers peer with each other at multiple points. To overcome this problem, Multicast BGP (MBGP) can be used. MBGP is an extension of BGP that can carry routing information for multiple network layer protocols. MBGP can indicate whether an advertised prefix is to be used for unicast routing, multicast RPF checks or both.
For BGP to support IPv6 or multicast, BGP is extended. These extensions are called Multiprotocol BGP (MP-BGP) extensions. For multicast, you usually refer to these extensions as Multiprotocol BGP extensions for multicast. Multicast BGP is multicast BGP extensions for IP multicast.
The configuration of MBGP is identical to that of ordinary BGP configuration except for a few additional steps. MBGP involves issuing the address-family command in BGP configuration mode identify the type of route advertised. For example, router A has a LAN segment 172.16.1.0/24 attached to it, with multicast sources belonging to this segment. Router A is also connected to router B across two WAN links belonging to 192.168.10.0/24 and 192.168.20.0/24 networks, with receivers attached to a LAN segment on router B. Use the first WAN link for unicast traffic alone and the second WAN link for only multicast traffic.You need to configure multicast routing on both routers and enable pim on the interface you want to send multicast traffic ,2nd link in our case.you would also need to enable pim on LAN segment of both routers (multicast server and host) that will also enable IGMP enable on it.
This is the MBGP configuration on router A:
router bgp 65500
no bgp default ipv4-unicast
!-- This command disables automatic advertising of prefixes to all
neighbors for unicast traffic.
network 172.16.1.0 mask 255.255.255.0
!--- This line advertises the attached LAN segment for
neighbor 192.168.10.2 remote-as 65535
neighbor 192.168.10.2 activate
!--- These lines configure the neighbor on the first WAN link to router B
and enables exchange of prefixes for unicast traffic with this specific neighbor.
neighbor 192.168.20.2 remote-as 65535
!--- This line configures the neighbor on the second WAN link to router B.
address-family ipv4 multicast
!--- This command configures the MBGP session to exchange prefixes for
neighbor 192.168.20.2 activate
!--- This line activates the previously defined neighbor on the
second WAN link to router B. This is done to exchange prefixes for multicast traffic.
network 172.16.1.0 mask 255.255.255.0
!--- This line advertises this prefix for multicast
traffic apart from advertising the prefix previously for unicast traffic.
!--- This command exits out of the address-family configuration mode.
Due to this configuration, unicast traffic to the 172.16.1.0/24 subnet flows on the first WAN link between the routers, but the second WAN link is used for the RPF check when a source belonging to this subnet sends multicast traffic reaching router B.
For more information on MBGP, refer to these documents: