i have enclosed the topology diagram.
- In AS100, R1, R2, R3 and R4 have PIM sparse enabled with R4 as BSR RP. Within the AS100, multicast is working fine. R3 and R4 have one connection each to AS200. PIM is enabled for inter-domain multicast on R3-R5 link, not enabled on R4-R5 link. R3 has bsr border configured on the R3-R5 link.
- In AS200, R5 and R6 have PIM sparse enabled. R5 is the BSR RP. R5 is BSR border for AS200.
- Loopbacks for all routers are reachable from all routers. BGP is used to advertise the loopbacks.
- active path for unicast traffic from AS100 to AS200 is via R4-R5 link.
- active path for unicast traffic from AS200 to AS100 is via R5-R3 link.
(so we have asymmetric flow)
- mutlicast traffic should via R3-R5 link as primary path. PIM is only enabled on the R3-R5 link. (disabled on R4-R5 link)
my doubt is on MSDP config. Since R4 is the RP, eMSDP should be between R4 and R5? Should this use physical or loopback IP? how should the RPF be configured? BGP IPv4 multicast should be between R3 and R5? Do i need to setup an addition iMSDP between R3 and R4?
can sombody recommend the correct config guidelines?
All the sample config in every document uses the edge device as RP with PIM enabled between the RPs. I cant find non-RP to RP MSDP example.
You need Multicast BGP on all your routers as your unicast and multicast topology are not congruant. Then configuring an MSDP session between both RP's should work as the MSDP peer will be seen as an eBGP peer so the first AS of the best BGP path to join the RP originating the SA must match the AS the eBGP peer sending the SA (which is the same router in your case). Don't forget to build your MSDP peering with the same addresses used for BGP peering.
For more information about the MSDP SA RPF check rules, please refer to chapter 10 of the RFC-3618. I agree this is usually the most complex part to understand about MSDP.