This discussion is locked

ASK THE EXPERT - Border Gateway Protocol Multi-homing

Unanswered Question
Feb 17th, 2011

With Manigandan Ganesan

Read the bio

Welcome to the Cisco Networking Professionals Ask the Expert conversation. This is an opportunity to get an update on design and troubleshooting BGP with Cisco expert Manigandan Ganesan. Mani is an engineer working with the Routing Protocols team in the Cisco Technical Assistance Center in Bangalore. There he configures and troubleshoots various routing protocols like Enhanced Interior Gateway Routing Protocol, Open Shortest Path First protocol, Border Gateway Protocol, and Protocol Independent Multicast. He also focuses on filing technical and documentation bugs in these areas, and delivers training sessions on these technologies to other teams in Cisco. Mani holds a bachelor’s degree in electrical and electronics engineering from Anna University, Chennai. He also holds CCIE certification #27200 in Routing and Switching.

Remember to use the rating system to let Mani know if you have received an adequate response.

You can review the Live Webcast Video where Mani gave a presentation on this topic. You can also read the questions he answered during the live event in this FAQ Document.

Mani might not be able to answer each question due to the volume expected during this event. Remember that you can continue the conversation on the Security discussion forums shortly after the event. This event lasts through February 25, 2011. Visit this forum often to view responses to your questions and the questions of other community members.

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 4.8 (5 ratings)
ahmedchohan Fri, 02/18/2011 - 00:08

Hi Mani

Saw your "ATE" on cisco virtual and was informative. Thanks !

1.     Can u please elaborate on the role of iBGP on the CE routers. Would the CE routers need the complete routing table for it to make use of the AS path for the closest destination and other tweaking ?

2.      In case of both the CE routers getting the default route, how can we load balance ?

3.     What would be role of hsrp in both of the above scenarios.




Ibrahim Jamil Fri, 02/18/2011 - 03:33

Hi Manigandan

Thanks and it was an  informative session (I Prefer FfR Included in the Presantation) for me and  I had been waiting for long for this Presentation,okay now what the configuration Looks Like if we have dual ASA involved in the scenario Like ahmedchohan's Post ,and these dual ASA must be active/active coz we have own ASN/Address Block from RiR,so what is the recommend configuration on the ASA's ,Pls Bear in Mind that we  have servers tiers presented to Internet required continuous Internet connectivity

our configuration on the Border Router similar to the attached file,Pls Have a look to it while the overall diagram its the Exactly the same like ahmedchohan's diagram

Thank You

Mani Ganesan Fri, 02/18/2011 - 09:59

Hi Ibrahim,

Glad you liked the session.

As far as ASA configuration goes, both of them need to have a default route pointing to thier respective upstream devices either static or IGP ( if any ).

The important point to take care when it comes to ASA is asymmetric routing. If we send traffic to one upstream link, but receive the return traffic on the other one  ( or ) send end traffic via one ASA, but receive the return traffic on the other ASA would be a problem. FW would drop the return traffic. To avoid this, we have to make sure that the IGP is set up in such a way that for traffic from a specific block of the network would leave and reach ASA on the same link (or) go through and come back via the same ASA.

If there is any thing specific that you want to get clarified on the ASA part with BGP, Please let me know.

Best Regards,


Ibrahim Jamil Sat, 02/19/2011 - 04:43

HI Mani

i think the asr-group does solving the Asymetric routing,I have been diving in,but i didnt find this type of configuration with bgp multihoming to 2 different ISP with the present of two ASAs running as Active/Active,do u have such kind of this configuration for whole design configuation with ASAs active/active

Mani Ganesan Sun, 02/20/2011 - 03:12

Hi Ibrahim,

There is nothing specific needed on the ASA group running in Active / Active state for BGP multihoming, as the ASA group we create is meant for the security appliance failover. As we always do, we need to make sure both the ASAs are configured to allow traffic betwen the core and the border routers and stuffs like permitting port 179, if we have IBGP sessions going across FWs.

The below document talks about the overall design when we have a multihoming set-up and IBGP sessions running across the FWs

For active / active part on ASAs, configuration steps are explained below.  It is all about creating a logical group and dividing the security contexts on the device into failover groups,

Best Regards,


Ibrahim Jamil Fri, 02/18/2011 - 03:35

Hi ahmedchohan

I have exactly the same as ur Topology,would Please Share the configuration with me,however Put Fake IP address Instead

Mani Ganesan Fri, 02/18/2011 - 09:18

Hi Ahmed,

Glad that you liked the session.

Please see my answers inline :

1.   a)  Can u please elaborate on the role of iBGP on the CE routers.

There are multiple useful roles that IBGP between the border routers play. I will mention the important ones.

      If BGP with the primary ISP fails, but your core  devices keep sending traffic to router 1 ( because first router is still  running and your HSRP / IGP stay intact ), if we have an IBGP connection between two routers, R1 can send traffic to R2 over IBGP and  then to ISP 2.

       Another reason is setting of route preference. As we know, Local Preference is an attribute in BGP which is used to influence the way we route outbound packets. When we set Local preference on one router, it is shared with all the routers running BGP with in the SAME AS, which would mean all IBGP peers. So coming to the point, when we run IBGP, LP can be shared and both routers would be aware of the prefrence we set.

         For Example, If we are getting full routing table from both ISPs and on first router, you match half the routes and set a higher local preference. In this case, if you run IBGP with the second router, this LP value is propagated and the second router would be aware of the fact that, for the first half of the routes, router one is the preferred one.

     b) Would the CE routers need the complete routing table for it to make use  of the AS path for the closest destination and other tweaking ?

Go for complete routing table if you are really specific about the path it takes for every destination ( which is not the case most of the times ) and if your Routers have the memory / CPU for 350K routes. From my personal experience, a default and routes from the Provider's directly connected AS's are good enough to do good optimal routing. For the rest of the routes in internet, taking any of the ISPs would be fine.

2.      In case of both the CE routers getting the default route, how can we load balance ?

In this case, do not set any LP. So the question boils down to your IGP / GLBP / MHSRP load balancing. If RTR 1 gets the traffic from core, it would take it own default and the same case for RTR 2. If we have an IGP running, we can make sure that we have equal cost path defaukt routes to the both the routers from the core, so that we send out packets to internet via both the ISPs.

3.     What would be role of hsrp in both of the above scenarios.

HSRP would be important, if we are running BGP in fail over mode and we do not have IGP running between our core and the border routers. For instance, if we have a Flat L2 network, we can configure HSRP's vitual IP as the gateway and track the WAN interface. If the WAN interface ( and hence BGP ) goes down, HSRP will fail over to R2 and it will take over in BGP as well. We can also use HSRP's VIP as the next hop, if we have a static default route configured on the core.

I would be glad to answer if there are any further queries.



ahmedchohan Tue, 02/22/2011 - 04:41

Hi Mani

Thanks for the detailed reply

Just a few more doubts , if you please :

"      If  BGP with the primary ISP fails, but your core  devices keep sending  traffic to router 1 ( because first router is still  running and your  HSRP / IGP  stay intact ), if we have an IBGP connection between two  routers, R1 can send traffic to R2 over IBGP and  then to ISP 2. "

In the above scenario, Would R1's "IP redirect" come into play , and then alltraffice from HSRP would go to R2 ? or all traffic would always go to R1 and then go to R2 ( in case of R1 bgp failure (until unless we use advaced tracking))



Mani Ganesan Tue, 02/22/2011 - 09:42

Hi Ahmed,

That s a very good question.

Though redirects are being supported in HSRP now a days, Redirects to passive HSRP routers are not  permitted ( Redirects can be done in HSRP only if the next hop that is suggested in the ICMP redirect packet is that of another active router or a router not running HSRP ) . Redundancy may be lost if hosts learn the real IP addresses  of HSRP routers.

For more details on this --

So, Packets would traverse through R1( HSRP Active )  to R2 and then the ISP.

We can always track a route learned from BGP to track the availability of the ISP and change the HSRP state accordingly.

Please let me know if there are any more questions on this.



travis-dennis_2 Fri, 02/18/2011 - 06:04

Hello Manigandan,

In a configuration with a single router with 2 different ISPs on it but each ISP will not advertise routes for the other (Low end broadband connections)  What are my options for inbound traffic failover?  There are a few devices out there that claim to be able to still route the inbound traffic over ISP B if ISP A fails.  If it takes 2 routers to make something work that is something that can be done.

Thakns in advance!

Mani Ganesan Fri, 02/18/2011 - 09:38

Hello Travis,

When you say single router with 2 different ISPs, I assume you are ruuning BGP with both of them and not routing through static default.

To do inbound failoverin this scenario, you can advertise address blcocks assigned by both the ISPs to both of your BGP peers. The trick here is when you advertise the second's ISPs address block to the 1 st BGP peer, do it with AS-path prepending. Similary advertsie the first ISP's address block to your 2nd BGP peer, but with  AS-path prepending.

When both ISPs are up and running, traffic coming back to 1 st ISP's address block would come via ISP 1 and the same case for traffic destined to 2 nd block. If one of them fails, say ISP1,  we would be still be receiving traffic for block 1 via ISP 2, since we are advertising both the blocks to both of the ISPs.

Hope this helps.



unclerico Tue, 02/22/2011 - 19:46

Hi Mani,

Prefix length aside, is there any way that I can do BGP multi-homing with this topology??

My concern is forming the iBGP peering relationship between the CE router in HQ and the CE router in Colo with NAT involved. I'm using RFC1918 addresses inside the firewalls and globally routable addresses on the outside of the firewalls. Thank you very much for your assistance!!!

Mani Ganesan Tue, 02/22/2011 - 20:05


We can acheive perfect load balance / failover in this multihoming set-up.

If the CE routers are running different AS numbers (as they are peering with two different ISPs )  IBGP between the CE routers in not an option, we can still configure EBGP between the two CEs. In this scenario, each of the CE would have two EBGP peers ( 1 for the ISP and one for the other CE ),

- assign a higher local preference for the ISP peer and

-  Configure AS path prepending on the other CE peer for the networks you advertise

By doing this we can send and receive traffic through the local ISP at each site, as long as they are available. If one of them fails, CE would start routing traffic through the other CE. So fail over would be acheived seamlessly.

If both CE routers are running the same AS number, IBGP can be configured and the same logic explained above can be done for this as well.

If it is going to be IBGP, we just need ip conenctivity to the peer address and port 179 needs to be open through the FW.

If it is EBGP, we need neighbor ebgp-multihop  < TTL value > along with the above conditions.

OSPF as an IBGP is fine , we can inject a default route from each of the CEs in OSPF using ' default information originate ', which would inject only if it has the BGP default route locally available. so the inside devices would automatically start to send traffic to the other CE, if BGP fails on one of the CE.

When both routers inject a default route, we need to make sure we give preference to the local CE, not the other one.

Hope this helps.

Best regards,


unclerico Wed, 02/23/2011 - 07:47

Hi Mani,

Thanks so much for responding. I have a followup question in regards to this:

"If it is going to be IBGP, we just need ip conenctivity to the peer address and port 179 needs to be open through the FW."

What would this look like with NAT involved on the fiewalls?? If there were globally routable IP's on both sides of the firewall I could see how to do it, but with RFC1918 IP's on the inside I don't know how I would get this to work. Maybe I'm overanalyzing it. Thank you again.

Mani Ganesan Wed, 02/23/2011 - 19:37


Are you able to ping from the inside interface of the colo router to the main router through the FW ?

It does not matter if it is a private range or public range to run IBGP, as long as we can reach the peer's ip address.

If ping fails, then we need to get the basic conenctivity up between the inside interfaces of the router through the FW, where NAT and stuffs would come into the picture.



huangedmc Thu, 02/24/2011 - 05:48

hi Mani,

My question isn't related to BGP multi-homing, but control plane policing in regards to BGP traffic.

In the campus QoS SRND 4.0, an example was given to police BGP traffic to 4M.

Is 4M sufficient for most cases?

Even if that's the case, we'd like to analyze & baseline our environment, to make sure CoPP doesn't impact BGP prefix exchange between our switches & routers.

How do we go about that?

Is there a show command that can tell us the RATE of the BGP traffic?

"show ip bgp neighbor" shows you how much memory the prefixes consume, but it doesn't tell you how fast the router is receiving the route updates from its BGP neighbors.

Ditto for IGP traffic.



Mani Ganesan Fri, 02/25/2011 - 08:16

Hi Kevin,

4M would generally enough, however there are factors such as the platform ( hardware / software ), number of BGP peers, number of routes learned and how stable they are.

Regarding your question on, how to know how fast the router is receiving the route updates from its BGP neighbors, 'show ip bgp summary' shows the INQ for every peer, if it is non-zero, that would tell us if we receive BGP updates at a rate faster than what the CPU can handle. Based on that, we can adjust the threshlold, if it is not enough.

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
*    4 50000       2       2                 2    0    0        00:00:37        0

Also you can check the show policy map < name > command to see the exact number of BGP packets received in total on a real time basis. For example,

RTR1#show policy-map control-plane
Control Plane
Service-policy input: copp-policy
Class-map: coppclass-bgp (match-all)
1443 packets, 113844 bytes
5 minute offered rate 0 bps
Match: access-group name coppacl-bgp

On a side note, I have seen customers giving unrestricted access to BGP and IGP, so that the keepalives and updates never get dropped even if they cross the limit, and hence peers are never flapped because of the policies.

Best regards,



This Discussion

Related Content