WAN BGP Dual router/ Dual ISP routing -

[Steve Coady]

Hello

I am migrating from a single Internet provider to (2) new Internet providers

Current provider is receiving (6) class "C" subnets (170.x.1.0/24 - 170.x.6.0/24) from me via static. They are then advertising my class "B" prefix network as a 170.x.0.0/16 via ibgp to their cloud and the rest of the world. I have verified this via http://bgpinspect.merit.edu

My (2) new ISP's

I am advertising my subnet (170.x.254.x/24) via BGP to both ISP's (ISP_1 and ISP_2)

No matter what I do, ISP_1 is always seen asthe path back to my company

I administratively shut down my WAN interface to ISP_1 and it still tried to come back across the ISP_1 path.

Finally the ISP_1 path disappeared and now instead of coming back across my ISP_2 path

traffic for the new subnet, (170.x.254.x/24) is being seen by the World as coming from my current ISP (170.x.0.0/16)

WHAT AM I MISSING???

Attached is a diagram of new WAN

Also included are the BGP statements for both new routers.

[Steve Coady]

here is topology

[Joseph Nelson]

sMc,

Can we take this one offline? What's your timezone? Email me at joseph.nelson08<at>gmail<dot>com

There's some additional questions I want to ask you and the feedback loop on the forum is too long.

Joe

[e.ciollaro]

Not very clear, it seems me that now your network is adverised by 3 ISP: the old one and the two new ISPs. In any case let me do some question:

I am migrating from a single Internet provider to (2) new Internet providers

Current provider is receiving (6) class "C" subnets (170.x.1.0/24 - 170.x.6.0/24) from me via static. They are then advertising my class "B" prefix network as a 170.x.0.0/16 via ibgp to their cloud and the rest of the world. I have verified this via http://bgpinspect.merit.edu

My (2) new ISP's

I am advertising my subnet (170.x.254.x/24) via BGP to both ISP's (ISP_1 and ISP_2)

No matter what I do, ISP_1 is always seen asthe path back to my company  what do you mean, what you did ?

I administratively shut down my WAN interface to ISP_1 and it still tried to come back across the ISP_1 path.

Finally the ISP_1 path disappeared and now instead of coming back across my ISP_2 path how much does it take ? Which value did you negotiated for this BGP session ?

traffic for the new subnet, (170.x.254.x/24) is being seen by the World as coming from my current ISP (170.x.0.0/16) Seems the this ISP send a better advertisement then the second ISP; if so you have to agree a different metric for you networks usually as-prepend is used ore configre an eBGP session and change the NLRI attribute advertised to this ISP. 

In any case seems me that there are some problem in your BGP config: why did you configure 

neighbor 12.x.x.9 default-originate

this way you are advertising a default route to the ISP, isn't  it ?

Moreover I would use iBGP instead of tracking a remote subnet. Finally  if you want one link to be primary and one backup, you have to differentiate the NLRI attribute advertised to the ISPs so that traffic from all over the world will prefer the primary instead of the backup.

Let me know, bye

enrico

[Joseph Nelson · Accepted solution]

Enrico,

Regarding your comment:

"In any case seems me that there are some problem in your BGP config: why did you configure 

neighbor 12.x.x.9 default-originate

this way you are advertising a default route to the ISP, isn't  it ?"

I'm not to worried about the "default-originate" the OP has toward the ISP. Indeed it should not be there, however no ISP in the world is going to accept a default from a stub AS-- and if they did, the OPs routers would likely shutdown.

Regarding your comment:

traffic for the new subnet, (170.x.254.x/24) is being seen by the World as coming from my current ISP (170.x.0.0/16) Seems the this ISP send a better advertisement then the second ISP; if so you have to agree a different metric for you networks usually as-prepend is used ore configre an eBGP session and change the NLRI attribute advertised to this ISP. 

The OP is advertising his aggregate to his current IP and more specifics to his new IPs during his migration. This way, the site stays up while he migrates subnet by subnet to the new ISPs ( that's how I read it). At any rate, this situation can occur if ISP_2 doesn't have the more-specific he's trying to announce.

[e.ciollaro]

Hi Joseph,

you're right about the default route: no ISP in the world will accept it but I could be interesting to know what was the goal of sMc was trying to achieve.

About more-specific advertisement you're right, I did notice (may be it's time for me to stop working and going home ;-)

Bye,

Enrico

[Steve Coady]

Joseph

Thank you for the response

"The OP is advertising his aggregate to his current IP and more specifics to his new IPs during his migration. This way, the site stays up while he migrates subnet by subnet to the new ISPs ( that's how I read it)." YES

The goal:

I have (2) physical sites. Each with it's own ISP connection.

Traffic originated at Site_1 will prefer ISP_1.

             should I use the Local pref attribute to assure this?

Traffic originated at Site_2 will prefer ISP_2

             should I use the Weight attribute to assure this?

ISP_1 will be failover for ISP_2 and vice verse.

The failover is accomplished by using the dual HSRP config shown in the diagram.

I had to use a /24 because one of the ISP' does not allow BGP advertisements less than /24.

I am advertising the same 170.x.254.0/24 in each WAN router to  each ISP

    WAN_1 / ISP_1 has 170.x.254.3, 170.0.254.5, 170.x.254.7 etc...

    WAN_2 / ISP_2 has 170.x.254.4, 170.x.254.6, 170.0.254.8 etc...

Instead of advertisng the subnet/24, should I advertise the specific interface addresses/24?

Will this cause BGP problems in future as more of the 170.x.254.0 addresses are used?

This really has me stumped. I am advertsing the same subnet 170.x.254.0/24 to (2) ISP's

How can I assure that local ip addresses are advertised to the local ISP?

[Joseph Nelson]

sMc,

I'd like to help you solve your original problem. Please see my original response and provide that output ( if you have already, thanks). 

Regarding your other problems/questions, lets take them part by part:

Goal: Traffic originated at Site_1 will prefer ISP_1, Traffic originated at Site_2 will prefer ISP_2

Suppose the following:

• RS1 connected to ISP_1
• RS2 connected to ISP_2
• RS1 and RS2 have iBGP connection

Then

• On RS1, incoming route-map for ISP_1 peer should increase local preference
• On RS2, incoming route-map for ISP_2 peer should increase local preference
• On RS1, outgoing route-map for RS2 should reset local preference (to 100)
• On RS2, outgoing route-map for RS2 should reset local preference ( to 100)

This configuration achieves your stated goals. If egress bound traffic hits RS1, it will use local preference and pick ISP_1 as egress, same for RS2. RS1 will also know a path through RS2, however, they will be receive-only because it already has chosen a path through ISP_2. 

Goal: ISP_1 will be failover for ISP_2 and vice verse.

No. You should not have a problem. Its okay to advertise your prefixes from more than one site, however, you have to be careful about which way ingress traffic flows. A lot of people like to play around with as_path...its really only a temporary solution on the Internet, because, realistically no one cares about your as-path length. If I'm an ISP, I'm not routing based on _your_ preference, I'm routing based on my preference/business drivers. 

That said, if you want to have an Active/Active ingress traffic flow, you'll keep playing around with as-path ( your your ISP's traffic engineering mechanisms). If you want an Active/Passive ingress traffic flow, look into BGP Conditional Annoucement

Question: Instead of advertisng the subnet/24, should I advertise the specific interface addresses/24?

Not really sure what you mean. You want to advertise the subnet/24, in your case 170.x.254.0/24.

Question: Will this cause BGP problems in future as more of the 170.x.254.0 addresses are used?

Not really sure which problem you are referring to here.

[Steve Coady]

Enrico

Thank you for the response.

             I removed the default-originate statement.

             Not sure what you mean here "

                  how much does it take ? Which value did you negotiated for this BGP session ?

The goal:

I have (2) physical sites. Each with it's own ISP connection.

Traffic originated at Site_1 will prefer ISP_1.

             should I use the Local pref attribute to assure this?

Traffic originated at Site_2 will prefer ISP_2

             should I use the Weight attribute to assure this?

ISP_1 will be failover for ISP_2 and vice verse.

The failover is accomplished by using the dual HSRP config shown in the diagram.

I had to use a /24 because one of the ISP' does not allow BGP advertisements less than /24.

I am advertising the same 170.x.254.0/24 in each WAN router to  each ISP

    WAN_1 / ISP_1 has 170.x.254.3, 170.0.254.5, 170.x.254.7 etc...

    WAN_2 / ISP_2 has 170.x.254.4, 170.x.254.6, 170.0.254.8 etc...

[Joseph Nelson]

Hi sMc,

Not sure if you saw my original reply, can you advise?

Joe

[Steve Coady]

Joe

You mean the AS Prepend?

[An Nguyen]

I have similar configuration, with two different ISPs. Each connecting to a single router. These two routers have EIGRP full mesh and redistribute.

BGP is configured on both routers, to their respective ISP. I have configured higher local preference on inbound to 200. Default is 100 is left alone.

Our netblock is a /23 which is advertised by both ISPs.

However, it seems the path to ISP2 is always preferred since I have used BGP Looking Glass and Traceroute, all of the results showing going through secondary ISP.

What am I missing?

Thanks

[Joseph Nelson]

anh2lua@outlook.com,

Why don't you message me here or start another thread--we'll go through it together. I don't want to mix threads because I'll get confused.