Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements

Welcome to Cisco Support Community. We would love to have your feedback.

For an introduction to the new site, click here. And see here for current known issues.

New Member

WAN BGP Dual router/ Dual ISP routing -

Hello

 

I am migrating from a single Internet provider to (2) new Internet providers

Current provider is receiving (6) class "C" subnets (170.x.1.0/24 - 170.x.6.0/24) from me via static. They are then advertising my class "B" prefix network as a 170.x.0.0/16 via ibgp to their cloud and the rest of the world. I have verified this via http://bgpinspect.merit.edu

 

My (2) new ISP's

 

I am advertising my subnet (170.x.254.x/24) via BGP to both ISP's (ISP_1 and ISP_2)

No matter what I do, ISP_1 is always seen asthe path back to my company

I administratively shut down my WAN interface to ISP_1 and it still tried to come back across the ISP_1 path.

 

Finally the ISP_1 path disappeared and now instead of coming back across my ISP_2 path

traffic for the new subnet, (170.x.254.x/24) is being seen by the World as coming from my current ISP (170.x.0.0/16)

 

WHAT AM I MISSING???

 

Attached is a diagram of new WAN

Also included are the BGP statements for both new routers.

 

 

 

 

 

sMc
2 ACCEPTED SOLUTIONS

Accepted Solutions

 It sounds like ISP_2 does

 

It sounds like ISP_2 does not have your 170.x.254.x/24 prefix and/or is not propagating it. ISP_2 may need to update is Inbound Route Filter for your peer.

Have you verified the following:

  • You are actually advertising the 170.x.254.x/24 prefix to ISP_2? ( show ip bgp nei <peer_ip> advertised-routes)
  • ISP_2 is actually accepting the 170.x.254.x/24 prefix ( check looking glass )?

 

Can you provide the output for the two question above?

Enrico, Regarding your

Enrico,

 

Regarding your comment:

"In any case seems me that there are some problem in your BGP config: why did you configure 

neighbor 12.x.x.9 default-originate

this way you are advertising a default route to the ISP, isn't  it ?"

I'm not to worried about the "default-originate" the OP has toward the ISP. Indeed it should not be there, however no ISP in the world is going to accept a default from a stub AS-- and if they did, the OPs routers would likely shutdown.

 

Regarding your comment:

traffic for the new subnet, (170.x.254.x/24) is being seen by the World as coming from my current ISP (170.x.0.0/16) Seems the this ISP send a better advertisement then the second ISP; if so you have to agree a different metric for you networks usually as-prepend is used ore configre an eBGP session and change the NLRI attribute advertised to this ISP. 

The OP is advertising his aggregate to his current IP and more specifics to his new IPs during his migration. This way, the site stays up while he migrates subnet by subnet to the new ISPs ( that's how I read it). At any rate, this situation can occur if ISP_2 doesn't have the more-specific he's trying to announce.

 

26 REPLIES

 It sounds like ISP_2 does

 

It sounds like ISP_2 does not have your 170.x.254.x/24 prefix and/or is not propagating it. ISP_2 may need to update is Inbound Route Filter for your peer.

Have you verified the following:

  • You are actually advertising the 170.x.254.x/24 prefix to ISP_2? ( show ip bgp nei <peer_ip> advertised-routes)
  • ISP_2 is actually accepting the 170.x.254.x/24 prefix ( check looking glass )?

 

Can you provide the output for the two question above?

New Member

Joe Thanks for the response.

Joe

 

Thanks for the response.

 

Apparently ISP_2 did have some sort of routing/config issue on their end, but they won't elaborate.

 

I went to this site, http://tools.pingdom.com/ping and did traceroute to 170.x.254.4. It did take the correct path.

 

WAN_2-ASR1002x#show ip bgp nei 50.x.x.81 advertised-routes
BGP table version is (###), local router ID is (loopback0 ip)
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
              x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path
 *>  170.x.254.0/24  0.0.0.0                  0         32768 i

Total number of prefixes 1

 

 

IS this the site you are talking about? http://www.bgp4.as/looking-glasses

 

 

 

 

sMc

Yep. That's the correct one

Yep. That's the correct one.

Might be the ISP Engineer in me but when I'm dealing BGP, looking glasses are indispensable. As far as the "routing/config issue" ISP_2 wouldn't elaborate on...it was an inbound route filter/route-map. 

Get used to it. If your a stub AS, you have to tell them that you want to announce a new prefix like an animal. If you're a transit AS ( like ISP_1/ISP_2, Level3, Tata, etc.), you use a route registry like a Proper Sir :)

Since I don't know your IP Address space, can use one of the looking glass servers  to verify that your prefix is known via ISP_2? I don't want you to have a false resolve because ISP_2 did some routing trickery to make things work. Come back and post the output.

Joe

 

New Member

Joe Router: SF-200PAUL

Joe

 

Router: SF-200PAUL-CORE01
Command: show ip bgp 170.x.254.4


BGP routing table entry for 170.x.254.0/24, version 260553305
BGP Bestpath: med
Paths: (2 available, best #2, table default)
Multipath: eBGP
  Advertised to update-groups:
     1          40        
  Refresh Epoch 1
  174 (ISP_2 AS#) 33491 (MyAS#), (received & used)
    38.122.64.5 from 38.122.64.5 (66.28.1.182)
      Origin IGP, metric 2020, localpref 100, valid, external
      Community: 174:21000 (peer route, learned in NA) 174:22013
      rx pathid: 0, tx pathid: 0
  Refresh Epoch 1
  4436 (ISP_2 AS#) 33491 (MyAS#), (received & used)
    208.74.64.2 from 208.74.64.2 (208.74.64.41)
      Origin IGP, metric 23, localpref 110, valid, confed-internal, best
      Community: 4436:999 4436:31413
      rx pathid: 0, tx pathid: 0x0

sMc

sMc,Make sure you test the

sMc,

Make sure you test the failover, you could very well have the same issue with ISP_1 not propagating your 170.x.254.0/24 prefix as well.

While we're on it, this HSRP based failover you have...are you sites geographically distant? Or are you in campus. The only reason I ask is because if your using HSRP between sites, it implies you are spanning a layer 2 network between your sites ( might be on your diagram, just to lazy to look).

From a design perspective, you should span the layer 2 vlan/network. You should route between your two sites. The exception you _could_ get away with is if the sites were in the same building or on the same campus.

New Member

Joe The sites are approx 4

Joe

 

The sites are approx 4 miles apart. Dark fiber connects my DMZ switches that sit behind each ASR1002x.

DEFINATELY WILL TEST FAILOVER!

 

Question

 

The following is from an ISP perspective (PER)? Should I reverse the logic since I am coming from CER :

 

Suppose the following:
•    RS1 connected to ISP_1
•    RS2 connected to ISP_2
•    RS1 and RS2 have iBGP connection
 

Then
•    On RS1, incoming route-map for ISP_1 peer should increase local preference
•    On RS2, incoming route-map for ISP_2 peer should increase local preference
•    On RS1, outgoing route-map for RS2 should reset local preference (to 100)
•    On RS2, outgoing route-map for RS2 should reset local preference (to 100)

sMc

sMc, Oh, okay. Dark fiber is

sMc,

 

Oh, okay. Dark fiber is good, not active components so things like latency, loss, and jitter are bounded. In that case, works fine.

 

What I wrote above was from a customer perspective ( your perspective). The crux of the solution is whether or not your ASRs have an iBGP connection--it doesn't seem to be the case because you're using HSRP on the same LAN segment. If they don't run iBGP with each other, I'll need to revise the solution I gave you ( Hint: You'll end up load balancing with HSRP groups ). Otherwise, do you run an IGP between ASRs?

 

Joe

New Member

I have EIGRP runnning on ASR

I have EIGRP runnning on ASR routers

 

My topology looks like:

ISP>ASR>DMZ stack?ASA5545x>(Nexus7K Distribution switch - this physical link not made yet)

 

WAN_1 / ISP_1 router example

           router eigrp 171
           network 12.a.a.16 0.0.0.7
           network 170.x.254.0 0.0.0.255
           passive-interface Loopback0
            !
            router bgp AAAAA
            bgp log-neighbor-changes
            network 12.a.a.16 mask 255.255.255.248
            network 170.x.254.0 mask 255.255.255.0
            neighbor 12.y.y.9 remote-as ####

 

Will have static between ASA and ASR using the HSRP address

 

sMc
New Member

Joe Does this look like a

Joe

 

Does this look like a valid config

 

route-map LOCALPREF permit 10
set local-preference 200

router bgp AAAAA
neighbor 12.x.x.9 route-map LOCALPREF out

sMc
New Member

Also  Do i want to use local

Also

 

 

Do i want to use local pref on both WAN_1 ASR and WAN_2 ASR?/

sMc

You're missing a "match

You're missing a "match statement" and a prefix-list.

 

config t

ip prefix-list full-routes seq 5 permit 0.0.0.0/0 le 24 !!! Note, you don't want to accept more than a /24 from your ISP

end

config t

route-map LOCALPREF permit 10

match ip address prefix-list full-routes

set local-preference 200

end

config t

router bgp AAAAA

neighbor 12.x.x.9 router-map LOCALPREF in

end

 

The direction is inbound from your ISP. So that anything matched by the route-map "match" clause gets the "set" applied to it. 

 

But as I said in my other comment. My original assumptions were wrong, none of this applies if you don't run iBGP between ASRs.

New Member

I am running EIGRP between

I am running EIGRP between the ASR's

sMc

I'd a much more detailed

I'd a much more detailed topology before I can talk about failover. What I described was contigent on you having an iBGP connection between the ASRs.

Minimally, I'd need to know:

  • What devices have EIGRP adjacencies? Just ASR<->ASR? ASR<->ASA? ASR<->Nexus7K?
  • Where and how are you using static routing? Just at ASA? How is the traffic getting to the ASA? What is the logical connectivity like between sites?

I'm guessing you just took a fully redundant DMZ architecture and split that over two sites. Do you have two ASAs at either site (are they in Active/Active configuration)? 

New Member

EIGRP is between ASR>ASA at

EIGRP is between ASR>ASA at each site and will include the Nx7k.

 

The switch I refer to as DMZ is now simply a 3750x switch stack(2) between the ASR and the ASA. The new switch stack at Site_1 is connected via trunk to the switch stack at Site__2. HSRP is dependent on these (2) stacks.

We want to have these switch stacks as simply a passthru, no routing.

 

The current Production Internet/VPN network has the following topology:

WAN router>ASA>DMZ

Most of the traffic on the current DMZ is being filtered off for a completely separate project called "Affiliates".

 

ASA's are ACTIVE/Standby

 

sMc
New Member

here is topology

here is topology

sMc

sMc,Can we take this one

sMc,

Can we take this one offline? What's your timezone? Email me at joseph.nelson08<at>gmail<dot>com

 

There's some additional questions I want to ask you and the feedback loop on the forum is too long.

Joe

Silver

Not very clear, it seems me

Not very clear, it seems me that now your network is adverised by 3 ISP: the old one and the two new ISPs. In any case let me do some question:

 

I am migrating from a single Internet provider to (2) new Internet providers

Current provider is receiving (6) class "C" subnets (170.x.1.0/24 - 170.x.6.0/24) from me via static. They are then advertising my class "B" prefix network as a 170.x.0.0/16 via ibgp to their cloud and the rest of the world. I have verified this via http://bgpinspect.merit.edu

 

My (2) new ISP's

 

I am advertising my subnet (170.x.254.x/24) via BGP to both ISP's (ISP_1 and ISP_2)

No matter what I do, ISP_1 is always seen asthe path back to my company  what do you mean, what you did ?

I administratively shut down my WAN interface to ISP_1 and it still tried to come back across the ISP_1 path.

 

Finally the ISP_1 path disappeared and now instead of coming back across my ISP_2 path how much does it take ? Which value did you negotiated for this BGP session ?

traffic for the new subnet, (170.x.254.x/24) is being seen by the World as coming from my current ISP (170.x.0.0/16) Seems the this ISP send a better advertisement then the second ISP; if so you have to agree a different metric for you networks usually as-prepend is used ore configre an eBGP session and change the NLRI attribute advertised to this ISP. 

In any case seems me that there are some problem in your BGP config: why did you configure 

neighbor 12.x.x.9 default-originate

this way you are advertising a default route to the ISP, isn't  it ?

Moreover I would use iBGP instead of tracking a remote subnet. Finally  if you want one link to be primary and one backup, you have to differentiate the NLRI attribute advertised to the ISPs so that traffic from all over the world will prefer the primary instead of the backup.

 

Let me know, bye

enrico

Enrico, Regarding your

Enrico,

 

Regarding your comment:

"In any case seems me that there are some problem in your BGP config: why did you configure 

neighbor 12.x.x.9 default-originate

this way you are advertising a default route to the ISP, isn't  it ?"

I'm not to worried about the "default-originate" the OP has toward the ISP. Indeed it should not be there, however no ISP in the world is going to accept a default from a stub AS-- and if they did, the OPs routers would likely shutdown.

 

Regarding your comment:

traffic for the new subnet, (170.x.254.x/24) is being seen by the World as coming from my current ISP (170.x.0.0/16) Seems the this ISP send a better advertisement then the second ISP; if so you have to agree a different metric for you networks usually as-prepend is used ore configre an eBGP session and change the NLRI attribute advertised to this ISP. 

The OP is advertising his aggregate to his current IP and more specifics to his new IPs during his migration. This way, the site stays up while he migrates subnet by subnet to the new ISPs ( that's how I read it). At any rate, this situation can occur if ISP_2 doesn't have the more-specific he's trying to announce.

 

Silver

Hi Joseph,you're right about

Hi Joseph,

you're right about the default route: no ISP in the world will accept it but I could be interesting to know what was the goal of sMc was trying to achieve.

 

About more-specific advertisement you're right, I did notice (may be it's time for me to stop working and going home ;-)

 

Bye,

Enrico

New Member

Joseph Thank you for the

Joseph

 

Thank you for the response

"The OP is advertising his aggregate to his current IP and more specifics to his new IPs during his migration. This way, the site stays up while he migrates subnet by subnet to the new ISPs ( that's how I read it)." YES

 

The goal:

I have (2) physical sites. Each with it's own ISP connection.


Traffic originated at Site_1 will prefer ISP_1.

             should I use the Local pref attribute to assure this?

Traffic originated at Site_2 will prefer ISP_2

             should I use the Weight attribute to assure this?


ISP_1 will be failover for ISP_2 and vice verse.

The failover is accomplished by using the dual HSRP config shown in the diagram.


I had to use a /24 because one of the ISP' does not allow BGP advertisements less than /24.

I am advertising the same 170.x.254.0/24 in each WAN router to  each ISP

    WAN_1 / ISP_1 has 170.x.254.3, 170.0.254.5, 170.x.254.7 etc...

    WAN_2 / ISP_2 has 170.x.254.4, 170.x.254.6, 170.0.254.8 etc...

 

Instead of advertisng the subnet/24, should I advertise the specific interface addresses/24?

Will this cause BGP problems in future as more of the 170.x.254.0 addresses are used?

 

This really has me stumped. I am advertsing the same subnet 170.x.254.0/24 to (2) ISP's

How can I assure that local ip addresses are advertised to the local ISP?

 

 

 

 

sMc

sMc, I'd like to help you

sMc,

 

I'd like to help you solve your original problem. Please see my original response and provide that output ( if you have already, thanks). 

 

Regarding your other problems/questions, lets take them part by part:

 

Goal: Traffic originated at Site_1 will prefer ISP_1, Traffic originated at Site_2 will prefer ISP_2

Suppose the following:

  • RS1 connected to ISP_1
  • RS2 connected to ISP_2
  • RS1 and RS2 have iBGP connection

 

Then

  • On RS1, incoming route-map for ISP_1 peer should increase local preference
  • On RS2, incoming route-map for ISP_2 peer should increase local preference
  • On RS1, outgoing route-map for RS2 should reset local preference (to 100)
  • On RS2, outgoing route-map for RS2 should reset local preference ( to 100)

This configuration achieves your stated goals. If egress bound traffic hits RS1, it will use local preference and pick ISP_1 as egress, same for RS2. RS1 will also know a path through RS2, however, they will be receive-only because it already has chosen a path through ISP_2. 

GoalISP_1 will be failover for ISP_2 and vice verse.

No. You should not have a problem. Its okay to advertise your prefixes from more than one site, however, you have to be careful about which way ingress traffic flows. A lot of people like to play around with as_path...its really only a temporary solution on the Internet, because, realistically no one cares about your as-path length. If I'm an ISP, I'm not routing based on _your_ preference, I'm routing based on my preference/business drivers. 

That said, if you want to have an Active/Active ingress traffic flow, you'll keep playing around with as-path ( your your ISP's traffic engineering mechanisms). If you want an Active/Passive ingress traffic flow, look into BGP Conditional Annoucement

Question: Instead of advertisng the subnet/24, should I advertise the specific interface addresses/24?

Not really sure what you mean. You want to advertise the subnet/24, in your case 170.x.254.0/24.

Question: Will this cause BGP problems in future as more of the 170.x.254.0 addresses are used?

Not really sure which problem you are referring to here.

 

 

New Member

EnricoThank you for the

Enrico

Thank you for the response.

             I removed the default-originate statement.

 

             Not sure what you mean here "

                  how much does it take ? Which value did you negotiated for this BGP session ?

 

The goal:

I have (2) physical sites. Each with it's own ISP connection.


Traffic originated at Site_1 will prefer ISP_1.

             should I use the Local pref attribute to assure this?

Traffic originated at Site_2 will prefer ISP_2

             should I use the Weight attribute to assure this?


ISP_1 will be failover for ISP_2 and vice verse.

The failover is accomplished by using the dual HSRP config shown in the diagram.


I had to use a /24 because one of the ISP' does not allow BGP advertisements less than /24.

I am advertising the same 170.x.254.0/24 in each WAN router to  each ISP

    WAN_1 / ISP_1 has 170.x.254.3, 170.0.254.5, 170.x.254.7 etc...

    WAN_2 / ISP_2 has 170.x.254.4, 170.x.254.6, 170.0.254.8 etc...

 

 

 

 

 

 

 

 

 

 

 

 

 

 

sMc

Hi sMc, Not sure if you saw

Hi sMc,

 

Not sure if you saw my original reply, can you advise?

 

Joe

New Member

Joe You mean the AS Prepend?

Joe

 

You mean the AS Prepend?

sMc
New Member

I have similar configuration,

I have similar configuration, with two different ISPs. Each connecting to a single router. These two routers have EIGRP full mesh and redistribute.

BGP is configured on both routers, to their respective ISP. I have configured higher local preference on inbound to 200. Default is 100 is left alone.

Our netblock is a /23 which is advertised by both ISPs.

However, it seems the path to ISP2 is always preferred since I have used BGP Looking Glass and Traceroute, all of the results showing going through secondary ISP.

What am I missing?

 

Thanks

anh2lua@outlook.com,Why don't

anh2lua@outlook.com,

Why don't you message me here or start another thread--we'll go through it together. I don't want to mix threads because I'll get confused.

1255
Views
0
Helpful
26
Replies
CreatePlease login to create content