Solved: Re: BGP Multihoming & single Router, load sharing and automatic fail over - Page 2

gasparmenendez · ‎11-06-2017

hi folks,

after been waiting for a while finally I'm ready to have BGP with my 2 ISP's...right now I have this configured on my ASR:

router bgp 2625
bgp log-neighbor-changes
network 170.X.X0.0 mask 255.255.252.0
neighbor 208.Y.Y.181 remote-as 35498
neighbor 208.Y.Y.181 password 7 151E66
neighbor 208.Y.Y.181 route-map L3 out
!
ip prefix-list L3 seq 10 permit 170.X.X0.0/22 le 27
ip prefix-list L3 seq 15 deny 0.0.0.0/0
!
route-map L3 permit 10
match ip address prefix-list L3
!

Contencion1001-X#sh ip bgp summary
BGP router identifier 208.Y.Y.182, local AS number 2625
BGP table version is 2508974, main routing table version 2508974
655803 network entries using 162639144 bytes of memory
655803 path entries using 78696360 bytes of memory
103801/103751 BGP path/bestpath attribute entries using 25742648 bytes of memory
89989 BGP AS-PATH entries using 4634324 bytes of memory
4619 BGP community entries using 575242 bytes of memory
48 BGP extended community entries using 1344 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 272289062 total bytes of memory
BGP activity 684995/29188 prefixes, 705406/49603 paths, scan interval 60 secs

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
208.Y.Y.181 4 35498 563553 3349 2508734 0 0 1d01h 655802
Contencion1001-X#

and it's working fine, but as you can see I still don't establish BGP with my second ISP. What I want to achieve is the following (after establish BGP session with my second ISP):

want to split my whole public network (/22) in two /23 to make load-sharing
want to pass 170.X.X0.0/23 inbound and outbound traffic via LVL3
want to pass 170.X.X2.0/23 inbound and outbound traffic via MTC

both routes must be active at the same time
in case of failure of any one of the ISP's, traffic must be automatically routed via the other one

things to be considered:

I have a whole block of public IP addresses 170.80.240.0 /22
ISP LVL3 is sending me the full routing table and ISP MTC only default-route

I found this post: https://supportforums.cisco.com/t5/wan-routing-and-switching/bgp-multihoming-with-2-isp-on-single-router/td-p/2722860which is very similar to what I want to do and tried to adjust to my specific scenario, but since my network is in production I need to be very carefull before make any changes This is what I think I need to set up:

router bgp 2625
bgp log-neighbor-changes
no synchronization
no auto-summary
network 170.X.X0.0 mask 255.255.254.0
network 170.X.X2.0 mask 255.255.254.0
neighbor 208.Y.Y.181 remote-as 35498
neighbor 208.Y.Y.181 desc LVL3
neighbor 208.Y.Y.181 password ********
neighbor 208.Y.Y.181 route-map L3 out
neighbor 208.Y.Y.181 route-map UPSTREAM1 out
neighbor 208.Y.Y.181 prefix-list FULL in
neighbor 187.Z.Z.112 remote-as 1933
neighbor 187.Z.Z.112 desc MTC
neighbor 187.Z.Z.112 password ********
neighbor 187.Z.Z.112 route-map MT out
neighbor 187.Z.Z.112 route-map UPSTREAM2 out
neighbor 187.Z.Z.112 prefix-list DEFAULT-ONLY in
!
!
interface TenGigabitEthernet0/0/0
description *** ISP Level3 ***
ip address 208.Y.Y.182 255.255.255.252
!
interface TenGigabitEthernet0/0/1
description *** ISP Metrocarrier ***
ip address 187.Z.Z.113 255.255.255.254
!
interface TenGigabitEthernet0/1/0
description *** ASA 5580 ***
ip address 170.X.X0.1 255.255.255.240
ip policy route-map OUTBOUND-INTERNET
!
!
ip prefix-list DEFAULT-ONLY seq 10 permit 0.0.0.0/0 =======> should I set another one for the FULL table ????
!
!
ip prefix-list L3 seq 10 permit 170.X.X0.0/22 le 27
ip prefix-list L3 seq 15 deny 0.0.0.0/0
!
route-map L3 permit 10
match ip address prefix-list L3
!
!
ip prefix-list MT seq 10 permit 170.X.X0.0/22 le 27
ip prefix-list MT seq 15 deny 0.0.0.0/0
!
route-map MT permit 10
match ip address prefix-list MT
!
!
ip access-list extended LVL3
permit ip 170.X.X0.0 0.0.1.255 any
ip access-list extended MTC
permit ip 170.X.X2.0 0.0.1.255 any

!
ip sla 1
icmp-echo 208.Y.Y.181 source-interface TenGigabitEthernet0/0/0
frequency 30
ip sla schedule 1 life forever start-time now
ip sla 2
icmp-echo 187.Z.Z.112 source-interface TenGigabitEthernet0/0/1
frequency 30
ip sla schedule 2 life forever start-time now
!
track 1 ip sla 1 reachability
track 2 ip sla 2 reachability
!
route-map OUTBOUND-INTERNET permit 10
description send 170.X.X0.0/23 out Level3
match address LVL3
set ip next-hop verify-availability 208.Y.Y.181 10 track 1
!
route-map OUTBOUND-INTERNET permit 20
description send 170.X.X2.0/23 out Metrocarrier
match address MTC
set ip next-hop verify-availability 187.Z.Z.112 20 track 2
!
ip access-list standard bgp-up1
permit 170.X.X0.0 0.0.1.255
!
ip access-list standard bgp-up2
permit 170.X.X2.0 0.0.1.255
!
route-map UPSTREAM1 permit 20
match address bgp-up1
route-map UPSTREAM1 permit 40
match address bgp-up2
set as-path prepend 47117
!
route-map UPSTREAM2 permit 20
match address bgp-up2
route-map UPSTREAM2 permit 40
match address bgp-up1
set as-path prepend 47117
!

Can anybody help me please and tell me what I need to modify in order to achieve my gold??

Thanks in advance.

Georg Pauwen · ‎11-09-2017

Hello,

I think the problem is that you prefix list only allows networks with a mask between 22 and 27, while all the routes have masks that are larger, so they never get advertised:

S 170.Z.Z0.0/23 is directly connected, Null0
C 170.Y.Y.0/28 is directly connected, TenGigabitEthernet0/1/0
L 170.Y.Y.1/32 is directly connected, TenGigabitEthernet0/1/0
S 170.Y.Y.16/28 [1/0] via 170.Y.Y.2
C 170.Y.Y.32/29 is directly connected, GigabitEthernet0/0/1
L 170.Y.Y.33/32 is directly connected, GigabitEthernet0/0/1
C 170.Y.Y.40/29 is directly connected, GigabitEthernet0/0/2
L 170.Y.Y.41/32 is directly connected, GigabitEthernet0/0/2
S 170.Y.Y.48/28 [1/0] via 10.170.18.2
C 170.Y.Y.64/30 is directly connected, GigabitEthernet0/0/4.65
L 170.Y.Y.65/32 is directly connected, GigabitEthernet0/0/4.65
C 170.Y.Y.68/30 is directly connected, GigabitEthernet0/0/4.69
L 170.Y.Y.69/32 is directly connected, GigabitEthernet0/0/4.69
C 170.Y.Y.72/30 is directly connected, GigabitEthernet0/0/4.44
L 170.Y.Y.73/32 is directly connected, GigabitEthernet0/0/4.44
S 170.Y.Y.80/28 [1/0] via 170.Y.Y.2
C 170.Y.Y.96/30 is directly connected, GigabitEthernet0/0/4.20
L 170.Y.Y.97/32 is directly connected, GigabitEthernet0/0/4.20
S 170.Y.Y.100/30 [1/0] via 10.147.24.2
S 170.Y.Y.108/30 [1/0] via 10.147.89.2
S 170.Y.Y.112/30 [1/0] via 10.147.21.2
S 170.Y.Y.116/30 [1/0] via 10.147.197.2
S 170.Y.Y.120/30 [1/0] via 10.147.27.2
S 170.Y.Y.124/30 [1/0] via 10.147.29.2
S 170.Y.Y.128/30 [1/0] via 10.147.26.2
S 170.Y.Y.132/30 [1/0] via 10.147.28.2
S 170.Y.Y.136/30 [1/0] via 10.147.25.2
C 170.Y.Y.140/30 is directly connected, GigabitEthernet0/0/4.75
L 170.Y.Y.141/32 is directly connected, GigabitEthernet0/0/4.75
S 170.Y.Y.144/30 [1/0] via 10.147.20.2
S 170.Y.Y.148/30 [1/0] via 10.147.31.2
S 170.Y.Y.152/29 [1/0] via 10.147.22.2
S 170.Z.Z2.0/23 is directly connected, Null0
B 170.Z.Z4.0/22 [20/15184] via 208.X.X.181, 01:37:02

So change the prefix list to /22 le 32 in order to include all routes. You probably also need to 'redistribute connected' and 'redistribute static' under the BGP process, or manually advertise the networks you require, with the 'network' statement...

Richard Burts · ‎11-10-2017

I do not agree with this advice. If this were a lab exercise or if it were BGP entirely within a private network then I might agree. But this is using BGP to advertise routes to an ISP in the real Internet. Remember that many ISP have policies that they will not accept any advertisement smaller than /24. The only networks that really need to be advertised are the 2 /23 networks. And output in a previous post shows that these are matching the prefix list and are being advertised.

And similar reaction to the suggestion about redistribute of connected or static. There is no benefit in advertising ALL of the entries in the routing table. The only ones that matter for BGP are the 2 /23.

I find this issue quite puzzling. It appears that there is successful connectivity to the Internet when using the first ISP but not connectivity to the Internet when using only the second ISP. Previous output shows that the second ISP is advertising a single route (which we assume is the default route) and shows that the two interior networks are being advertised to the second ISP. So I am having trouble seeing this situation as an issue with routing for the second ISP. So what is the issue here? I think we need to check on the possibility that the second ISP is not accepting the routes being advertised. I think we also need to look for the possibility that there is some policy in the network that is not using the second ISP. Perhaps we can see the full config of the Internet router to see if there is something there?

HTH

Rick

HTH

Rick

Georg Pauwen · ‎11-10-2017

Ouch...I fully agree with Richard not agreeing putting my advice into a live network. I recreated this in GNS3, and it is purely a hypothetical approach and meant as a lab excercise to prove that everything actually does get advertised to both ISPs...my apologies.

Either way, I see the followng route in the routing table:

B 170.Z.Z4.0/22 [20/15184] via 208.X.X.181, 01:37:02

Is this the same address space you are trying to advertise, and is it coming from the same IP address as the one you configured for your second ISP neighbor...?

gasparmenendez · ‎11-10-2017

I'm seeing these in my routing table:

B        170.Z.Z44.0/22 [20/15184] via 208.X.X.181, 23:01:03
B        170.Z.Z52.0/22 [20/15054] via 208.X.X.181, 22:16:14
B        170.Z.Z52.0/23 [20/3703] via 208.X.X.181, 18:15:38
B        170.Z.Z54.0/23 [20/3703] via 208.X.X.181, 18:15:38
Contencion1001-X#

all those networks are outside my block of public ip addresses, in the config. I sent you yesterday by PM you can see which is my block. Besides, all of them are coming from the IP address of my first ISP neighbor (208.X.X.181, this is the one is working fine)...

Georg Pauwen · ‎11-10-2017

Hello,

looking at your config again, the below doesn't seem right:

interface TenGigabitEthernet0/0/1
description *** ISP Metrocarrier ***
ip address 187.Z.Z.113 255.255.255.254

!

neighbor 187.Z.Z.112 remote-as 1933

Change it to:

interface TenGigabitEthernet0/0/1
description *** ISP Metrocarrier ***
ip address 187.Z.Z.113 255.255.255.252

neighbor 187.Z.Z.114 remote-as 1933

and check if that makes a difference...

Richard Burts · ‎11-11-2017

While I agree that an interface with mask 255.255.255.254 is unusual, I would say that it is not necessarily a problem. And considering that output in previous posts shows that the interface is active and that it has successfully formed a BGP neighbor relationship with the peer at 187.Z.Z.112 it seems that changing the interface address configuration and changing the BGP neighbor address would do damage to the production environment (unless the existing peer at 187.Z.Z.112 agrees to the changes).

HTH

Rick

HTH

Rick

gasparmenendez · ‎11-11-2017

I just spoke with ISP and addressing is correct, besides you're right, the interface is active and it has successfully formed a BGP neighbor relationship with the peer...the ISP is checking their configuration. Keep you posted

thanks.

Richard Burts · ‎11-11-2017

I am glad that the ISP has confirmed that the interface configuration and BGP neighbor configuration are correct. I am also glad that they are checking their configuration. Am I correct in assuming that the /22 IP address block that you have was assigned to you from ISP 1? If so was there a letter of authorization from them for you to advertise the address block to ISP 2? Is ISP 2 checking their routing policies to verify that they do accept the advertisement from you?

HTH

Rick

HTH

Rick

gasparmenendez · ‎11-11-2017

The /22 IP address block that I have was assigned to me by the IANA in Mexico, so that address block belong to the company I work for and I advertise to both ISPs...

Georg Pauwen · ‎11-11-2017

This thread has become quite long so I don't know if the output has already been posted, but Richard suggested earlier to check if anything gets advertised at all ?

show ip bgp neighbor 187.x.x.x advertised-routes

Richard Burts · ‎11-11-2017

Yes this thread is a bit long. But it does contain this output

I'm indeed advertising my network to 187.Z.Z.112:

Contencion1001-X#sh ip bgp neighbors 187.Z.Z.112 advertised-routes
BGP table version is 7410037, local router ID is 208.X.X.182
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
              x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path
*> 170.Z.Z0.0/23 0.0.0.0                  0         32768 i
*> 170.Z.Z2.0/23 0.0.0.0                  0         32768 i

Total number of prefixes 2

So at that point both /23 were being advertised. I assume that the config has not changed much and that both routes are still being advertised. But it would not hurt to ask for fresh output to determine that this is still the case.

HTH

Rick

HTH

Rick

gasparmenendez · ‎11-11-2017

hi folks, as right now my priority is to pass traffic through both ISPs (and not load-sharing), I'm back to only the whole ip addreses block /22. So here's what you asked:

Contencion1001-X#sh ip bgp neighbors 187.X.X.112 advertised-routes
BGP table version is 4119689, local router ID is 208.51.117.182
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
              x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path
*> 170.Z.Z.0/22 0.0.0.0                  0         32768 i

Total number of prefixes 1
Contencion1001-X#

thanks.

Richard Burts · ‎11-11-2017

This output does verify that you are advertising your address space to ISP 2. Previous output showed that they are advertising 1 route to you. Assuming that this is a default route then from a routing perspective you should be able to be successful in sending and receiving traffic through ISP 2. If that is not working then we need to look for the problem, but I do not believe that the problem will turn out to relate to routing.

As I said in a previous post there might be some possibility that ISP 2 is not accepting your advertisement. You should ask them to verify that they see and do accept your advertisement. There is also some possibility that they have some policy in place that impacts your traffic. You should ask them to check on this also.

Is ISP 2 a smaller ISP than ISP 1? Is it possible that ISP 2 is a downstream peer of ISP 1 (would traffic from the Internet coming to you go through ISP 1 to get to ISP 2)?

HTH

Rick

HTH

Rick

gasparmenendez · ‎11-11-2017

ISP 2 is checking its confgiuration, to see if they can find the problem....

and about the comparative between both ISPs, I think ISP 1 is bigger than ISP 2 as provider....

Keep you posted

thanks.

Tinashe Ndhlovu · ‎11-11-2017

Hi,

I have gone quickly through the thread which is quite long and i would like to throw in a few ideas here.

Your BGP Sessions come up and you receive routes including the default route from ISP1 and ISP2

I understand your issue to be when ISP1 fails the internet drops... how long did you wait after disabling the interface to ISP1? I could be wrong but outgoing traffic may very well have been going out via ISP2 and return traffic coming back via ISP1 because the world hadn't converged and observed that your ISP1 link was down. Unless you are peering over a direct xconnect in ISP1's DC or in a colocation DC, ISP1 may have been waiting for the BGP hold down timer to expire before withdrawing your prefixes. This is normally the case when VLANs are incorporated by ISP's for /30 connections meaning the VLAN interface on their end is always up even if your remote physical interface is down because they connect to a switch interface. So the ISP will wait the duration of the timers before considering the peer down and withdrawing your routes. During this period you will have no connectivity as packets are routed to a dead interface until re-convergence occurs. I hope this makes sense.