We have a fairly common edge config:
- Our own public ASN
- An ARIN-assigned address block
- Two routers, two ISPs. Full BGP routes from the providers and iBGP running between the edge.
- IPSec SAs established from each router to various upstream services
A recent flap reminded me that the IPSec SAs aren't tied to the respective routers - there's a shorter path to the upsteam IPSec peer through one of our ISPs, so both routers use that ISP for the endpoint IP. Obviously, we want the edge routers to ignore the iBGP routes for the remote IPSec endpoints so that an outage of one ISP will not cause all the tunnels to go down and reconnect.
I can think of a few different ways to do this (route-map with a prefix list inbound from the iBGP peer which reduces the local preference, route-map setting a community for inbound routes and then modifying local pref accordingly, etc). However, how would I go about this when I can assume that I don't know the prefix length?
Example - I have disparate IPSec endpoints, a.a.a.a/32 and a.a.a.z/32. Sure, I can look at the BGP table and see that I'm receiving an advertisement for a.a.a./22 - but I would be remiss in assuming that this will never change. If I set a prefix-list for a.a.a.a/22 le 32 or similar, that will get hosed if the upstream starts advertising a /21 or larger.
I'm sure there are probably numerous ways to accomplish this, but I'm looking some ideas on how others may have gone about influencing the outbound traffic for a single IP or a subnet that will be contained in a summary route which may change.
To influence outbound traffic for a single IP or a subnet (part of a summary route), you need to have some means by which you can define exit-interface,next-hop which would be different from the summary route.
One option is PBR but that will always route traffic to same mentioned interface/next-hop irrespective of the fact that ISP advertising that prefix or not.
Now my question is why would any ISP would do route aggregation for prefixes which they dont own. Any ISP does prefix aggregation of their own prefixes. In your case who is owning prefixes for IPSEC endpoints.
I was trying to avoid PBR to reduce the troubleshooting list for my team. In the end, I think I'm going to implement a local policy (since the tunnel endpoints on my side are loopback addresses and not physical adapters).
I never indicated that any ISP would aggregate non-administered addresses. My original question (and I think I'm still curious about this) relates to ensuring that we can manipulate the path even in the event that the end user / network either aggregates their address space or changes their prefix length. If the remote VPN endpoint network admin frees up a block of addresses and started advertising a /21 instead of a /22, any specific prefix-lists would not work as expected until I notice the change and update the list. This scenario is not unheard of (I have done exactly this in order to reduce our advertisements and/or influence traffic inbound to our ASN)
I assume that end points are not in your control that means you cannot restrict a.a.a.a/32 and a.a.a.z/32 advertisement to respective ISP's only.
For example your router one has two connectivity’s one internal and other external to ISP 1
I am assuming that you have eBGP peering with ISP1 and iBGP to internal peers in this scenario I am interested to know how iBGP path is getting preferred over eBGP. Is there more specific route learning from iBGP peer (Advertising one he learned from ISP2 connected router)? Route output from multiple routers would need to verify in detail.
In other scenario where you have eBGP peering with your both peers on router one and receiving similar length prefix and router one installed the route receiving from internal peer in its routing table. In this scenario you have two options -
One you mentioned increasing local preference (Using community string). You can have multiple route map instances example route-map XYZ 10 which will permit the routes you wanted to be preferred over any other path and second instance will permit rest traffic or you can keep it blank to permit all in-case of trusted partner.
Second, on Router two you can configure route-map with two instance and first will select the traffic you don’t want or to be less preferred from ISP2. On first instance you permit the traffic you wanted to be preferred from ISP1 and prepend the As-path. So that similar route at router one will get re-pointed to shorter as-path (ISP1) and in second instance permit all traffic you wanted to be preferred from ISP2.
If you think my assumption of the scenario is not right then I would need more detail like diagram with peering detail, etc.
Happy to help
Rate if you think this is helpful.
what about following idea:
Let's say you want to use your ISP A to reach the a.a.a.a/32 host.
But you are receiving a prefix a.a.a.a/x from your ISP B making your router to forward the packets to a.a.a.a via the ISP B.
So what about asking the ISP A to advertise some loopback x.x.x.x/32 to you?
And to configure a static route
ip route a.a.a.a 255.255.255.255 x.x.x.x
on your router (not advertise it anywhere though!).
As long as your router will be receiving the x.x.x.x/32 it will use recursive routing for a.a.a.a/32 and forward the packets to ISP A.
In a case the BGP session to ISP A fails, it will use some more general prefix received from ISP B (default route worst case) to reach x.x.x.x and forward packets for a.a.a.a host via ISP B (still using the recursive routing).
In case the ISP A would refuse to advertise the loopback IP for you, you could possibly choose some other x.x.x.x host typically reachable via ISP A?
Your solution will solve the purpose but if ISP-1 loose route to destination and our BGP with ISP-1 is up, static route will be active and traffic may get blackholed in ISP-1.
I can see two possibilities then:
a) a route-map modifying weight or local preference for all reasonable a.a.a.a/x prefixes received foem ISP-1.
But this solution wouild fail in a case ISP-2 would sent a more specific prefix at the same time.
b) Asking ISP-1 to configure conditional route injection advertisinga.a.a.a/32 in a case it has got any a.a.a.a/x prefix in his BGP table.
Which would work in any case but I'm afraid too much work and the ISP will refuse such a request.
The root issue isn't that ISP1 or ISP2 would advertise a more specific prefix. The issue would be if the upsteam (2-3 AS hops away) application provider where the tunnel is terminated decides to change their advertisement to the world - thus, our ISP would have zero control over this.
In addition, I don't know of any ISP that would modify their advertisements and start sending a /32 "just because we want it" for someone else's prefix.
Therein lies the problem. So, basically - if we are to handle this with our own path modification, here are the requirements if we are to use a route map (as I see it; there may very well be some methods which I am not familiar with - in which case, I would love to hear about them)
1. Identify the prefix containing the remote endpoints we need (each endpoint being a /32)
2. Once this prefix is identified, perform some modification to the advertisement - this could be adding a community and changing the local pref based on this, modifying the incoming local pref from our iBGP peer, something - there are a few options here)
3. All should be well.
#1 is the problem here. To reiterate - I can do a 'sh ip bgp x.x.x.x' to get the currently received prefix - for example, a /22. If the application provider decides to start sending a /21 because they changed their address policy, I've just lost control of the path selection. I need to ensure that whatever the prefix ends up being, I send traffic for the /32 out my local router interface.
That being said - I don't care if the traffic gets blackholed when the next hop is unavailable. The idea is that each edge router has an SA with the application provider - so if one gets routed to nowhere, our internal traffic should still flow.
Seems complicated, but in the end - it's all about identifying the prefix when we only know the most specific address (/32) - and we have to assume that the prefix may change someday, so a prefix-list can't be hardcoded to what it is today. Sorry if I'm confusing anybody.
In the meantime, the local policy route-map is working - I just wish there was a more elegant solution using BGP path manipulation