Cisco Support Community
Showing results for 
Search instead for 
Did you mean: 

Borrowed IP Address from upstream ISP is giving me headache.

Its like this;

I have two routers (R1 and R2), running my own AS and connected to three upstream ISP (ISP1, ISP2, ISP3). I have one customer (Customer_Pain)comes from ISP1 and previously using their IP, because of technical reason from the customer side, they brought over the ISP1 two class C address to my Data Centre. Now since I have a connection to ISP1, what ISP1 do is statically route this two class C IP Address to my router R1. ISP1 allowed me to configure this two class C IP Address in my router and add it to my BGP advertisement to ISP2 (because ISP2 is a local high speed cloud > not going international).

Now my problem is, I connected my router R2 to ISP3 and only advertised my own IP Address block in my AS (cause ISP1 doesn't allow me to advertise the two IP Address > because it will be going international). If I increase the AS Local Preference to ISP3 (so that outgoing international traffic will go via ISP3), Customer_Pain encounter an outgoing traffic problem but not incoming and reply traffic (they cannot reach their destination > this is explanable because I didn't advertised their two class C IP Address to my BGP peering with ISP3).

My question is this;

How can I increase the AS Local Preference to ISP3 without affecting Customer_Pain outgoing traffic.

NOTE: We can't use policy-routing here because Customer_Pain have 5 ligitimate exit points, two to ISP1, two to ISP2 and one to their NOC.

What I See: When the route is in the router table everybody will choose the route to a destination with the highest Local Preference.

What I'm Thinking: To make Customer_Pain see only the routes thru and to ISP1, ISP2, and their NOC but not thru and to ISP3 and other ISP in the future. But how to do it? Is there any other way?

The option of changing Customer_Pain two class C from ISP1 to my IP Adress is not an option here because Customer_Pain doesn't want to do it (that's why I call them Customer_Pain).


Re: Borrowed IP Address from upstream ISP is giving me headache.

Without knowing a lot more about how you have things set up, it's hard to make a specific recommendation. But when you say you want the customer to only see routes from ISP1 and ISP2, does this mean that the customer has a router that runs BGP one of your routers? If so, it would be easy to filter the routes you send to the customer via a distribute list or route map so that the customer doesn't see routes from ISP3.

If this isn't the case, or if you only send the customer a default route, I can't think of another way to do what you want other than policy routing. This doesn't mean there isn't one... Again, the details of the topology oftentimes need to be known to fix these types of issues.

Re: Borrowed IP Address from upstream ISP is giving me headache.

Bear with this, I think you might need to draw it. I'm sorry about that, I intent to draw it, maybe you can give me your email address and I send it to you. Anyway thanks for trying to help, hope you can help me more about my problem.












1. Legend

ISP1-RTR1 and ISP1-RTR2 = ISP1 router 1 and router 2

ISP2-RTR1 and ISP2-RTR2 = ISP2 router 1 and router 2

NOC-RTR1 = Customer_Pain router 1

MY-RTR1 and MY-RTR2 = My data centre router 1 and router 2

ISP3-RTR1 = ISP3 router 1

2. Connection

ISP1-RTR1/ISP1-RTR2, ISP2-RTR1/ISP2-RTR2, and NOC-RTR1 is conneted to my MY-RTR1 via one ATM Port (5 PVCs). ISP3-RTR1 is connected to my MY-RTR2. There is a Switch connected to my routers (MY-RTR1/MY-RTR2), all my customers are connected to my switch by firewall (not router). All my customer use a subnet from One of my customer (Customer_Pain) comes from ISP1, which brought ISP1 IP subnet of Since I have a connection to ISP1, ISP1 statically route to my MY-RTR1. ISP1 BGP peering with upstream ISP/IX is By right when I'm BGP peering with ISP1, ISP2, and ISP3, I should peer only I BGP peer with ISP1 and ISP3 using only, but when I'm BGP peering with ISP2 I use plus Reason for this is because ISP2 is local only, the traffic never go international (or say the internetworld), and ISP1 allow me to add in my BGP peering with ISP2 the

3. Type of connection

MY-RTR1 is BGP peering with ISP1-RTR1/ISP1-RTR2 and ISP2-RTR1/ISP2-RTR2. MY-RTR1 have static routing with NOC-RTR1. MY-RTR2 is BGP peering with IPS3-RTR1. From ISP1-RTR1/ISP1-RTR2 is statically route to MY-RTR1.

4. IP Address

My IP is

ISP1 IP is

ISP2, ISP3, and NOC IP is not needed for this example.

5. Problem

After you digest number 2 and draw the diagram for the scenario my problem is this. When I'm only connected to ISP1 and ISP2 I don't have a problem with routing for Customer_Pain (the one using ISP1 IP of Now I connected to ISP3 and I'm not allow to add to my BGP peering with ISP3 (I use only After established connection with ISP3, some of the Far Away Country IP Address is reachable via ISP3 (best path). Then my Customer_Pain have problem because from his source IP of he can see the best path to via ISP3 but ISP3 doesn't know this IP (ISP3 only know that I have therefore drop before it leaves MY-RTR2 router (I have access list that only can go out to ISP3 and ISP3 have access-list that only should come in to his router from my router). What I did now is make the AS Local Preference to ISP3 very low so that in my BGP routing table, to go out, ISP2 is the best path, ISP1 is the second best path, ISP3 is the last best path.

6. What I want.

I want to make ISP3 as the second best path so that I can use fully the pipe, but now there is no traffic flowing there but BGP TCP Keepalives.

7. What I observe.

Once you established peering with 3 ISP, the BGP routing table will populate with Best Path to reach a prefix outside and which AS advertising the path.

8. What to do

How to make not to choose the path thru ISP3 even if I make ISP3 as the best path to anywhere.

9. Policy routing? have 5 ligitimate exits, ISP1-RTR1/ISP1-RTR2, ISP2-RTR1/ISP2-RTR2, and NOC-RTR1.

10. My routing for

I dont' do any static or default routing to upstream ISP, only eBGP. Internally I use OSPF for I have only static routing to NOC-RTR1 for to reach Customer_Pain NOC.


Re: Borrowed IP Address from upstream ISP is giving me headache.

Couple follow-up questions:

- Do MY-RTR1 and MY-RTR2 run any routing protocols with each other? BGP? OSPF?

- If customers connect to your datacenter switch via a firewall and not a router, how is a given customer able to make routing decisions for outbound traffic? The customer firewall isn't running BGP, is it? At some point, a routing decision needs to be made to send a given outbound packet to either MY-RTR1 or MY-RTR2. How is this done? Is the switch that customers connect to a layer 3 device that is running a routing protocol with your routers? Or does each customer have a router that runs BGP with your routers?

If this customer runs BGP with you, then you should easily be able to, on the customer's router, filter all BGP advertisements that come from ISP3's AS. The result would be routes only from ISP1 and ISP2 in the customer's routing table. But I get the feeling it isn't this easy, so let me know what piece I'm missing here.

Re: Borrowed IP Address from upstream ISP is giving me headache.

MY-RTR1 and MY-RTR2 is using OSPF and IBGP with HSRP as configured gateway for each customer per vlan.

My customer with I only BGP peer with ISP2, ISP1 have static routing to MY-RTR1 for because ISP1 own that IP. ISP3 see from ISP1 not from my AS. I don't have any customer BGP peering with me.


Re: Borrowed IP Address from upstream ISP is giving me headache.

After all the time you took to explain the situation, I'm unfortunately unable to think of a solution with your current architecture. As mentioned above, this would be easily fixable if the customer connected to you via a router instead of a firewall. Is this an option? I would consider it a better design choice anyway because of the flexability it offers (as this situation proves).

Other than that, the only option I see is to always send traffic from this customer to a given router (e.g., ISP1-RTR1) unless that ATM PVC is down, in which case you send it out ISP1-RTR2, and so on. This can be done with policy routing and 'set ip next-hop [ISP1-RTR1 ip] [ISP1-RTR2 ip] ... etc' . And you can get traffic to the customer's NOC as necessary by using 'match ip address' on an extended access list that specifies the customer's netblock as the source and the NOC's netblock as the destination.

Re: Borrowed IP Address from upstream ISP is giving me headache.

Well, purchasing a router for this customer is not an option. The only sure option that I can get is to make the customer return the IP Address block to ISP1, but this customer is a blockhead and doesn't want to do it, even asking for a very high SLA for internet connectivity which we can't give because of their IP depencency to ISP1.

For the policy routing, there are four internet exit point, ISP1-RTR1 and RTR2, ISP2-RTR1 and RTR2. The NOC is easy. IF the internet exit is only two (ISP1-RTR1 and RTR2) it should be easy also.

Anyway thanks for the effort.

CreatePlease to create content