BGP Failover didn't fail back? Help!

[davidbnbf]

We have a netblock of IPs that are configured for BGP between 2 ISPs for failover and redundancy.  We had an issue with our primary site ISP and brought it back online.  Everything seemed fine and existing IPs and NATs are working properly for the primary site.  However I tried to set up a new NAT at that site with another public IP that should be tied to this ISP and I can't get any reply packets back from the internet.  Our theory is that the IP is tied to the secondary ISP for some reason during a BGP failover and never came back.

Unfortunately the person who configured BGP is long gone and I am not network expert.  I know enough to be dangerous :)

Can someone help me figure out how to reset the BGP so that the IPs are associated with the primary ISP instead of the secondary?  How can I restore this?  It was my understanding this configuration was supposed to be pretty automatically but something has clearly gone awry.

[davidbnbf]

We have owned this for a long time:
NetRange: 204.152.150.0 - 204.152.151.255

CIDR: 204.152.150.0/23


We have 204.152.150.0 on one side and 204.152.151.0 on the other side of the BGP. We only really use 204.152.150.0 side. All existing NATs are working on the firewall but I somehow can no longer setup new nats and have them work. About to get back on the phone with cisco. Going to take a look at the router next. So perplexed....

[Georg Pauwen]

Hello,

I would get on the phone with your ISP to make sure you still have access to the entire range. You own 510 public addresses...how many of those have you actually been actively using ?

204.152.150.207 happens to be the broadcast address for the 204.152.150.0/28 network.

[Jon Marshall]

Broadcast address for the 204.152.150.192/28 subnet ? 

Jon

[Georg Pauwen]

Sorry, I meant the 204.152.150.200/28 subnet...

Either way, checking with the ISP can't hurt.

[Georg Pauwen]

ARIN registration looks good, but that doesn't mean the ISP could not have made a mistake:

Source: whois.arin.netIP

Address: 204.152.150.207

Name: KKAMERICA

Handle: NET-204-152-150-0-1

Registration Date: 2/22/11

Range: 204.152.150.0-204.152.151.255

Org: K+K America CorporationOrg

Handle: KAC-21Address: 770 S 70th Street
City: MilwaukeeState/Province: WIPostal Code: 53214Country: UNITED STATES

[Jon Marshall]

See my last post, I have just done traceroutes to working and non working IPs and they all end up at the 2911 router so i think the ISP is routing everything correctly. 

Jon

[davidbnbf]

Agreed. Right now I am on phone with cisco packet capturing on the router and firewall.

[davidbnbf]

How do you know 204.152.150.207 is the broadcast address?

208,209,210,211,212 are all in use and working fine. I've tried to setup this new NAT with 205, 207, and 216 and they all have the same issue.

The IP block is registered with ARIN and configured on our BGN routers. What will our ISP be able to do?

[Georg Pauwen]

Hello,

what if you configure your static NAT entry with the actual interfaces instead of any/any:

object network VM-STOCKIQ

nat (inside,outside) static VM-STOCKIQ-PUBLIC net-to-net

[davidbnbf]

Figured it out with cisco tech.
There was a static route that only allows 208-215 to be routed to the firewall.
ip route 204.152.150.208 255.255.255.248 204.152.150.249 name NBF-Public-Block-1

It was picking up this other route which was not sending the traffic back to the firewall.
ip route 204.152.150.0 255.255.255.0 204.152.150.253 tag 666

[Jon Marshall]

Thanks for letting us know. 

What I don't understand is why it did not show when I asked for a static route output from the router, still good to hear it is working now. 

Jon

[davidbnbf]

The static route was configured on the switch and not the router that is why we didn't see it earlier. I only got on the switch and started looking at that very late in the day.

[Jon Marshall]

So the switch was L3 after all. 

Okay, that makes sense now, thanks for clearing that up. 

Jon