CGN Configuration for BGP Router

[ingot.marulam]

Hi all,

I am in the middle of NAT configuration testing for new ASR9010 implementation. Customer need NAT feature and ordered ISM for the CGN. Need your help to check our configuration and the testing environtment, because we found this configuration didn't work with this testing environtment. The testing environtment is described below.

The configurations are:

-----------------------------------------

!

vrf InsideUserNAT

address-family ipv4 unicast

!

!

vrf InsideWifiNAT

address-family ipv4 unicast

!

!

vrf InsideOfficeNAT

address-family ipv4 unicast

!

!

hw-module service cgn location 0/0/CPU0

!

interface GigabitEthernet0/1/1/0

description NAT Test 1

ipv4 address 10.1.9.129 255.255.255.0

transceiver permit pid all

!

interface GigabitEthernet0/1/1/1

description NAT Test 2

ipv4 address 100.62.16.5 255.255.255.252

transceiver permit pid all

!

interface ServiceApp1

description ASVI for InsideUserNAT

vrf InsideUserNAT

ipv4 address 1.1.1.1 255.255.255.252

service cgn cgn1 service-type nat44

!

interface ServiceApp2

description ASVI for OutsideUserNAT

ipv4 address 2.1.1.1 255.255.255.252

service cgn cgn1 service-type nat44

!

interface ServiceApp3

description ASVI for InsideOfficeNAT

vrf InsideOfficeNAT

ipv4 address 3.1.1.1 255.255.255.252

service cgn cgn1 service-type nat44

!

interface ServiceApp4

description ASVI for OutsideOfficeNAT

ipv4 address 4.1.1.1 255.255.255.252

service cgn cgn1 service-type nat44

!

interface ServiceApp5

description ASVI for InsideWifiNAT

vrf InsideWifiNAT

ipv4 address 5.1.1.1 255.255.255.252

service cgn cgn1 service-type nat44

!

interface ServiceApp6

description ASVI for OutsideWifiNAT

ipv4 address 6.1.1.1 255.255.255.252

service cgn cgn1 service-type nat44

!

interface ServiceInfra1

ipv4 address 100.10.10.1 255.255.255.252

service-location 0/0/CPU0

!

!

router static

address-family ipv4 unicast

  100.62.16.0/22 Null0 210

  100.62.16.0/24 ServiceApp2

  100.62.17.0/24 ServiceApp4

  100.62.18.0/24 ServiceApp6

!

vrf InsideUserNAT

  address-family ipv4 unicast

   0.0.0.0/0 ServiceApp1

   10.1.9.0/24 GigabitEthernet0/1/1/0 10.1.9.130

  !

!

vrf InsideWifiNAT

  address-family ipv4 unicast

   0.0.0.0/0 ServiceApp5

  !

!

vrf InsideOfficeNAT

  address-family ipv4 unicast

   0.0.0.0/0 ServiceApp3

  !

!

!

service cgn cgn1

service-location preferred-active 0/0/CPU0

service-type nat44 nat1

  inside-vrf InsideUserNAT

   map ip one-to-one

   map address-pool 100.62.16.0/24

  !

  inside-vrf InsideWifiNAT

   map address-pool 100.62.17.0/24

  !

  inside-vrf InsideOfficeNAT

   map address-pool 100.62.18.0/24

  !

  protocol udp

   session active timeout 20

  !

!

!

end

RP/0/RSP0/CPU0:BGP-NAT#term leng 24

Wed Jul 10 00:08:35.907 UTC

-----------------------------------------

We can reach internet ip address from GigabitEthernet0/1/1/1. ServiceInfra interface and all serviceapp interfaces are up. Need help check this issue because we will do migration by the end of this week.

Thanks in advance and really appreciate your help.

[Harold Ritter · Accepted solution]

Hi Marulam,

There is a few things a noticed.

1. There is no VRF configured on interface GigabitEthernet0/1/1/0. This interface should be part of InsideUserNAT.

2. I do not see a default route configured in the global routing table to reach the Internet. The default route should be added with a next hop address of 202.62.16.6.

3. The pool used (202.62.16.0/24) overlaps with the outside interface (202.62.16.4/30).

Regards

[ingot.marulam]

Hi Harold,

Thanks for the reply. I tried to change the configuration as you said and found this log:

cgv6_services[140]: core_event_logging_init[835] core 2, Error code 3, Msg: **  i-vrf: 0x400c :: ipv4 addr: 0xa010982 :: port: 0xde5c ::  ** , 'cgv6_services' detected the 'warning' condition 'no pool for any'

Any Idea?

[ingot.marulam]

Hi Harold,

I did some ISM module restart and configure the box same as you told before. But still found the Host couldn't reach internet. I got this capture:

!

!

RP/0/RSP1/CPU0:BGP-NAT#sh cgn nat44 nat1 statistics

Wed Jul 10 17:40:06.145 UTC

Statistics summary of NAT44 instance: 'nat1'

Number of active translations: 100

Number of sessions: 142

Translations create rate: 0

Translations delete rate: 0

Inside to outside forward rate: 6

Outside to inside forward rate: 0

Inside to outside drops port limit exceeded: 5144

Inside to outside drops system limit reached: 0

Inside to outside drops resource depletion: 0

No translation entry drops: 70

PPTP active tunnels: 0

PPTP active channels: 0

PPTP ctrl message drops: 0

Number of subscribers: 0

Drops due to session db limit exceeded: 0

Pool address totally free: 767

Pool address used: 1

Pool address usage:

-------------------------------------------------

  External Address       Ports Used 

-------------------------------------------------

  100.62.16.126          100

-------------------------------------------------

!

show cgn nat44 nat1 outside-translation protocol icmp outside-vrf default outside-address 100.62.16.126 port start 1 end 65535:

!

RP/0/RSP1/CPU0:BGP-NAT#show cgn nat44 nat1outside-translation protocol icmp outside-vrf default outside-address 100.62.16.126$

Wed Jul 10 17:41:59.044 UTC

Outside-translation details

---------------------------

NAT44 instance : nat1

Outside-VRF    : default

--------------------------------------------------------------------------------------------

   Inside          Protocol  Outside      Inside         Translation  Inside      Outside

   Address                   Destination  Destination    Type         to          to

                             Port         Port                        Outside     Inside

                                                                      Packets     Packets

--------------------------------------------------------------------------------------------

  10.1.9.130         icmp    32007        1             dynamic       42          0          

=====

Any suggestion?

[Nicolas Fevrier · Accepted solution]

Hi Marulam,

you are hitting the default port limit (100),

please increase the port limit to something higher in the inside VRFs,

Kind regards,

N.

[ingot.marulam]

Hi Nicolas,

Does it make the host couldn't reach internet?

I will try increase the port limit.

Thanks btw.

[Nicolas Fevrier]

Hi,

if it doesn't change anything, please share the latest config after the modification recommend byHarold have been applied,

Thanks,

N.

[ingot.marulam]

Hi Nicolas,

I tried and it didn't change anything, here the latest configuration:

Building configuration...

!! IOS XR Configuration 4.3.1

!! Last configuration change at Wed Jul 10 19:10:07 2013 by user

!

!

interface GigabitEthernet0/1/1/0

description NAT Test 1

vrf InsideUserNAT

ipv4 address 10.1.9.129 255.255.255.0

transceiver permit pid all

!

interface GigabitEthernet0/1/1/1

description NAT Test 2

shutdown

transceiver permit pid all

interface GigabitEthernet0/1/1/19

shutdown

!

interface TenGigE0/1/0/0

load-interval 30

dampening

!

interface TenGigE0/1/0/1

dampening

!

interface TenGigE0/1/0/1.751

load-interval 30

encapsulation dot1q 751

!

interface TenGigE0/1/0/1.752

load-interval 30

encapsulation dot1q 752

!

interface TenGigE0/1/0/2

shutdown

!

interface TenGigE0/1/0/3

shutdown

!

interface ServiceApp1

description ASVI for InsideUserNAT

vrf InsideUserNAT

ipv4 address 1.1.1.1 255.255.255.252

service cgn cgn service-type nat44

!

interface ServiceApp2

description ASVI for OutsideUserNAT

ipv4 address 2.1.1.1 255.255.255.252

service cgn cgn service-type nat44

!

interface ServiceApp3

description ASVI for InsideOfficeNAT

vrf InsideOfficeNAT

ipv4 address 3.1.1.1 255.255.255.252

service cgn cgn service-type nat44

!

interface ServiceApp4

description ASVI for OutsideOfficeNAT

ipv4 address 4.1.1.1 255.255.255.252

service cgn cgn service-type nat44

!

interface ServiceInfra1

ipv4 address 100.10.10.1 255.255.255.252

service-location 0/0/CPU0

!

router static

address-family ipv4 unicast

  100.62.16.0/22 Null0 210

  100.62.16.0/24 ServiceApp2

  100.62.17.0/24 ServiceApp4

!

vrf InsideUserNAT

  address-family ipv4 unicast

   0.0.0.0/0 ServiceApp1

  !

!

vrf InsideOfficeNAT

  address-family ipv4 unicast

   0.0.0.0/0 ServiceApp3

  !

!

!

service cgn cgn

service-location preferred-active 0/0/CPU0

service-type nat44 nat1

  inside-vrf InsideUserNAT

   map address-pool 100.62.16.0/24

  !

  inside-vrf InsideWifiNAT

   map address-pool 100.62.17.0/24

  !

  inside-vrf InsideOfficeNAT

   map address-pool 100.62.18.0/24

  !

!

!

end

What do you think?

And the weird thing is every configuration changes in "service cgn" the ism kind a stuck and need to reload in order to commit the changes.

[Nicolas Fevrier]

Hi,

first the card stuck after config is everything but normal, you should open a TAC case for this point.

Probably you need to verify you have the very latest FPD on the router cards too

(sh hw-module fpd location xxx)

If you do a "show platform", make sure the card is in IOS XR RUN for CPU0 and in APP-READY for CPU1.

I've checked the config and it looks ok to me.

I don't see any port-limit config though...

And in the show cgn nat44 x stat output:

RP/0/RSP1/CPU0:BGP-NAT#sh cgn nat44 natsitra statistics

Number of active translations: 100 <--- currently translating 100 udp or tcp or icmp sessions

Inside to outside forward rate: 6 <--- 6 pps in i2o, translation is working

Outside to inside forward rate: 0 <--- nothing received though

Inside to outside drops port limit exceeded: 5144 <---- drops because of exceeding the port limit, by default 100

Pool address totally free: 767

Pool address used: 1 <---- a single address is used in the pool for the 100 translations

Pool address usage:

-------------------------------------------------

  External Address       Ports Used

-------------------------------------------------

  202.62.16.126          100 <---- currently you reached the port limit, all new sessions, including your ping, will be discarded

-------------------------------------------------

!

If you want to check who is consuming the ports,

show cgn nat44 natsitra outside-translation protocol udp outside-vrf default outside-address 202.62.16.126 port start 1 end 65535

show cgn nat44 natsitra outside-translation protocol tcp outside-vrf default outside-address 202.62.16.126 port start 1 end 65535

Please configure an higher port-limit globally or in your inside-VRF:

service cgn cgnsitra

service-location preferred-active 0/0/CPU0

service-type nat44 natsitra

  portlimit 2048

  inside-vrf InsideUserNAT

...

Last, if the traffic is not coming back, you need to check that your upstream peer/transit are aware of these prefixes...

Cheers,

N.

[ingot.marulam]

Hi Nicolas,

Thanks for the explanation, I really appreciate that. Right now the host could reach the internet and I set the port limit to 65535. And I dont think that we have problem with advertisement to the upstream.

But I experienced another problem when I change the vrf to InsideWifiNAT or InsideOfficeNAT. It didn't translating and the host couldn't reach internet. When I turned back the vrf into InsideUserNAT, it went well.

The configurations:

!

service cgn cgn

service-location preferred-active 0/0/CPU0

service-type nat44 nat1

  portlimit 65535

  inside-vrf InsideUserNAT

   map address-pool 100.62.16.0/24

  !

  inside-vrf InsideWifiNAT

   map address-pool 100.62.17.0/24

  !

  inside-vrf InsideOfficeNAT

   map address-pool 100.62.18.0/24

!

router static

address-family ipv4 unicast

  100.62.16.0/22 Null0 210

  100.62.16.0/24 ServiceApp2

  100.62.18.0/24 ServiceApp4

  100.62.17.0/24 ServiceApp6

!

vrf InsideUserNAT

  address-family ipv4 unicast

   0.0.0.0/0 ServiceApp1

  !

!

vrf InsideOfficeNAT

  address-family ipv4 unicast

   0.0.0.0/0 ServiceApp3

  !

vrf InsideWifiNAT

  address-family ipv4 unicast

   0.0.0.0/0 ServiceApp5

!

Do you have suggestion?

[Harold Ritter]

Can you try increasing the port limit for the other VRFs as well and see if it solves the issue.

Regards

[Nicolas Fevrier]

Normally, the way it's configured, the port limit will be the same for all inside-VRFs.

So, I doubt the new symptoms you describe are linked to port limit this time.

You are moving inside interface (gi 0/1/1/0) from VRF InsideUserNAT to VRF InsideOfficeNAT, that means you remove IP address, remove the VRF config on the interface, commit, reassign the port to the new vrf and re-apply the IP address config and commit again. Right ?

In that case, check what's going on in the translation table while you generate traffic from your host to the internet and make sure an address from range 202.62.17.0/24 is allocated.

BR,

N.

[ingot.marulam]

@Nicolas:

The way I change the vrf is by assign different port to each InsideOfficeNat, InsideWifiNat and InsideUserNat. When testing each vrf I move the physical connection. When I was experienced problem using InsideOfficeNat, the Inside to Outside Packet result in  "show cgn nat44 nat1 outside-translation protocol udp outside-vrf  default outside-address 100.62.16.126 port start 1 end 65535" is counting but not for Outside to Inside Packets. And same symptom with InsideWifiNat.

@Nicolas and Harold:

Actually right now all vrf-inside successful doing translation and we also change the CGN configuration without vrf on physical inside interface. But we found the browsing experiences are very slow and some contents were not loaded successfully. Any idea what happened? 

Here the configuration:

!

ipv4 access-list inside-nat-abf-test

10 permit ipv4 10.3.15.0/24 any nexthop1 vrf InsideOfficeNAT ipv4 3.1.1.2

20 permit ipv4 10.1.9.0/24 any nexthop1 vrf InsideUserNAT ipv4 1.1.1.2

30 permit ipv4 10.5.5.0/24 any nexthop1 vrf InsideWifiNAT ipv4 5.1.1.2

!

interface GigabitEthernet0/1/1/0

description NAT Test 1

ipv4 address 10.1.9.129 255.255.255.0

transceiver permit pid all

ipv4 access-group inside-nat-abf-test ingress

!

interface GigabitEthernet0/1/1/1

description NAT Test 2

ipv4 address 10.3.15.1 255.255.255.0

transceiver permit pid all

ipv4 access-group inside-nat-abf-test ingress

!

interface GigabitEthernet0/1/1/2

description NAT Test 3

ipv4 address 10.5.5.1 255.255.255.0

transceiver permit pid all

ipv4 access-group inside-nat-abf-test ingress

!

!

interface ServiceApp1

description ASVI for InsideUserNAT

vrf InsideUserNAT

ipv4 address 1.1.1.1 255.255.255.252

service cgn cgn1 service-type nat44

!

interface ServiceApp2

description ASVI for OutsideUserNAT

ipv4 address 2.1.1.1 255.255.255.252

service cgn cgn1 service-type nat44

!

interface ServiceApp3

description ASVI for InsideOfficeNAT

vrf InsideOfficeNAT

ipv4 address 3.1.1.1 255.255.255.252

service cgn cgn1 service-type nat44

!

interface ServiceApp4

description ASVI for OutsideOfficeNAT

ipv4 address 4.1.1.1 255.255.255.252

service cgn cgn1 service-type nat44

!

interface ServiceApp5

description ASVI for InsideWifiNAT

vrf InsideWifiNAT

ipv4 address 5.1.1.1 255.255.255.252

service cgn cgn service-type nat44

!

interface ServiceApp6

description ASVI for OutsideWifiNAT

ipv4 address 6.1.1.1 255.255.255.252

service cgn cgn service-type nat44

!

router static

address-family ipv4 unicast

  100.62.16.0/22 Null0 210

  100.62.16.0/24 ServiceApp2

  100.62.17.0/24 ServiceApp6

  100.62.18.0/24 ServiceApp4

!

vrf InsideUserNAT

  address-family ipv4 unicast

   0.0.0.0/0 ServiceApp1

   10.1.9.0/24 vrf default GigabitEthernet0/1/1/0 10.1.9.130

  !

!

vrf InsideWifiNAT

  address-family ipv4 unicast

   0.0.0.0/0 ServiceApp5

   10.5.5.0/24 vrf default GigabitEthernet0/1/1/2 10.5.5.2

  !

!

vrf InsideOfficeNAT

  address-family ipv4 unicast

   0.0.0.0/0 ServiceApp3

   10.3.15.0/24 vrf default GigabitEthernet0/1/1/1 10.3.15.2

  !

!

service cgn cgn

service-location preferred-active 0/0/CPU0

service-type nat44 nat

  inside-vrf InsideUserNAT

   map outsideServiceApp ServiceApp2 address-pool 100.62.16.0/24

   protocol tcp

    mss 1400

   !

   portlimit 65535

   o2i-vrf-override default

  !

  inside-vrf InsideWifiNAT

   map outsideServiceApp ServiceApp6 address-pool 100.62.17.0/24

   protocol tcp

    mss 1400

   !

   portlimit 65535

   o2i-vrf-override default

  !

  inside-vrf InsideOfficeNAT

   map outsideServiceApp ServiceApp4 address-pool 100.62.18.0/24

   protocol tcp

    mss 1400

   !

   portlimit 65535

   o2i-vrf-override default

  !

!

!

end

[Nicolas Fevrier]

Nice to see you made it works fine with ABF

Not sure what you are trying to do with the MSS config, did you try withouth it ?

Also, give it a try without the routes to the three uplinks but just one.

Cheers,

N.

[ingot.marulam]

Hi Nicholas,

Yes my colleagues found about ABF and configure it well . So the ABF configuration looks good, right? Is there any suggestion from you that must be configured to improve the performance?

The reason we configured the mss were tried to fix the slow browsing experiences. But any way, I will try to change the configuration without mss.

Cheers,

Ingot