Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 

CGv6 on ISM: CGN / NAT44 Deployment Guide

Introduction

Cisco's CGv6 (Carrier Grade IPv6) solution helps customer transitioning from IPv4 to IPv6 gradually, maintaining business continuity. ISM (Integrated Services Module) on ASR9K is a Service Card which provides different CGv6 Applications. This document talks about how to deploy NAT44 (Network Address Translation - from IPv4 to IPv4), one of the applications fall under CGv6 suite of applications.

This document does NOT intend to cover details on ISM, how to install CGv6 software on ISM, how to trouble-shoot CGv6 issues, CGv6 Applications other than NAT44.

 

DISCLAIMER: This document contains some Public IP addresses to explain different deployment use-cases and features related to NAT44. The Public IP addresses are picked up randomly and matching of these addresses with any specific customer's or organization's public IP is purely coincidental. Please DO NOT use these public addresses in your NAT setup / deployment, unless the public IP addresses are not owned by you / your organization. Cisco is not responsible for any issues arising out of this.

Introduction to NAT44

This section introduces NAT44 and terminologies related to it.

1. Why NAT44 ?

IPv4 address is 32-bit long which means there can be 4,294,967,296 (i.e., 4 billions) unique addresses. However, population on earth has crossed 7 billions and is still growing. Also, nowadays, a single person can own multiple IP devices, each having its own IP address. Thus, we are running short of IPv4 addresses. In 2 of the 5 RIR (Regional Internet Registries), IPv4 address exhaustion is declared (in its last /8 stage).

Apat from moving to IPv6 (which uses 128-bit long IP address), an alternate mechanism is to use NAT44 where Private IP addresses can be re-used in different Private networks, but can be translated to a unique Public IP address when the packet enters Internet or Public network.

If you are a Service Provider and running short of Public IP addresses, NAT44 should be of great usefulness to you as an intermediate solution before you slowly convert your IPv4 network into IPv6.

2. NAT44 Overview

NAT44 literally means translating one IPv4 address to another IPv4 address. "Typically", first IPv4 address is "Private" IPv4 Address (defined by RFC 1918) and second IPv4 address is "Public" (non-Private) IPv4 address. As defined by RFC 1918, Private IPv4 addresses fall in the following blocks.

Start IP AddressEnd IP AddressComments
10.0.0.010.255.255.25510.0.0.0/8 prefix - contains 16,777,216 addresses
172.16.0.0172.31.255.255172.16.0.0/12 prefix - contains 1,048,576 addresses
192.168.0.0192.168.255.255192.168.0.0/16 prefix - contains 65,536 address

However, it is to be noted that not always "Private" IPv4 address is to be translated to "Public" IPv4 address. There are deployment cases where NAT44 is used to convert "Private" IPv4 address (say, 192.168.x.x range) to another "Private" IPv4 address (say, 10.x.x.x range) as well.

NOTE: In this document, however, we will refer "Private" side as the pre-NAT side of the NAT device and "Public" side as the post-NAT side of the NAT device, irrespective of the actual IP address being used.

 

One more important point, although NAT indicates Network Address Translation, it is actually NAPT (Network Address and Port Translation). What it means is - in addition to IPv4 address, Layer 4 (UDP/TCP) port is also translated by NAT44.

NOTE: Unless mentioned otherwise, in this document, NAT44 will indicate translation of both IPv4 address and Port.

 

Following Layer 4 Protocols are supported by NAT44 implementation on ISM.

  • UDP (IP Protocol 17) - UDP Port number (2 bytes) is translated along with IPv4 address.
  • TCP (IP Protocol 6) - TCP Port number (2 bytes) is translated along with IPv4 address.
  • ICMP (IP Protocol 1) - ICMP Identifier field (2 bytes) in Echo Request and Echo Response messages is used as (dummy) Layer 4 port.
  • GRE (used with PPTP ALG, IP Protocol 47) - Call ID field (2 bytes) is used as (dummy) Layer 4 port.

Another useful property of NAT44 is multiple Private IP addresses can be mapped to one Public IP address. This is possible as all of the 65,536 ports (0 to 65,535) on any Private IP address are typically not used by Applications.

Following diagram shows how NAT44 works.

NAT44.png

As shown above, the Private IP addresses 192.168.1.1 and 192.168.1.5 are translated to same Public IP address 20.1.1.1 (along with specific Layer 4 ports) by the NAT44 device (ISM on ASR9K). The translation entries are maintained in NAT DB at the NAT44 device.

3. Basic Terminologies

In this section, let us get introduced with different terminologies related to NAT44 deployment on ISM.

TerminologyDefinition / What it means
Inside / Subscriber Side / Access Side / Private SidePrivate side of the NAT44 device (which usually contains Private IPv4 addresses)
Outside / Network Side / Core SidePublic side of the NAT44 device (which usually contains Public IPv4 addresses)
Inside-to-Outside (I2O)Traffic flowing from Inside to Outside across NAT44 device
Outside-to-Inside (O2I)Traffic flowing from Outside to Inside across NAT44 device
Stateful (SF)Which preserves State or Database. E.g. - NAT44 is Stateful protocol.
Endpoint Independent Mapping (EIM) / Filtering (EIF)

Any device (IP + Port) can send O2I traffic to a Public IP + Port which exists in NAT DB. ISM implements this behavior. It is also termed as "Full-Cone NAT".

Endpoint Dependent Mapping (EDM) / Filtering (EDF)

Only the device (IP + Port) which has received I2O traffic can send O2I  traffic to a public IP + Port which exists in NAT DB. Destination IP and  port information is also maintained in NAT DB in this case. ISM does  not implement this behavior.

Port Parity Preservation

Odd port should be translated to Odd port and Even port should be translated to Even port. ISM supports this.

4. Standard Compliance

Following are the important RFCs that the NAT44 solution on ISM complies (most of the cases, fully) to.

RFC Number / IETF Draft NameTitle of the SpecificationAdditional Comments
RFC 768User Datagram Protocol (UDP) 
RFC 792Internet Control Message Protocol (ICMP) 
RFC 793Transmission Control Protocol (TCP) 
RFC 959File Transfer Protocol (FTP) 
RFC 1812Requirements for IP Version 4 Routers 
RFC 3954Cisco System Netflow Services Export Version 9 
RFC 4787Network Address Translation (NAT) Behavioral Requirements for Unicast UDP 
RFC 5382NAT Behavioral Requirements for TCP 
RFC 5424The Syslog Protocol 
RFC 5508NAT Behavioral Requirements for ICMP 
RFC 6888Common Requirements for Carrier Grade NATs (CGN)Evolved from draft-nishitani-cgn

Basic Deployment Scenario: NAT444

This section talks about one of the basic deployment scenario for NAT44 which is called "Double NAT44" or "NAT444".

5. Network Diagram

Following diagram depicts NAT444.

Dual_NAT44.png

In the above diagram, there are 2 NAT44 instances shown (hence, it is termed as "Dual NAT44" or "NAT444").

  • CPE (Customer Premise Equipment) NAT44 - This typically happens within customer premise. Each customer / end user is given one (or, more) IPv4 address (10.8.1.11 in the above example) by the Service Provider. There could be multiple devices within customer premise which will have IPv4 addresses in 192.168.1.x network (as shown in the above diagram). As an example, CPE NAT44 device translates IPv4 address/Port of 192.168.1.1:4128 to 10.8.1.11:6593, while sending the packet out to provider network. A NAT table is maintained at CPE NAT level as well. But, the scale of this is less (as there won't be too many end devices). As you can see, both Private-side and Public-side IP addresses used here are from RFC 1918 (i.e., Private IPv4 addresses).
  • CGN (Carrier Grade NAT) or LSN (Large Scale NAT) - This happens within the Service Provider network. CGN device (as per the diagram) translates Private IPv4 address/Port of 10.8.1.11:6593 to a public IPv4 address/Port of 170.0.0.1:9185 before sending the packet to Internet. A relatively huge NAT table is maintained at CGN level as it may need to deal with millions of CPEs.

ISM with ASR9K will be positioned as CGN in Service Provider network.

6. NAT44 on ISM Configuration

Before we start configuring NAT44 on ISM, let us understand the following concepts related to NAT44 implementation on ISM.

  • CGN Instance - On each ISM card, we can configure one CGN instance (<cgn1> in the diagram below).
  • NAT44 Instance - On each ISM card, we can configure one NAT44 instance (<nat1> in the diagram below).
  • ServiceInfra interface - ServiceInfra interface is a virtual interface which is used by the internal (to ASR9K) management / control traffic. We need 1 ServiceInfra interface per ISM card. ServiceInfra interface cannot be in any VRF (needs to be in default VRF).
  • ServiceApp interface - ServiceApp interface is a virtual interface which is used to bring CGv6 Application (NAT44) data traffic to ISM card. We need 1 pair of ServiceApp interface for each VRF instance - 1 Inside ServiceApp interface (ServiceApp <x> in the diagram below) to bring Inside traffic to ISM (I2O direction) and 1 Outside ServiceApp interface (ServiceApp <y> in the diagram below) to bring Outside traffic to ISM (O2I direction).
NOTE: ServiceInfra and ServiceApp interfaces are used within the ASR9K router only and do not need to be announced/redistributed into IGP.
  • Inside VRF instance - Within one NAT44 instance, we can have multiple Inside VRF instances. For each Inside VRF instance, we can have different NAT44 parameters configured. Inside ServiceApp must be within one unique Inside VRF. Outside ServiceApp may or may not be within a (Outside) VRF.
NOTEs:
- We cannot have multiple Inside ServiceApps within same Inside VRF. 
- We can, however, have multiple Outside ServiceApps within same Outside VRF or default VRF.
- Inside and Outside VRF cannot be same.
  • Logical Partitioning inside ISM - Internally, ISM is divided into 2 Logic Partitions. Hence, in order to achive full throughput and scale limit (per ISM), you should have minimal of 2 Inside VRFs, 4 ServiceApp interfaces (2 in each partition), 2 Public IP address pool, etc. Traffic / subscribers which needs to avail NAT44 service also needs to be divided into 2 groups (each one associated to each Inside VRF). Following table shows the partitioning of ServiceApp interfaces between these 2 groups (which needs to be kept in mind during configuration).
Logical Partition  / Group 1Logical Partition / Group 2
ServiceApp 1, ServiceApp 2ServiceApp 3, ServicApp 4
ServiceApp 5, ServiceApp 6ServiceApp 7, ServiceApp 8
ServiceApp 9, ServicApp 10ServiceApp 11, ServiceApp 12
......
ServiceApp 241, ServiceApp 242ServiceApp 243, ServiceApp 244

 

NOTE: ServiceApp interafce pair needs to be selected from same Logical Partition (as per the above table). Otherwise, packets will be dropped. Using consecutive numbers for ServiceApp interface pair will be recommended (as shown above).

Following diagram provides summary / overview of NAT44 configuration for ISM.

NAT44_Config_Summary.png

Below are sample configurations for each of the above steps.

Step 1: Configure CGN Instance on ISM Card at Slot 1

 

! Define location where CGN will run

! ----------------------------------

hw-module service cgn location 0/1/cpu0

Step 2: Configure ServiceInfra interface

 

! Define Service Infra interface

! ------------------------------

interface ServiceInfra 1

  ! Assign IPv4 address and netmask

  ipv4 address 100.1.1.1 255.255.255.252

  ! Define service location

  service-location 0/1/CPU0

Step 3 and 4: Configure Ingress and Egress LC Interfaces

 

! Configure Ingress interfaces

! ----------------------------

interface TenGigE0/0/0/0

  ! Add it to Inside VRF

  vrf ivrf1

    ! Assign IP address to it

    ipv4 address 10.10.10.1 255.255.255.0


 

interface TenGigE0/0/0/1

vrf ivrf2
ipv4 address 10.10.11.1 255.255.255.0 ! Configure Egress interface
! --------------------------

interface TenGigE0/2/0/0

  vrf ovrf1

    ipv4 address 20.20.20.1 255.255.255.0

interface TenGigE0/2/0/1

vrf ovrf2

ipv4 address 20.20.21.1 255.255.255.0

Step 5: Configure CGN and NAT44 Service parameters (Important ones are shown)

 

! Define CGN Instance

! -------------------

service cgn cgn1

! Define location

  service-location preferred-active 0/1/CPU0

  ! Define NAT44 Service Type and instance (one per ISM card)

  service-type nat44 nat1

  ! Define per-user Port limit for NAT44 instance

  portlimit 250

  ! Enable Active FTP ALG

  alg ActiveFTP

  ! Define UDP protocol session timeout values

  protocol udp

    session initial timeout 240

    session active timeout 600

  ! Define inside VRF

  inside-vrf ivrf1

    ! Define Public IP address Pool

    map outside-vrf ovrf1 address-pool 100.1.1.0/24

  ! Define inside VRF

  inside-vrf ivrf2

    ! Define Public IP address Pool

    map outside-vrf ovrf2 address-pool 100.1.2.0/24

Step 6 and 7: Configure Inside and Outside ServiceApp interfaces

 

! Define Inside Service App interfaces

! ------------------------------------

interface ServiceApp 1

  ! Define VRF where it should belong to (optional)

  vrf ivrf1

    ! Assign IPv4 address and netmask

    ipv4 address 1.1.1.1/30

    ! Define CGN instance and service type in which it belongs to

    service cgn cgn1 service-type nat44

interface ServiceApp 3

  vrf ivrf2

    ipv4 address 1.1.2.1/30

    service cgn cgn1 service-type nat44

! Define Outside Service App interfaces

! -------------------------------------

interface ServiceApp 2

  vrf ovrf1

    ipv4 address 2.1.1.1/30

    service cgn cgn1 service-type nat44

interface ServiceApp 4

  vrf ovrf2

    ipv4 address 2.1.2.1/30

    service cgn cgn1 service-type nat44

Step 8 and 9: Configure Traffic Diversion Rules

 

! Define static route for Inside-to-outside traffic

! -------------------------------------------------

router static

 

  ! Specify the Inside VRF

  vrf ivrf1

 

    ! Define address family

    address-family ipv4 unicast

 

      ! Re-direct all traffic to Inside Service App

      0.0.0.0/0 ServiceApp 1

router static

  vrf ivrf2

    address-family ipv4 unicast

      0.0.0.0/0 ServiceApp 3

! Define static route for Outside-to-inside traffic

! -------------------------------------------------

router static

  vrf ovrf1

    address-family ipv4 unicast

      100.1.1.0/24 ServiceApp 2

router static

  vrf ovrf2

    address-family ipv4 unicast

      100.1.2.0/24 ServiceApp 4

7. I2O Packet Flow on ISM

Following diagram shows the packet flow within ASR9K (including ISM) for Inside to Outside NAT44 traffic at a high-level.

I2O_Traffic_Flow.png

Step 1: Packet from Inside / Private side reaches ASR9K Ingress Line Card.

Step 2: Forwarding lookup (rule added by static route, ACL-based forwarding, etc.) should forward the packet to ISM card, via Inside ServiceApp 1 interface. Inside VRF (ivrf1) will be used during the forwarding lookup.

Step 3: NAT44 Application performs necessary action for the packet. As it is Inside-to-Outside packet, it first checks if there is a NAT entry exists for the source IP (10.10.10.2) and port (5000). If it does not exists, it allocates a Public IP (from the Public IP pool configured) and port and creates a NAT entry. From the NAT entry, it finds the corresponding Public IP (100.2.0.192) and port (23156) and replaces the existing Source IP and port with the same.

Step 4: After NAT translation is done, a forwarding lookup is done in Outside VRF (ovrf1) for the destination IP address (5.5.5.2) which forwards the packet to egress LC interface.

Step 5: Egress Line Card interface sends the packet out to Outside / Public side.

8. O2I Packet Flow on ISM

Following diagram shows the packet flow within ASR9K (including ISM) for Outside to Inside NAT44 traffic at a high-level.

O2I_Traffic_Flow.png

Step 1: Packet from Outside / Public side reaches ASR9K Ingress Line Card.

Step 2: Forwarding lookup (rule added by static route, ACL-based forwarding, etc.) should forward the packet to ISM card, via Outside ServiceApp 2 interface. Outside VRF (ovrf1), if being configured, will be used during the forwarding lookup.

Step 3: NAT44 Application performs necessary action for the packet. As it is Outside-to-Inside packet, it first checks if there is a NAT entry exists for the Destination IP (100.2.0.192) and port (23156). If it does not exists, packet is dropped. Otherwise, from the NAT entry, it finds the corresponding Private IP (10.10.10.2) and port (5000) and replaces the existing Destination IP and port with the same.

Step 4: After (reverse) NAT translation is done, a forwarding lookup is done in Inside VRF (ivrf1) for the destination IP address (10.10.10.2) which forwards the packet to egress LC interface.

Step 5: Egress Line Card interface sends the packet out to Inside / Private side.

Parameters to be considered before Deployment

Before deploying NAT44 solution with ISM, you need to analyze the deployment scenario/requirement and come up with the appropriate values of different configurable parameters related to NAT44. Improper selection of the parameters may not address the deployment goals accurately.

9. Performance and Scale

These are possibly the most important parameters to be considered during deployment. These are not configurable parameters, but these will determine how many ISM cards will be needed for the NAT44 solution.

  • Identify the performance and scale need for NAT44 solution
  • Please check "CGv6 on ISM Feature Support" page, to find out the supported scale and performance values.

Based on the above two, find out how many ISM cards will be needed for the solution.

10. Address Pool and Mapping

Identify the following for the deployment scenario.

  • How many Public IPv4 addresses the customer has to form the Public IP pool ?
  • How many (approximately) Private (Pre-NAT) IPv4 addresses the customer has ?
  • Do you need to limit the ratio of Private to Public IP addresses ?

Based on the same, number of inside VRFs, address pools, many-to-one ratio can be configured as follows.

Address Mapping related parameter configuration

 

service cgn cgn1

  service-type nat44 nat1

 

 

    inside-vrf ivrf1

 

 

      ! Define Public IP address Pool

      map outside-vrf ovrf1 address-pool 100.1.1.0/20

      ! Define Many to one IP addresses maping

      map ip many-to-one 6


 

NOTE: Network address and Broadcast address in the subnet (as configuredby address-pool parameter) are also used as NAT'ed Public / Outside IP address.

11. VRF (Inside and Outside)

Identify the following VRF related information from deployment requirement.

  • Whether Private side traffic comes / can / should come in VRF or not ?
  • Whenther Public side traffic can / should go in a VRF or not ?
  • Is there a overlap of Private IP addresses or not ?
  • Are there different values of certain parameters (which are configured per-VRF) needed or not ?
  • What is the Scale and Bandwidth requirement for each VRF

Above information will determine,

  • How many Inside and Outside VRFs will be needed in the solution
  • Which per-Inside VRF parameters need to be set
  • How the Public IP pool needs to be divided

12. Port Limit (Per-User)

As NAT DB is shared among all users, there could be a situation where few users (Private IP owners) uses lots of NAT entries (in thousands) and because of the same, new users cannot create NAT entries. In order to keep control on creation of NAT entries by a single user, Port Limit parameter is introduced. Port limit indicates how many port numbers are allowed per Private / Inside IP address. If any flow arrives after Port Limit is reached, no NAT entry will be created for the same and the packets for the new flows will be dropped. The Port limit value will be same for all Private IP addresses.


 

NOTE: Port Limit is specified per Private / Inside IP address and NOT per Public / Outside IP address.

Find out the following:

  • How many ports will be needed by a Private IP address holder ?
  • How many Public IP addresses are available ?

Based on the above Port Limit can be configured as follows.

Configuration of Port Limit

 

service cgn cgn1

  ! Define NAT44 Service Type and instance (one per ISM card)

  service-type nat44 nat1

    ! Define per-user Port limit for NAT44 instance

    portlimit 250

13. External Logging (Netflow and Syslog)

As you have seen above, a NAT DB entry is created for each new flow (Private IP address and port) which keeps the mapping of Private / Inside IP address + Port and Public / Outside IP address + Port. The NAT DB entry is deleted when the flow terminates (because of timeout and any user intervention).

On ISM, we support millions of NAT translations with high (in thousands and millions) setup and deletion rate (Please refer to CGv6 on ISM Feature List for scale numbers). This produces huge amount of NAT translation entries / records which cannot be stored / saved on ISM or ASR9K because of limited disk space. Hence, there is a need of having external servers where these translations can be saved.

The creation and deletion of NAT transactions (also called "Logging Records") are very important to be saved to get association of different IP addresses. In almost all countries, it is mandated by the Lawful Agencies / Authorities that the Service Provide must provide these records accurately so that the agencies can find out the owner of these APIs for different reasons. Thus it becomes a very important feature for deployment of NAT44.

On ISM, we support the 2 formats of logging records - Netflow (version 9) and Syslog. Table below compares the two.

Items for ComparisonNetflow Version 9 (NFv9)Syslog
Standard DocumentRFC 3954RFC 5424
Text / Binary formatBinaryText
Underlying Layer 4 protocolUDP (Server usually uses Port 2055, 2056, 4432, 4739, 9995, 9996, 6343)UDP (Server typically uses Port 514)
Average Logging Recod Size20 Bytes (with DBL and BPA disabled)70 Bytes (with DBL and BPA disabled)
Usage of TemplateUses Netflow TemplateDoes not use any template

Logging records are sent to Logging Server (IP address and Port for which is configurable) either when the Logging packet is complete or timeout occurs. Logging packets are sent with ServiceInfra interface's remote IP (for e.g., 100.1.1.2 is the remote IP of 100.1.1.1) as Source address.

NOTE: External Logging Server IP address should be reachable via any ASR9K LC interface (NOT via Management Ethernet interface), in default VRF.

Destination-Based Logging (DBL)

In some countries, Lawful Agencies also require the Service Provider to provide the Destination IP address and Port (along with Source IP address and Port information) as part of Logging Record. This is termed as Destination-Based Logging (DBL).

NOTE: Including Destination information in the Logging Record will increase the size of the Record. 

DBL can be supported with Netflow v9 as well as with Syslog.

Bulk Port Allocation (BPA)

In order to reduce the size of the logging record, a feature called Bulk Port Allocation (BPA) is introduced.

Without BPA, for every new flow, a Port is allocated for the Public IP and a new logging record is created with Private IP + Port and Public IP + Port. Similarly, another Logging record is created when a flow or NAT entry is deleted.

Following table captures the working of it (without BPA) with some sample IP addresses and port numbers.

EventInside Source IP + PortOutside Source IP + PortLogging RecordAdditional Comments
A new flow arrives10.1.1.1:1000170.0.0.5:817510.1.1.1:1000 <-> 170.0.0.5:8175

New port (8175) allocation happens.

New logging record is created

Existing flow arrives10.1.1.1:1000170.0.0.5:8175N/A

No new Port allocation happens.

No logging record is created

A new flow arrives (with new port, but same IP)10.1.1.1:2000170.0.0.5:1359110.1.1.1:2000 <-> 170.0.0.5:13591

New port (13591) allocation happens.

New logging record is created.

Without BPA, for the first new flow, a bulk (group) of Ports are allocated (but only one port from the group is assigned for the new flow) for the Public IP and a logging record is created with Private IP and Public IP + Start Port + End Port. Next time, a new flow comes, a new port from the already allocated pool is taken and no logging record is created. Only when all the ports in the group are used, a new bulk is allocated and logging record is created. Similarly, when all the ports from a group is released, a (deletion) logging record is generated.

Following table captures the working of it (with BPA enabled) with sample IP addresses and port numbers.

EventInside Source + PortBulk Port Allocation (size 64)Outside Source IP + PortLogging RecordAdditional Comments
A new flow arrives10.1.1.1:1000Allocates Bulk Port range (8192 - 8255) for 170.0.0.5170.0.0.5:820110.1.1.1 <-> 170.0.0.5:8192-8255

Bulk port allocation and new port (8201) assignment happens.

New logging record is created.

Existing flow arrives10.1.1.1:1000N/A170.0.0.5:8201N/A

No new Port allocation happens.

No logging record is created.

A new flow arrives (with new port, but same IP)10.1.1.1:2000N/A170.0.0.5:8199N/A

New Port (8199) allocation happens (from same port range).

No logging record is created.

A new flow arrives (with new port, but same IP)10.1.1.1:5000N/A170.0.0.5:8236N/A

New Port (8236) allocation happens (from same port range).

No logging record is created.

Following table illustrates the amount stoage space needed per second with varying translation creation + deletion rate for both Netflow and Syslog (with DBL disabled). It also shows the impact of BPA on the storage.

NAT translation creation + deletion rate per secondNetflow without BPASyslog without BPANetflow with Bulk size 256 and 50% utilizationSyslog with Bulk size 256 and 50% utilization
20,0000.8 MB2.8 MB6.25 KB21.875 KB
50,0002 MB7 MB15.625 KB54.6875 KB
100,0004 MB14 MB31.25 KB109.375 KB
500,000 (maximum)20 MB70 MB156.25 KB546.875 KB

 

NOTE: Both DBL and BPA cannot be enabled at the same  time. This is because BPA requires logging record to be generated for a  set of NAT entries whereas DBL requires logging record to be generated  for each NAT entry. These are conflicting requirements.

Please collect the following information and accordingly determine External Logging related parameter configuration.

  • What setup rate needs to be supported ?
  • Do the Lawful Agencies mandate DBL ?
  • What protocol (netflow or syslog) the external logging server support ?
  • How many packets per second the logging server can support ?
  • How many logging servers need to be configured ?
  • How much space would be needed for storing logging record ? How much would the cost of disk space ?

Following table shows how to configure different external logging parameters.

Configuration of External Logging Parameters

 

service cgn cgn1

  service-type nat44 nat1

    inside-vrf ivrf1

 

      ! Define Netflow V9 logging configuration

      external-logging netflow version 9

        ! Define netflow server configuration

        server

          ! Define logging server address and port

          address 10.10.0.0 port 2055

          ! Define Path MTU size

          path-mtu 1200

          ! Define refresh rate

          refresh-rate 50

          ! Define timeout

          timeout 50

      ! Define Syslog logging configuration

      external-logging syslog

        server

          address 10.10.0.1 port 514

14. NAT Session Timeouts (TCP, UDP, ICMP)

As the new flows creates NAT entries, we need a way to delete those NAT entries as well. Protocols like UDP is connection-less and hence, there is no way to identify when a session / connection has ended by inspecting the packets. Also, in some cases, if no packet flows between the applications for a long period of time, there is no point in holding on to the NAT entries, as that will disallow creation of new NAT sessions.

Due the above reasons, NAT entries are deleted when session timeout happens.

For every protocol (TCP, UDP, ICMP), we have different types and values of session timeouts, as depicted below.

  • TCP - It has two timeouts.                                                               
    • Initial Timeout - It kicks in before the connection moves into Active state. Default: 120 sec.
    • Active Timeout - It kicks in when the connection is in Active state. Default: 30 Minutes.
  • UDP - It has two timeouts as well.                                                               
    • Initial Timeout - It kicks in before the connection moves into Active state. Default: 30 sec.
    • Active Timeout - It kicks in when the connection is in Active state. Default: 120 sec.
  • ICMP - It has one timeout. Default: 60 sec.

If the timeout is configured to be too less, NAT sessions will be deleted too soon, resulting in changing of Public IP addresses. This may cause issues with the Applications.

If the timeout is configured to be too high, NAT sessions will remain in the system for long time (even though there is no data is flowing using the session) and it will restrict new sessions to be created as well (resulting in dropping of those packets).

Hence, timeout value needs to be set appropriately.

NOTE: These timeouts are set for the NAT44 instance (per ISM card) and cannot be configured differently for each Inside VRF.

You need to have some knowledge about the following, to determine the timeout values.

  • Application traffic that is going through the network                                                                
    • How sensitive it is with respect to change in IP address ?
    • How periodically, it would generate packets
  • How many NAT sessions are expected ?

Following table shows how to configure timeout values for different protocols.

Configuration of Session Timeout (TCP, UDP, ICMP)

 

service cgn cgn1

  service-type nat44 nat1

 

    ! Define UDP protocol session timeout values

    protocol udp

      session initial timeout 65535

      session active timeout 65535

    ! Define TCP protocol session timeout values

    protocol tcp

      session initial timeout 65535

      session active timeout 65535

    ! Define ICMP protocol session timeout value

    protocol icmp

      timeout 908

15. Traffic Diversion (to ISM)

We can use different mechanisms to divert the traffic to ISM card (to specific ServiceApp interface).

Default Route

If the inside traffic is arriving on a (Inside) VRF and ALL traffic needs to undergo NAT44 operation, we can add a default route under the VRF to divert traffic to the Inside ServiceApp (as shown below).

Configuration for Traffic Diversion using Default route

 

! Define static route for Inside-to-outside traffic

! -------------------------------------------------

router static

  ! Specify the Inside VRF

  vrf insidevrf1

    ! Define address family

    address-family ipv4 unicast

      ! Re-direct all traffic to Inside Service App

      0.0.0.0/0 serviceapp 1

Static Route

In some cases, Outside ServiceApp interfaces reside in same / default VRF. Hence, in order to divert O2I traffic to specific Outside ServiceApp interfaces, static route can be used in the following way.

Configuration for Traffic Diversion using Static Route

 

router static

 

  address-family ipv4 unicast

    ! Diverting traffic for one Public IP pool to one Outside ServiceApp

    100.1.1.0/24 serviceapp 2

    ! Diverting traffic for another Public IP pool to another Outside ServiceApp

    200.1.1.0/24 serviceapp 4

ACL-based Forwarding (ABF)

In certain deployment cases, default route may not work.

  • Deployment Use-case 1: Customer wants to divert traffic coming from specific subscriber network (containing Private IP address) to ISM (for NAT44) and the remaining traffic (which may already contain Public IP address) need not go to ISM.
  • Deployment Use-case 2: Customer wants to distribute subscriber traffic into different Inside VRFs which may go to same ISM card or different ISM cards (but, will certainly using different Public IP pool).
  • Deployment Use-case 3: Subscriber / Access / Inside traffic comes in default / global VRF and we need to distribute the traffic to different Inside ServiceApps which are in different Inside VRFs.
NOTE: Typically, ABF is to be used in Private side / Subscriber side and NOT towards Public / Core / Network side. 

 

In the above cases, ABF provides a very good solution. Below is sample configuration on how to use ABF for traffic diversion.

Configuration for Traffic Diversion using ABF

 

! Define IPv4 Access List

ipv4 access-list nat44_abf

  ! Divert to ServiceApp 1

  10 permit ipv4 10.0.100.0/24 any nexthop1 vrf ivrf1 ipv4 5.1.1.2

  ! Divert to ServiceApp 3
  20 permit ipv4 10.0.200.0/24 any nexthop1 vrf ivrf2 ipv4 6.1.1.2 100 permit ipv4 any any ! Configure Ingress Interface

interface TenGigE0/5/0/2

  vrf subscriber

    ipv4 address 10.1.1.1 255.255.255.0

    ! Apply ABF on Ingress Interface

   ipv4 access-group nat44_abf ingress ! Define one Inside ServiceApp interface
interface ServiceApp1

vrf ivrf1

    ipv4 address 5.1.1.1/30

    service cgn cgn1 service-type nat44

! Define another Inside ServiceApp interface
interface ServiceApp3   vrf ivrf3

    ipv4 address 6.1.1.1/30

    service cgn cgn1 service-type nat44

16. Static Port Forwarding

There is a very important point about Dynamic NAT44. All the NAT translations / sessions are created by I2O traffic. If O2I traffic reaches ISM for which there is no NAT entry, the packets will be dropped. This works fine for most of the cases as the session / traffic is initiated by Applications which reside on Private side.

However, there are cases, where a server resides on the Private side of the network. In this case, there are 2 issues.

  • Clients (on Public side) cannot initiate session to this Server, as those (O2I) packets will be dropped by ISM.
  • Clients do not know the Public IP of the Server (and cannot use Private IP to communicate either)

To address these problems related to Server sitting on Private side of the network, a new feature called "Static Port Forwarding" is introduced. Following diagram explains "Static Port Forwarding" feature functionality.

Static_Port_Fwd.png

Using Static Port Forwarding, user can provide the Private IP address and Port (via CLI) and NAT44 Application on ISM generates (dynamically) the corresponding Public IP and Port. Also, these mapping are made permanent - i.e., they do not get timed out and get deleted. The Public IP and port generated by the Application can be added to DNS database and thus made available to all.

With this design, any device on Public side of network can now send traffic to the known Public IP and port (to reach the Server sitting inside Private network). Also, as the NAT entry is permanent, O2I traffic always finds the entry and using the same can translate the packet (without dropping it).

Staic Port Forwarding entries can be created separately for each protocol - TCP, UDP and ICMP. To know the maximum number of such entries, please check CGv6 on ISM Feature page.


 

NOTES:

- User cannot provide the Outside / Public IP and port while configuring Static Port Forwarding.

- Outside / Public IP is generated by the NAT44 Application and hence, can be random.

- Outside port is usually assigned same as Inside Port, unless the Outside port is already being used

As these are used for Servers located in Private network, it is desirable that the Public IP address remain constant across RP / ISM failover.

As Outside / Public IP address for a Static Port Forwarding entry is allocated by NAT44 application based on the instant situation of Public IP availability stock as well as the allocation algorithm's state, it is highly probable that if the Static Port Forwarding entry is deleted and re-added or, if the ISM card is reloaded), it may allocate a different Public IP address.

In order to keep the same Outside IP address, it is recommended to add the Static Port Forwarding configuration as a first thing after ISM card comes up and when there is no NAT traffic on the router/ISM card (could be in a "maintenance window"). That way, every time, ISM is reloaded, the Public IP addresses assigned for Static Port Forwarding entries will remain the same.

If you are using ISM 1:1 redundancy, Backup ISM will assign the same Public IP as assigned in Active ISM (irrespective when the entries are added).

Try to find out the following, before using Static Port Forwarding.

  • Will there be any Servers sitting in the Private side of the deployment network ?
  • How many of those Servers will be ?
  • Can the Public IP address of the Servers be random and can be published via DNS or similar mechanism ?

A sample configuration for Static Port Forwarding is shown below.

Configuration for Static Port Forwarding

 

service cgn cgn1

  service type nat44 nat1

    inside-vrf ivrf1

    map address-pool 180.0.0.0/24

    ! Define TCP related configuration

    protocol tcp

      ! Define Static Port Forwarding configuration

      static-forward inside address 10.0.0.50 port 6050

      static-forward inside address 10.0.0.50 port 80

    ! Define UDP related configuration

    protocol udp

      ! Define Static Port Forwarding configuration

      static-forward inside address 10.0.0.61 port 550

    ! Define ICMP related configuration

    protocol icmp

      ! Define Static Port Forwarding configuration

      ! For ICMP, 'identifier' field in ICMP header is used for 'port'

      static-forward inside address 10.0.0.189 port 750

Following table shows the NAT entries corresponding to the above Static Port Forwarding entries (for TCP protocol).

"show cgn nat44 <> inside--translation" output for Static Port Forwarding
 

 

RP/0/RSP0/CPU0:VSM-Lab#show cgn nat44 nat44-1 inside-translation protocol tcp inside-vrf vrf1

                       inside-address 10.0.0.50 port start 25 end 8000

Fri Nov  1 13:44:44.874 UTC
Inside-translation details
---------------------------
NAT44 instance : nat44-1
Inside-VRF     : vrf1
--------------------------------------------------------------------------------------------
   Outside         Protocol  Inside       Outside       Translation   Inside      Outside
   Address                   Source       Source        Type          to          to
                             Port         Port                        Outside     Inside
                                                                      Packets     Packets
--------------------------------------------------------------------------------------------
  180.0.9.0          tcp       80           80            static       1218         45
  180.0.9.0          tcp     6050         6050            static        516       7162
 


 

NOTE:

- Outside IP (180.0.9.0) and Ports are assigned by the NAT44 Application when user supplies Inside IP (10.0.0.50) and Ports (80, 6025).

- "Translation Type" is shown as "static" (as opposed to "dynamic")

- Outside port is same as Inside port (usually, unless it is not available)

17. Application Layer Gateway (ALG)

Application Layer Gateway (ALG) is a device which has the capability of inspecting into a packet beyond layer 4 and can change Application layer content to meet certain need.

NAT44 changes IP address and Layer 4 (UDP/TCP/ICMP etc.) port. There are certain applications which embeds IP address and/or Port information in Application content. Now, the issue is while the packet traverses across a NAT44 device, layer 3 and 4 addresses get changed. However, in the Application layer, the content still contains old layer 3 and / or layer 4 addresses. This causes an issue in the Application as this is unexpected. Hence, the need of ALG functionality within a NAT device which while changing Layer 3 and 4 packet content, also changes Application content where Layer 3 and 4 addresses were embedded. With this change, end Application will be happy as it does not need to deal with 2 sets of addresses any more.

However, there are 2 ways this problem can be solved.

  • Adding ALG functionality to NAT device - ALG functionality needs to be added to NAT device for each such Applicaton which embeds Layer 3 and/or 4 addresses in Application content.
  • Changing the Application to deal with NAT device in the network - Application needs to assume that the Layer 3 and/or 4 address can get changed on the way (as there could be NAT device in the network) and hence, either avoid embedding Layer 3/4 addresses in Application content or find other mechanism to handle 2 sets of addresses.

The main challenges in adding ALG functionality to NAT device are:

  • Every time a new Application is written which embeds Layer 3/4 address in Application content, all NAT devices need to be changed to address the new Application. This is not scalable as lot of new Applications are written everyday.
  • In order to implement ALG for an Application, NAT device needs to understand the Application protocol as well as packet formt/content. Many Applications implement proprietary protocol and hence, do not want to make the same as public information. Hence, those Application packets will face issues with NAT device.
  • There are applications (like, crypto) which do not want any intermediate device to understand / peek into the Application content and modify it. These packets should only be encoded/decoded by end Applications. These application packets would not be able to deal with intermediate NAT device.

On the contrary,

  • Application can be written in such a way not to use Layer 3/4 addresses in Application content and to use Application-level ID, instead.
  • There can be other mechanisms, e.g. - using STUN (Session Traversal Utilities for NAT) that applications can take to handle intermediate NAT device.
NOTE: Due to the above reasons, our solution does not encourage to support ALG within NAT44 functionality.

However, few basic ALG functionality is implemented with NAT44 Application. These are:

  • Active FTP (File Transfer Protocol) - Passive FTP does not need ALG.
  • RTSP (Real Time Streaming Protocol)
  • PPTP (Point-to-Point Tunneling Protocol)

Following table show how to configure / enable different ALGs on ISM.

Configuration of ALGs

 

service cgn cgn1

 

  service-type nat44 nat1

   ! Enable Active FTP ALG

   alg ActiveFTP

    ! Enable RTSP ALG (from 4.2.1 release onwards)

    alg rtsp

   ! Enable PPTP ALG (from 4.3.1 release onwards)

   alg pptpAlg

18. Redundancy

Redundancy is an integral part of CGN (Carrier Grade NAT). Majority of the Service Providers want Redundancy in their network to make it more resilient. It helps in minimizing the traffic outage when one device fails.

On ISM, we support 2 types of redundancy.

  • 1:1 warm standby redundancy
  • N:1 ABF-based redundancy

1:1 Warm Stand-by Redundancy

In this mode, one ISM card acts as Primary and another as backup. Following are the properties of 1:1 Warm Stand-by redundancy.

  • Easy to configure (Active and Stand-by locations are specified once)
  • A single Public IP address pool is used
  • Configuration is sent to both ISM cards
  • NAT Application data traffic reaches to Active card only
  • During switchover (usually takes few seconds), dynamic NAT entries get deleted (on failed ISM cards) and re-created on newly Active ISM card. Hence, there would be some traffic drop. Also, Private to Public IP/Port mapping gets changed during Switchover.
  • When the previously Active ISM comes up, it stays as Stand-by and the traffic does not come back to it (until the Active ISM card fails)
  • Each ISM can support full scale and throughput

Following diagram explains the working of 1:1 Redundancy for NAT44.

1_1_Redundancy.png

Please note that both Primary and Backup ISM cards share same ServiceApp interfaces and NAT44 configuration (including Public IP pool).

Following tables shows how to configure 1:1 ISM redundancy.

Configuration for 1:1 Warm Standby Redundancy

 

! Configure role for both Primary and Backup ISMs

hw-module service cgn location 0/1/cpu0

hw-module service cgn location 0/2/cpu0

! Configure ServiceInfra interfaces on both ISMs

interface ServiceInfra 1

  ipv4 address 1.1.1.1/30

  service-location 0/1/cpu0

interface ServiceInfra 2

  ipv4 address 2.2.2.1/30

  service-location 0/2/cpu0

! Configure preferred-active and preferred-standby ISM

service cgn cgn1

  service-location preferred-active 0/1/cpu0 preferred-standby 0/2/cpu0

  service-type nat44 nat1

    inside-vrf inside1

      map address-pool 151.0.1.0/24

      ! Remaining configurration

      ...

To check the health of the ISM card and trigger redundancy switchover, in case any failure occurs, following HA-specific tests are executed.

  • Punt Path Test - It checks if the packet path between ISM LC (PPC) CPU and CGv6 Application works fine or not.
  • Data Path Test - It checks if the packet path from CGv6 Application to Fabric works fine or not.

If these tests are enabled and any failure is detected by these, ISM card will be reloaded.

NOTE: By default, these tests run in the back-ground however, no acton is taken in case failure is detected. Using the following CLIs, these tests are enabled.

Configuration of HA Punt-Path and Data-Path tests
! To enable Punt Path Test

service-cgv6-ha location 0/1/cpu0 puntpath-test

 
! To enable Data Path Test

service-cgv6-ha location 0/1/cpu0 datapath-test

N:1 ABF-Based Redundancy

In this case, there needs to be N Primary ISM card and 1 Backup ISM card. Value of N can be 1 as well, but typically would be 2 or 3. Following are different characteristics of this solution.

  • Each ISM card needs to be configured independently                                                                       
    • Configuration (usually) should be similar (w.r.to parameters, number of ServiceApp interfaces, etc.)
  • In ABF configuration, For each Primary ISM,                                                                        
    • ServiceApp interface on Primary ISM becomes 1st Next-Hop and ServiceApp interface on Back-up ISM becomes 2nd Next-hop
  • Each ISM will have different Public IP address pools (Address Pool is not same in Primary and Backup)
  • When one Primary ISM card fails, Backup ISM card receives all the traffic of Primary ISM card.
  • When Primary ISM card comes up, all traffic again switches back to it from Backup ISM card (it causes, another traffic outage)
  • Each ISM card can support full scale and bandwidth

Following table shows how the ABF Next-Hop configuration would look like for N:1 (N = 3) ABF-based Redundancy.

ServiceApp interfacesABF Next-Hop 1ABF Next-Hop 2
ServiceApp interfaces on Primary ISM 1ServiceApp interfaces on Primary ISM 1

ServiceApp interfaces on Backup ISM

ServiceApp interfaces on Primary ISM 2

ServiceApp interfaces on Primary ISM 2

ServiceApp interfaces on Backup ISM

ServiceApp interfaces on Primary ISM 3

ServiceApp interfaces on Primary ISM 3

ServiceApp interfaces on Backup ISM

Following diagram depicts N:1 (N = 2) configuration with 1 set of ServiceApps.

N_1_Redundancy.png

As shown in the diagram,

  • before 1st failure, traffic coming from 10.2.0.0/24 subnet will go via Primary 1 ISM (Inside ServiceApp 1) which will use Public IP pool of 151.0.1.0/24.
  • if and when Primary 1 ISM fails, traffic coming from 10.2.0.0/24 subnet will go via Backup ISM (Inside ServiceApp 5) which will use Public IP pool of 151.0.3.0/24.
  • Instead of Primary 1 ISM, if Primary 2 ISM would have failed, traffic from 10.2.1.0/24 (not 10.2.0.0/24) should have switched to Backup ISM.

Following table captures the relevant ABF configuration for this sample setup.

Sample Configuration for N:1 Redundancy using ABF
 

 

! ABF rule for Inside to Outside traffic

ipv4 access-list abf_2_1

  10 permit ipv4 10.2.0.0/24 any nexthop1 vrf inside1 ipv4 1.1.1.2

                                 nexthop2 vrf ibackup ipv4 1.1.5.2

  20 permit ipv4 10.2.1.0/24 any nexthop1 vrf inside2 ipv4 1.1.3.2

                                 nexthop2 vrf ibackup ipv4 1.1.5.2

  30 permit ipv4 any any

! Static route for Outside to Inside traffic

router static

  address-family ipv4 unicast

    151.0.1.0/24 ServiceApp 2

    151.0.2.0/24 ServiceApp 4

    151.0.3.0/24 ServiceApp 6

19. NAT44 Licenses

In order to use NAT44 functionality on ASR9K, Licenses needs to be purchased. Depending on the number of Stateful Translations needed by the solution, required number of NAT44 specific licenses can be purchased. This fits nicely with "Pay as you grow" model as well.

Following licenses need to be purchased for NAT44 and other CGv6 applications.

License Part NumberLicense DescriptionAdditional Comments
A9K-CGN-LIC-5MCisco ASR 9000 CGN / NAT44 License (1 per 5 million translations)For 20 Millions translations, 4 of these licenses will be needed.
A9K-DSLT-LIC-5MCisco ASR 9000 DS-Lite License (1 per 5 million translations)Similar logic as above. Not needed for NAT44 / CGN solution.
A9K-NAT64-LIC-5MCisco ASR 9000 Stateful NAT64 License (1 per 5 million translations)Similar logic as above. Not needed for NAT44 / CGN solution.

Details of the Licensing (how to install, etc.) can be obtained from ASR9K S/w License Operations White Paper.

20. Summary of parameter values

In the following table, all the useful parameters are listed along with their default and valid range of values. You can consult this table to come up with the parameter value for your deployment case.

Parameter (CLI)Defined at what levelDefault ValueRange of ValuesSupported from XR ReleaseAdditional Comments

alg ActiveFTP

per NAT44 instance

Disabled

Enable / Disable4.2.0Active FTP ALG
alg pptpAlg

per NAT44 instance

DisabledEnable / Disable4.3.1PPTP ALG
alg rtsp

per NAT44 instance

DisabledEnable / Disable4.2.1RTSP ALG

bulk-port-alloc size

per Inside VRFDisabled

16, 32, 64, 128, 256, 512, 1024, 2048, 4096

4.2.1 

dynamic-port-range start

per NAT44 instance10241 - 655354.2.0

Starting port number (for Public IP address) to be used for Dynamic NAT entries.

external-logging netflow v9 server path-mtuper Inside VRF1500 bytes100 - 20004.2.0 
external-logging netflow v9 server refresh-rateper Inside VRF500 packets1 - 6004.2.0

For sending NFv9 template

external-logging netflow v9 server timeoutper Inside VRF30 seconds1 - 36004.2.0For sending NFv9 template

external-logging syslog server path-mtu

per Inside VRF1500 bytes100 - 20004.2.1 

map ip many-to-one

per Inside VRFDisabledEnable / Disable4.3.2 
map ip one-to-oneper Inside VRFDisabled1 - 655354.2.3 
portlimitper NAT44 instance1001 - 655354.2.0 

protocol udp session initial timeout

per NAT44 instance30 sec1 - 655354.2.0 

protocol udp session active timeout

per NAT44 instance120 sec1 - 655354.2.0 

protocol tcp session initial timeout

per NAT44 instance120 sec1 - 655354.2.0 

protocol tcp session active timeout

per NAT44 instance1800 sec1 - 655354.2.0 
protocol tcp mssper Inside VRF 28 - 1500 bytes4.2.0 

Specific Deployment Scenarios / Use-Cases

In this section, specific deployment scenarion / use-cases and presented which can be useful for some deployment.

21. Sample Applications over NAT44

Following table lists some of the important / well-known applications and their working status over NAT44 (on ISM).


 

NOTE:

- This list is not a complete/exhaustive list of applications that would work with NAT44.

- It also does not mean that any application not listed here will not work with NAT44.

ApplicationDetails of Application (Protocol / Port)Working Status with NAT44
HTTPTypically HTTP server runs over TCP Port 80Works with NAT44
E-mailTypically uses TCP, server ports: SMTP (25, 587), POP3 (110), IMAP (143)

Works with NAT44

FTPTypically uses TCP port 20 (data) and port 21 (control)Works with NAT44
TelnetTypically uses TCP port 23Works with NAT44
SSHTypically uses TCP port 22Works with NAT44
VNCTypically uses TCP port 5900, 5800, 5500Works with NAT44
VPN Client (SSL)Typically uses TCPWorks with NAT44
VPN Client (PPTP+GRE)Typically uses TCP and GREWorks with NAT44
VPN Client (IPSec)Typically uses IPSec (via NAT-T)Works with NAT44 (as NAT-T uses UDP)
Xbox Gaming Works with NAT44

22. Multiple Outside ServiceApps sharing same Outside VRF

It is very common to have default VRF as Outside VRF and when there are multiple Inside VRFs mapped to the same. In the following table, a sample ServiceApp and VRF configuration is shown. Please note that in this case, multiple Outside ServiceApps share share a comon Outside VRF.

Inside VRFInside ServiceAppOutside VRFOutside ServiceApp
ivrf1ServiceApp 1default or ovrfServiceApp 2
ivrf2ServiceApp 3

default or ovrf

ServiceApp 4

To handle this scenario, we need to specify "outsideServiceApp" parameter configuration to indicate what is the Outside ServiceApp to be used with the mapping. Static route rule for specific Public IP address pool should guide the traffic to specific Outside ServiceApp. Only these configurations are shown below. Rest of the configuration remain the same.

Configuration for Shared / Common Outside VRF
service cgn cgn1
  service-type nat44 nat1

    ! 1st Inside VRF
    inside-vrf ivrf1

      ! MAP command using outsideServiceApp
      map [outside-vrf ovrf] outsideServiceApp ServiceApp2 address-pool 100.0.1.0/24

    ! 2nd Inside VRF
    inside-vrf ivrf2

      ! MAP command using outsideServiceApp
      map [outside-vrf ovrf] outsideServiceApp ServiceApp4 address-pool 100.0.2.0/24

router static
[vrf ovrf]
address-family ipv4 unicast

! Divert traffic to 1st Outside ServiceApp
100.0.1.0/24 ServiceApp 2

    ! Divert traffic to 2nd Outside ServiceApp
    100.0.2.0/24 ServiceApp 4

23. NAT Bypass

There is an use-case where traffic selected traffic coming onto ASR9K via a Private side interface needs to go via NAT44 application, whereas remaining traffic can bypass NAT44 and should get forwarded via regular forwarding rule. This is called NAT bypass.

To implement NAT bypass, let's consider this situation:

  • Private Traffic enters ASR9K via one VRF (say, RED)
  • Selected traffic gets diverted to ISM card's Inside ServiceApp which is in different VRF (say, BLUE) and after translation comes out of Outside ServiceApp and eventually goes out of ASR9K via core-facing interface which is in RED VRF.
  • Rest of the traffic (which bypasses NAT44) goes out of ASR9K via same core-facing i/f in RED VRF.

Please note that other than NAT bypass, there could be other deployment scenario where preserving VRF (across ISM) becomes a necessity. But, the solution would be same / very similar.

The above scenario can be achieved using ISM with some additional ABF and Static Route and appropriate VRF configuration.

Let us take a look at the following diagram as a sample scenario.

NAT_Bypass.png

In the above diagram, traffic enters and exits out of ASR9K in RED VRF. Also, please note that Inside ServiceApp is in BLUE vrf whereas Outside ServiceApp is in RED VRF.

Following table shows the specific or additional configuration that is needed for appropriate traffic diversion.

Additional Configuration to support NAT Bypass

 

! Define ACL to divert traffic to ServiceApp 1 (RED to BLUE VRF)

ipv4 access-list cgn_abf

  ! Divert traffic to ServiceApp 1 in BLUE VRF

  10 permit ipv4 10.10.10.0/24 any nexthop1 vrf BLUE ipv4 1.1.1.2

  20 permit ipv4 any any

! Apply ABF on Ingress LC interface

interface TenGigE 0/0/0/0.1

  vrf RED

  ipv4 address 10.10.10.1/24

  ! Apply Ingress ABF

  ipv4 access-group cgn_abf ingress

! We need a special static route to divert O2I traffic (after reverse NAT)

! to the right interface

router static

  ! After reverse NAT, look-up is done in BLUE (Inside) VRF

  vrf BLUE

    address-family ipv4 unicast

      ! Divert to TenGigE 0/0/0/0.1 interface

      10.10.10.0/24 vrf RED 10.10.10.2

24. Interworking of BNG and NAT44

Broadband Network Gateway (BNG) is the access point for subscribers, through which they connect to the broadband network.  When a connection is established between BNG and Customer Premise Equipment (CPE), the subscriber can access the broadband services provided by the Network Service Provide (NSP) or Internet Service Provider (ISP).

ASR9000 supports BNG functionality.

The goal of the BNG architecture is to enable the BNG router  to interact with peripheral devices (like CPE) and servers (like AAA and DHCP), in order  to provide broadband connectivity to subscribers and manage subscriber sessions.

Each subscriber (or more specifically, an application running on the CPE) connects to the network by a logical session. Based on the protocol used, subscriber sessions are classified into two types:

  • PPoE subscriber session —The PPP over Ethernet (PPPoE) subscriber session is established using the point-to-point (PPP) protocol that runs between the CPE and BNG.
  • PoE subscriber session—The IP over Ethernet (IPoE) subscriber session is established using IP protocol that runs between the CPE and BNG; IP addressing is done using the DHCP protocol.

For details on BNG, please refer to BNG Deployment Guide on ASR9K.

On the other hand CGN (NAT44) functionality is also supported on ASR9K. Hence, BNG and CGN (NAT44) functionality can interwork to provide NAT'ed (public) IP address to the subscriber traffic (both PPPoE and IPoE).

The basic BNG + CGN (NAT44) architecture is shown in the following figure.

BNG_NAT44.png

As shown in the above diagram, subscriber traffic reaches ASR9K router via Bundle Ether interface (in different VLANs), which is in VRF ivrf. It does not necessarily need to arrive in a VRF though. Please note that session establishment happens without the involvement of NAT44. NAT44 comes into picture when Subscriber data traffic reaches ASR9K.

After the BNG functionality (like, PPPoE or IPoE termination/de-capsulation) is performed on the Subscriber data traffic (on LC), internal IP packet (containing Private IP address as Source) is sent to ISM card where NAT44 functionality kicks in which converts the Private IP address into Public IP address. Similarly, on the reverse (O2I) path, after ISM performs NAT44 functionality (changes destination IP address to Private), paket is sent to LC where BNG functionality (like, PPPoE or IPoE encapsulation) is peformed before sending the packet to the Subscriber.

Following table shows sample snippet of BNG and CGN (NAT44) configuration (for IPoE subscriber) to achive the same. For PPPoE subscriber, corresponding BNG configuration needs to be used instead.

Please refer to ASR9000 Broadband Network Gateway Configuration Guide for detailed and release-specific BNG configuration.

Sample BNG and CGN (NAT44) Configuration for BNG+CGN Interworking
 

 

! =============================================

! --- BNG Configuration for IPoE Subscriber ---

! =============================================

! DHCP Configuration

dhcp ipv4

profile proxyProfile proxy

  helper-address vrf default 13.0.0.2 giaddr 13.0.0.1

!

interface Bundle-Ether1.2 proxy profile proxyProfile

! Dynamic Template Configuration

dynamic-template

type ipsubscriber dhcp

  vrf ivrf

  ipv4 unnumbered Loopback1

! Policy-Map and Class-map configuration

policy-map type control subscriber pol1

event session-start match-all

  class type control subscriber cls1 do-all

   1 activate dynamic-template dhcp

   2 authorize aaa list default identifier source-address-mac password cisco

 

end-policy-map

class-map type control subscriber match-all cls1

match protocol dhcpv4

end-class-map

! Interface (Bundle-Ether and Loopback) Configuration

interface Bundle-Ether1.2

vrf ivrf

ipv4 point-to-point

ipv4 unnumbered Loopback1

proxy-arp

service-policy type control subscriber pol1

encapsulation dot1q 10

ipsubscriber ipv4 l2-connected

  initiator dhcp

interface Loopback1

vrf ivrf

ipv4 address 10.0.0.1 255.255.0.0

! ==========================

! CGN (NAT44) Configuration

! ==========================

hw-module service cgn location 0/5/CPU0

! Interface (ServiceInfra and ServiceApp) Configuration

interface ServiceApp1

vrf ivrf

ipv4 address 1.1.1.1 255.255.255.0

service cgn cgn1 service-type nat44

!

interface ServiceApp2

ipv4 address 2.2.2.2 255.255.255.0

service cgn cgn1 service-type nat44

!

interface ServiceInfra1

ipv4 address 200.1.1.1 255.255.255.0

service-location 0/5/CPU0

! Service CGN Configuration

service cgn cgn1

service-location preferred-active 0/5/CPU0

service-type nat44 nat1

  inside-vrf ivrf

   map address-pool 100.2.0.0/16

! Static Route Configuration

router static

address-family ipv4 unicast

 

  100.2.0.0/16 ServiceApp2

!

vrf ivrf

  address-family ipv4 unicast

   0.0.0.0/0 ServiceApp1

25. MPLS (Layer 3) VPN and NAT Interworking

An Multiprotocol Label Switching (MPLS) Layer 3 Virtual Private Network (L3VPN) consists of a set of sites that are interconnected by means of an MPLS provider core network. At each customer site, one or more customer edge (CE) routers attach to one or more provider edge (PE) routers.

MPLS VPNs are easier to manage and expand than conventional VPNs. When a new site is added to an MPLS VPN, only the edge router of the service provider that provides services to the customer site needs to be updated. The components of the MPLS VPN are described as follows:

  • Provider (P) router — Router in the core of the provider network. PE routers run MPLS switching and do not attach VPN labels to routed packets. VPN labels are used to direct data packets to the correct private network or customer edge router.
  • Provider Edge (PE) router — Router that attaches the VPN label to incoming packets based on the interface or subinterface on which they are received, and also attaches the MPLS core labels. A PE router attaches directly to a CE router.
  • Customer edge (CE) router — Edge router on the network of the ISP that connects to the PE router on the network. A CE router must interface with a PE router.

Details on L3VPN can be obtained from ASR9K MPLS Layer 3 VPN Configuration Guide.

Service providers and enterprises that have network application services they want to offer or share with customers and partners will want to      minimize any connectivity burden placed on the user of the service. It is desirable, even mandatory, to extend the offering to as many potential users as needed to achieve the desired goals or return.

By deploying NAT within the common MPLS VPN infrastructure, communications service providers can relieve some of the connectivity burden on      customers and accelerate their ability to link more shared application services to more consumers of those services

There are two options for NAT deployment within the MPLS provider edge:

  • Distributed with ingress NAT PEs
  • Centralized with egress NAT PEs

Following diagram shows Distrbuted NAT scenario with Ingress NAT PEs.

With this design, scalability is maintained to a large extent while performance is optimized by distributing the NAT function over many edge devices. Each NAT PE handles traffic for sites locally connected to that PE. NAT rules and access control lists or route maps control which packets require translation.

L3VPN_NAT_Ingress_PE.png

In the above case, at ingress PE router, customer private IP addresses are NAT'ed to public IP address by ISM card and then MPLS labels are added by it before the packet goes out of ASR9K. On the return path, MPLS label(s) are removed by MPLS-facing LC and IT packets are sent to ISM. ISM performs reverse NAT and sends the packet to LC which then sends it back to CE router.

Following diagram shows Distrbuted NAT scenario with Ingress NAT PEs.

With this design, scalability is reduced to some degree since the central PE must maintain routes for all customer networks that access the shared service. The application performance requirements must also be considered so that the traffic does not overburden the router that must translate the IP addresses of the packets. Because NAT occurs centrally for all customers using this path, IP address pools can be shared; thus, the total number of subnets required is reduced.

L3VPN_NAT_Egress_PE.png

In the above case, in the egress PE router, first MPLS labels are removed by ingress LC and the IP packet is sent to ISM card. Customer private IP addresses are NAT'ed to public IP address by ISM card and the packets are forwarded to shared service network. In the reverse path, ISM translates Public destination IP to private IP addresses and then adds MPLS labels before sending it to egress LC. Egress LC swaps one label and sends the packet out of ASR9K towards MPLS network.

NOTE: MPLS labels should be removed by LC before sending the packet to ISM as IP packet. However, ISM can add the required MPLS labels before sending the packet to egress LC.

For sample MPLS VPN configuration, please refer to "Configuration Example: MPLS VPN on XR" document.

26. NAT Hairpining

Host behind a NAT device (on Private side) communicating with another host behind the same NAT device (on Private side) is called "NAT Hairpinning". Let us consider the following deployment use-case.

We have a server behind a NAT device which has a Public IP as well, created using Static Port Forwarding configuration. Now, this server can be accessed from hosts on public side of NAT device as well as Private side of the NAT device. When a host on Private side of the NAT device tries to communicate with the Server, it uses NAT Hairpinning. Following diagram depicts how NAT hairpinning works with NAT44 on ISM.

NAT44_Hairpinning.png

In this case, we first add a Static Port Forwarding entry for a service offered by the Server at 10.100.1.2:2055. Let's say, the Public IP pool configured is 200.2.0.0/24. NAT44 application allocates Public IP and port as 200.2.0.210:2055 (Please note that the port is preserved here). This Public IP and port is published / made available to the hosts. Let's the packet flow now when a host with Private IP 10.10.10.2 tries to access the service.

  • Step 1: Host sends a packet (with Source IP 10.10.10.2 and port 3000) to Server's Public IP and port and the packet reaches ASR9K on TenGigE 0/0/0/0.1 which is in VRF ivrf.
  • Step 2: Because of Static Route configuration in VRF ivrf, the packet is forwarded to ISM card via (Inside) ServiceApp 1 interface.
  • Step 3: Application treats this as I2O packet. A new dynamic NAT entry is created by NAT44 application and it translates source IP/Port from (Private) 10.10.10.2:3000 to (Public) 200.2.0.192:23516. Please note that the Destination IP address and port are not touched here. The packet is sent out and because of Static route definition for Public IP pool, the packet comes back to ISM card via (Outside) ServiceApp 2.
  • Step 4: Application treats this as O2I packet. As there is already a (Static) NAT entry exists for Destination IP and port, NAT44 application translates Destination IP address/Port from (Public) 200.2.0.210:2055 to (Private) 10.100.1.2:2055. Please note that the Source IP address and port are not touched here. The packet is sent out and via regular forwarding rule, it reaches egress i/f TenGigE 0/0/0/0.2.
  • Step 5: The packet is forwarded to the Server with Source IP/Port as 200.2.0.192:23516 and Destination IP/Port as 10.100.1.2:2055 and the server consumes it.

Response from the Server will take the reverse path and it will use Dynamic NAT rule in first reverse NAT and then Static NAT rule in second NAT processing, to reach the host.

In this case, Static Port Forwarding configuration is used for Server as an use-case. But, it can be another host as well for which there is no Static Port Forwarding configuration. Only important thing is sender needs to know the Public IP and port of the receiver.

Please note that there is no specific NAT44 configuration is needed for this. Regular / basic NAT44 configuration will work here. Hence, no configuration is provided here.

27. Handling of Asymmetric I2O and O2I traffic with better performance

ISM will provide optimal performance when I2O and O2I traffic is symmetric in nature i.e., equally balanced. However, it may not be always true. Many times, O2I traffic (download from internet) is much higher than I2O (upload to internet) traffic. To obtain optimal / maximum performance, you can divide traffic in each logical partition into 2 sub-groups (which means Public IP address pool as well needs to divided), with another pair of ServiceApps and switching the order of direction (Inside / Outside) of ServiceApps. Following table tries to summarize it with 2 pairs of ServiceApps.

Logical Group 1Logical Group 2
Inside ServiceAppOutside ServiceApp
ServiceApp 1ServiceApp 2
ServiceApp 6ServiceApp 5
Inside ServiceAppOutside ServiceApp
ServiceApp 3ServiceApp 4
ServiceApp 8ServiceApp 7

Please note that ServiceApp 5 and ServiceApp 7 are configured as Outside ServiceApp, instead of Inside ServiceApp. Similarly, ServiceApp 6 and ServiceApp 8 are configured as Inside ServiceApp, instead of Outside ServiceApp.

You will get a better performance provided you can distribute your traffic in nearly equal subscriber groups.

Following table captures sample configuration for the same.

Configuration for Handling Asymmetric I2O and O2I traffic for better performance

 

! Inside ServiceApp configuration

interface ServiceApp 1

  vrf ivrf1

  ipv4 address 1.1.1.1/30

interface ServiceApp 3

  vrf ivrf2

  ipv4 address 1.1.3.1/30

interface ServiceApp 6

  vrf ivrf3

  ipv4 address 1.1.6.1/30

interface ServiceApp 8

  vrf ivrf4

  ipv4 address 1.1.8.1/30

! Outside ServiceApp configuration

interface ServiceApp 2

  ipv4 address 1.1.2.1/30

interface ServiceApp 4

  ipv4 address 1.1.4.1/30

interafce ServiceApp 5

  ipv4 address 1.1.5.1/30

interface ServiceApp 7

  ipv4 address 1.1.7.1/30

! ABF Configuration to divert traffic to all Inside ServiceApp interfaces

ipv4 access-list cgn_abf

  ! For traffic to ServiceApp 1

  10 permit 10.0.1.0/24 any nexthop1 vrf ivrf1 1.1.1.2

  ! For traffic to ServiceApp 3

  30 permit 10.0.2.0/24 any nexthop1 vrf ivrf2 1.1.3.2

  ! For traffic to ServiceApp 6

  60 permit 10.0.3.0/24 any nexthop1 vrf ivrf3 1.1.6.2

  ! For traffic to ServiceApp 8

  80 permit 10.0.4.0/24 any nexthop1 vrf ivrf4 1.1.8.2

  100 permit any any

! Service CGN Configuration

service cgn cgn1

  service-type nat44 nat1

  ! Address Pool for ServiceApp 1 and 2 pair

  inside-vrf ivrf1

    map outsideServiceApp 2 address-pool 100.0.1.0/24

  ! Address Pool for ServiceApp 3 and 4 pair

  inside-vrf ivrf2

    map outsideServiceApp 4 address-pool 100.0.2.0/24

  ! Address Pool for ServiceApp 6 and 5 pair

  inside-vrf ivrf3

    map outsideServiceApp 5 address-pool 100.0.3.0/24

  ! Address Pool for ServiceApp 8 and 7 pair

  inside-vrf ivrf4

    map outsideServiceApp 7 address-pool 100.0.4.0/24

Validation of Configuration

Once NAT44 on ISM is configured, we need different ways (mainly, via "show" CLIs) to validate those. In this section, important NAT44 related CLIs are listed and also explained how those can be used for validation.

28. show run

As a first step, different "show run" output should be verified to ensure that configuration commands are rightly applied. Following commands can be used.

'show run' commandsUsage of it
show running-configuration  or show run

- Displays all the configuration on ASR9K.

- If you have less configuration, this is better way to get all NAT44 configuration at one shot.

- Otherwise (if you have lot of configuration, this may not be the best way to get all NAT44 configuration

show run service cgn *- Displays only CGN related information
show run interface service*

- Displays all ServiceInfra and ServiceApp interface configuration

- Also includes Service-Mgmt and Service-Engine interface configuration (which can be ignored)

show run router static- Displays all static route information.

29. show cgn nat44 <> statistics

This is a very important CLI related to NAT44. A sample output of the CLI is shown below.

Sample "show cgn nat44 <> statistics" CLI Output
 

 

RP/0/RSP0/CPU0:nat-tasix#Show cgn nat44 nat statistics

Fri Aug 16 23:34:40.018 TSK Statistics summary of NAT44 instance: 'nat'
Number of active translations: 91407

Number of sessions: 39716

Translations create rate: 542
Translations delete rate: 1206

Inside to outside forward rate: 147827

Outside to inside forward rate: 204847

Inside to outside drops port limit exceeded: 10077470

Inside to outside drops system limit reached: 0

Inside to outside drops resource depletion: 0

No translation entry drops: 194533431

PPTP active tunnels: 3

PPTP active channels: 2

PPTP ctrl message drops: 1246

Number of subscribers: 0

Drops due to session db limit exceeded: 0

Pool address totally free: 498
Pool address used: 1550
Pool address usage:

-------------------------------------------------

  External Address       Ports Used
-------------------------------------------------
  213.230.92.0           79   213.230.92.4           176   213.230.92.8           59   213.230.92.12          36   213.230.92.16          79   213.230.92.20          52   213.230.92.24          65
  ...

  213.230.103.239        2

  213.230.103.247        1   213.230.103.255        1

-------------------------------------------------

Following table explains the impotant fields from the above CLI and how it can be used for validation.

Please note that the values displayed by the CLI is valid at that instant for that particular ISM card. It may change at a later point of time.

Field NameDescriptionAdditional Comments for Validation
Number of active translations
Number of NAT DB (Static + Dynamic) EntriesMonitoring this over a period of time will provide the NAT44 translation scale used in the ISM card.
Number of sessions

Number of Session DB (Static + Dynamic) Entries

Monitoring this over a period of time will provide the NAT44 session  scale used in the ISM card.

Translations create rate
Number of NAT44 translations created in last 1 second

Monitoring this over a period of time will provide the NAT44 session  setup rate on the ISM card.

Translations delete rate

Number of NAT44 translations deleted  in last 1 second

Monitoring this over a period of time will provide the NAT44 session  teardown rate on the ISM card.

Inside to outside forward rate
Number of packets gone through NAT44 translation in I2O direction in last 1 second

Monitoring this over a period of time will provide the NAT44 packet processing rate on ISM card (PPS) in I2O direction

Outside to inside forward rate

Number of packets gone through NAT44 translation in O2I direction in last 1 second

Monitoring this over a period of time will provide the NAT44 packet processing rate on ISM card (PPS) in O2I direction

Inside to outside drops port limit exceeded
Number of I2O packets dropped because no NAT entries cannot be created for the specific users which has reached port limit.If you have configured port limit value too less, you may hit this errors very often. You may need to consider to increase the port limit configuration in that case.
Inside to outside drops system limit reached

Number of I2O packets dropped because NAT DB or User DB is full.

In case you are hitting this very often, you may need to check if you're hitting the scale limit and if so, you may need to go for another ISM card.
Inside to outside drops resource depletion

Number of I2O packets dropped because Layer 4 port could not be allocated.

 
No translation entry drops

Number of O2I packets dropped because there was no NAT entry existing.

 
Number of subscribers
Number of NAT Users 
Drops due to session db limit exceeded

Number of I2O packets dropped because Session DB is full.

 

30. show cgn nat44 <> inside-translation

This command is helpful in seeing the list of all Outside / Public IP address + Port mapping for a specific Inside / Private IP address (and a Port range).

Following is a sample output of the command.

Sample "show cgn nat44 <> inside-translation" CLI Output

 

RP/0/RSP0/CPU0:McKinely#show cgn nat44 nat1 inside-translation protocol udp inside-vrf red

                        inside-address 10.2.5.134 port start 1 end 4580

Wed May 15 14:25:09.713 UTC

Inside-translation details

---------------------------

NAT44 instance : nat1

Inside-VRF     : red

--------------------------------------------------------------------------------------------

   Outside         Protocol  Inside       Outside       Translation   Inside      Outside

   Address                   Source       Source        Type          to          to

                             Port         Port                        Outside     Inside

                                                                      Packets     Packets

--------------------------------------------------------------------------------------------

  100.200.2.75       udp     4500         65024         dynamic       9236        2213

  100.200.2.75       udp     4501         65153         dynamic       2236        0

  100.200.2.75       udp     4502         65152         dynamic       4232        1218

  100.200.2.75       udp     4503         65155         dynamic       2236        7128

  100.200.2.75       udp     4504         65154         dynamic       2235        1821

  100.200.2.75       udp     4505         65157         dynamic       5236        3129

  100.200.2.75       udp     4506         65156         dynamic       2238        8172

  100.200.2.75       udp     4507         65159         dynamic       7238        2189

  100.200.2.75       udp     4508         65158         dynamic       2236        3119

  ...

  100.200.2.75       udp     4576         65226         dynamic       1239        2178

  100.200.2.75       udp     4577         65229         dynamic       8235        4171

  100.200.2.75       udp     4578         65228         dynamic       2235        0

  100.200.2.75       udp     4579         65231         dynamic       4230        4127

  100.200.2.75       udp     4580         65230         dynamic       2235        0

RP/0/RSP0/CPU0:#

Above table shows all Outside IP (100.200.2.75) and Port combination for Inside IP (10.2.5.134) and Port Range (1 through 4580).


 

NOTE:

- As one Inside address can only map to one Outside address, there are not multiple Outside addresses in 1st column.

- The number of entries in this table should not exceed Port Limit value.

31. show cgn nat44 <> outside-translation

This command is helpful in seeing the list of all Inside / Private IP address + Port mapping for a specific Outside / Public IP address (and a Port range).

Following is a sample output of the command.

Sample "show cgn nat44 <> outside-translation" CLI Output

 

RP/0/RSP0/CPU0:McKinely#show cgn nat44 nat1 outside-translation protocol udp

                        outside-address 100.200.2.75 port start 1 end 65535

Wed May 15 14:17:44.640 UTC

Outside-translation details

---------------------------

NAT44 instance : nat1

Outside-VRF    : default

--------------------------------------------------------------------------------------------

   Inside          Protocol  Outside      Inside         Translation  Inside      Outside

   Address                   Destination  Destination    Type         to          to

                             Port         Port                        Outside     Inside

                                                                      Packets     Packets

--------------------------------------------------------------------------------------------

  10.2.5.134         udp     65024        4500          dynamic       4236        1216

  10.2.5.134         udp     65025        4885          dynamic       9233        5161

  10.2.5.134         udp     65026        4886          dynamic       4244        3491

  10.2.5.134         udp     65027        4887          dynamic       1281        6018

  10.2.5.134         udp     65028        4888          dynamic       2237        4187

  ...

  10.2.5.142         udp     65532        4882          dynamic       6261        3178

  10.2.5.142         udp     65533        4881          dynamic       7282        0

  10.2.5.142         udp     65534        4884          dynamic       1211        4178

  10.2.5.143         udp     65535        4883          dynamic       1261        5189

RP/0/RSP0/CPU0:#

Above table shows all Inside IPs (10.2.5.134, 10.2.5.142) and Port combination for Inside IP (100.200.2.75) and Port Range (1 through 65535). In this case, there are multiple Inside IP addresses are mapped to one Public IP address.

32. show services redundancy

This CLI is used to see the current state of CGN Service redundancy - which ISM is active and which is Standby, as shown by the sample output below.

Same "show services redundancy" CLI Output

 

RP/0/RSP0/CPU0:ios#sh services redundancy

Wed May  2 07:35:26.070 UTC

Service type     Name                    Pref. Active        Pref. Standby

--------------------------------------------------------------------------------

ServiceInfra     ServiceInfra1           0/2/CPU0 Active

ServiceInfra     ServiceInfra2           0/0/CPU0 Active

ServiceCgn       cgn1                    0/2/CPU0 Standby    0/0/CPU0 Active

As shown above, Preferred Active ISM card at slot 2 is now in Standby mode whereas, Preferred Standby ISM card at slot 0 is now in Active mode. It indicates, there was atleast one failure happened on ISM card at slot 2.

Useful Deployment Tips

  • Go through this Deployment Guide to clearly understand            
    • how different feature works
    • what parameters are configurable and need to be considered for specific deployment
    • what are the different deployment use cases.
  • Go through "CGv6 on ISM: Features Supported Across IOS-XR Release" page  to exactly know from which release specific features that you're  interested in are supported. This will help you in selecting the right  IOS-XR release.            
    • Select the configuration parameter values wisely, considering           
      • pros and cons,
      • best-case and worst-case scenarios
      • side effects
    • Consider using ISM redundancy, to minimize traffic downtime during failure.
    • Draw network diagram for deployment use-case including interface names (with VRF and IP address/subnet) and review for correctness.
    • Try to find out what type of application traffic (VPN, Games, SIP, Video, etc.) may undergo NAT44 and whether there could be issues or not (not all applications are NAT friendly)           
      • If possible, try it in a lab setup, before actual deployment
    • Estimate traffic throughput and nature of it (burstiness, average, etc.), scale of sessions/users and accordingly use correct number of ISM cards and related configuration.
    • If possible, deploy in phases. First, send selected traffic (you can use ABF) through ISM and see if there are any issues. If not, you can send more and more traffic through ISM.
    1. CGv6 on ISM: Features Supported Across IOS-XR Releases
    2. ASR9K Configuration Guides
    3. ASR9K Command Reference Guides
    4. ASR9K BNG Deployment Guide
    5. IANA Service Name and Transport Protocol Port Port Number Registry
    6. Configuration Example: MPLS VPN on IOS-XR
    7. Cisco ASR9000 S/w License Operations White Paper

    - 0 -

    Comments

     

    Hi

    Is it possible to use this ISM helth check in conjuction with EEM?

    What is your recomendation for redundancy in the scenario shown in the topology below where I have two ISM but in different locations. Is it better to use active/active or active/standby?

     

    To check the health of the ISM card and trigger redundancy switchover, in case any failure occurs, following HA-specific tests are executed.

    • Punt Path Test - It checks if the packet path between ISM LC (PPC) CPU and CGv6 Application works fine or not.
    • Data Path Test - It checks if the packet path from CGv6 Application to Fabric works fine or not.

     

     

     

    Thanks in advance !

     

    Renato Reis

    Cisco Employee

    Hi Renato,

    None of the two mechanism is intrinsically better. Which one to chose depends on the user requirements.

    If you have a pair of CGN routers in a PoP and you want to use the same public IP address pool, Warm Stand-by Redundancy would be a suitable choice.

    If you have a pair of CGN routers in a PoP and want load balance traffic across them, you would have to go for the N:1 redundancy using ABF (combined with next-hop tracking via IPSLA).

    If you have more than two CGN routers in a PoP, N:1 redundancy using ABF (combined with next-hop tracking via IPSLA) would be a better choice. 

    Hope this helps,

    Aleksandar

    Hi Aleksandar, thanks for your answer.

     

    We actually have a pair of CGN routers but they are in different locations, so I think none of the options you mentioned would address my need.  So is it possible to deploy EEM + ISM helth check to detect an aplicattion issue?

     

    Thanks,

     

    Renato Reis

     

     

     

    Cisco Employee

    Hi Renato,

    yes, you can combine the two using EEM/Tcl. 

    Say router B uses 10.3.3.3 as nexthop1 in ABF. IPv4 address 10.3.3.3 is owned by Loopback3 on router A and advertised through IGP towards router B. In this scenario you could use the punt/data-path failure as a trigger to invoke an EEM/Tcl script which shuts down Loopback3 interface. 10.3.3.3 won't be reachable any more from router B, so it will start using nexthop2 from ABF.

    There is a good article about ABF and tracking objects (in case you don't want to have 10.3.3.3 advertised via IGP) at:

    https://supportforums.cisco.com/discussion/11727191/ios-xr-abf-behavior

    If you want to deploy such a solution, I'll check which syslog message would be the best trigger for the EEM/Tcl script. 

    EEM/Tcl in XR is well documented on cisco.com and supportforums.

     

    Regards,

    Aleksandar

    Cisco Employee

    Hi Aleks, Renato,

    First, a quick precision: we don't support warn-standby between CGN routers but between two CGN cards in the same router.

    Second, we strongly advise using two different NAT pools, one in each CGN router (for each location). It's certainly possible to use ABF and IPSLA tracking and EEM but past experience (unless I've been particularly unlucky) proved this approach to be prone to false positives and potentially damaging.

    Unless you really can not do otherwise, my advise will be: keep it simple. Anycast routing for your two exit point / translation point to attract i2o traffic and one dedicated map pool range for each router to guarantee a clear path for your o2i traffic.

    My 2 euro cents,

    Cheers,

    Nicolas.

    Hi Aleksandar,

    Thank you so much for your time !  I would be glad if you could provide me the syslog messages.

     

    Thank you again.

     

    Hi Nicolas,

     

    Thank you for your information. I was looking for something to make it more resilient, I guess I will test both solutions and see each one is the best.

     

    Regards,

     

    Renato Reis

    Cisco Employee

    Hi Renato,

    the syslog message for the datapath failure contains this string:

    HB:Triggering recovery

    I would, however, warmly recommend to follow Nicholas's recommendation. Simple solutions are always the best.

     

    Regards,

    Aleksandar

    Thank you guys !

    Regards !

     

    Bronze

    Hello,

     

    I see that 5.1.3 is out on cisco.com, but I do not see 5.1.3 for ISM module, only 5.1.2.

    Until now there was always for every IOS-XR version also an ISM software with same numbering.

    e.g. IOS-XR 4.3.4 and ISM 4.3.4

    Question: Can I upgrade to 5.1.3 and leave ISM on 5.1.2?

    Cisco Employee

    Hi Smail,

     

    thanks for bringing this to attention. I'll check why the ISM install kit for 5.1.3 wasn't posted and get back to you. I wouldn't recommend deploying 5.1.3 XR with 5.1.2 ISM image.

     

    Regards,

    Aleksandar

    Bronze

    Hi Aleksandar,

     

    I could not postpone the MW and I did the upgrade. CGN is working fine, but I am not sure

    about fpga2 which was not upgraded in the process.

     0.1   Maintenance LC6            lc   fpga2   0       0.01     Yes

    Probably a new version is needed for firmware upgrade.

    Cisco Employee

    Hi Smail,

    5.1.3 ISM and VSM images will be posted later today.

    Was the FPD on all other entities upgraded with success?

    Regards,

    Aleksandar

    Bronze

    Hi,

    upgrade was not necessary, according to the log.

    This is the message I got after entered the activate command. First time that I have saw it and I already did a dozon of upgrades.

    FPD upgrade in progress on some hardware, reload/configuration change 
    on those is not recommended as it might cause HW programming failure 
    and result in RMA of the hardware.

    Info:     FPD Upgrade: No fpd on location 0/RSP0/CPU0 need upgrade at this time.
    Info:     FPD Upgrade: No fpd on location 0/RSP1/CPU0 need upgrade at this time.
    Warning:  FPD Upgrade: Failed to parse FPD desc file
    Warning:  FPD Upgrade: Failed to parse FPD desc file
    Info:     FPD Upgrade: No fpd on location 0/0/CPU0 need upgrade at this time.
    Info:     FPD Upgrade: No fpd on location 0/1/CPU0 need upgrade at this time.
    Info:     FPD Upgrade: No fpd on location 0/2/CPU0 need upgrade at this time.

     

    0/0/CPU0 is ISM

    0/1/CPU0 and 0/2/CPU0 are A9K-MOD80-SE with one A9K-MPA-2X10GE in each.

    Bronze

    Update:

     

    ISM Install kit for 5.1.3 is now running on our ISM module.

    Install process did not want to start until fpga2 (Maintenance LC6) was not upgraded.

    I have not got any message from the ASR, but I assumed that this was mandatory.

     

    New Member

    Hi, All:

     

    I was wondering if someone could offer advice to the following problem. I have a very similar setup to the one described here (with VSM 5.1.2) where I use ABF to divert traffic into the inside serviceapp3 interface. there are differences though:

    - ABF is configured on an interface in an NV Sat.

    - The nv sat interface is on the default VRF with ABF pointing to NH within inside-vrf.

    - static route pointing to ServiceApp33 (vrf default)

    - static route in inside-vrf pointing to NH on vrf default

     

    My problem is that everything points to traffic going into the VSM, NAT translation being generated and I actually see I2O packets, but no return whatsoever.

     

    RP/0/RSP0/CPU0:XXXXXX#show cgn nat44 NAT44-3 inside-translation protocol icmp inside-vrf INSIDE inside-address 192.168.136.129$
    Wed Jan 28 10:14:35.544 CST
    Inside-translation details
    ---------------------------
    NAT44 instance : NAT44-3
    Inside-VRF     : INSIDE
    --------------------------------------------------------------------------------------------
       Outside         Protocol  Inside       Outside       Translation   Inside      Outside
       Address                   Source       Source        Type          to          to
                                 Port         Port                        Outside     Inside
                                                                          Packets     Packets
    --------------------------------------------------------------------------------------------
      200.x.y.1      icmp    71           61327         dynamic       5           0           
    RP/0/RSP0/CPU0:XXXXXX#show run router static
    Wed Jan 28 10:14:49.528 CST
    router static
     address-family ipv4 unicast
      200.x.y.0/24 ServiceApp33

     !

    vrf INSIDE
      address-family ipv4 unicast
       192.168.136.128/25 vrf default GigabitEthernet100/0/0/0 192.168.136.2
     !

    At least the inside portion looks fine. On the outside serviceapp I see output packets, but no input packets. As you can see, I have the return static route pointing to Sapp33, but no luck.

     

    Any advice?

    Thanks,

    c.

     

    Cisco Employee

    Hi Carlos,

    AFAIK, the usage of nV Satellite and CGN is not supported (doesn't mean it doesn't work but it means it's not tested in our validation testbeds hence we can not guarantee what is the normal behaviour in such use case).

    Sorry about that,

    Best regards,

    N.

    New Member

    Well, it works. At least at this point and with ABF as traffic-redirection means.

     

    Next, I'm testing 'traffic-redirection' by using MPLS-VPN with the exact same setup. Hoping it'll work.

    New Member

    hi, all:

     

    Is the configuration of multiple map address-pools on a single outside vrf supported for NAT44 on both the CGSE and VSM?

     

    I have a single inside vrf and default outside vrf and I'm wondering how to configure pool growth if necessary. Would I just add a second map address-pool command to the inside-vrf section?

     

    thanks in advance!

    c.

    New Member

    Hi, there:

    Is there a way to 'clusterize' VSMs with CGN function? I mean a way to orchestrate them in a single group so that configuration for the ASR9k is the same as if only one VSM was installed, but having several operating as one, with the additional BW available.

    Can you please confirm available (total) CGN BW for a single VSM? It gets confusing, some people use total throughput, some unidirectional BW. I've found presentations that say 30Gbps, and another 60Gbps. Maybe 60Gbps is total throughput. Please clarify.

    Thanks,

    c. 

    Cisco Employee

    Hey Carlos,

    Im sorry to say but we do not have "clusterize" functionality on VSM.

    Regarding BW then both of your answers are valid.

    Throughput of 30Gbps is with IMIX traffic test where we use different type of short packets ~ 360 bytes.

    Where as throughput of 60Gbps is with default packet size.

    Thanks

    Nitin Pabbi

    New Member

    Thanks, Nitin.

    A quick comment: ciscolive presentation says specifically 60Gbps IMIX. Still, got your answer, thank you.

    Bronze

    Hello,

    since RSP switchover session-logging (netflow) is not working anymore. We tried to remove and re-enter the config again but it didn't help. 

    Second thing what we could try is to reload the ISM, but I would like to try something less interruptive. What process could I restart? I know about cgn_ma and this one was restarted after RSP switchover. 

    Cisco Employee

    hi Smail,

    I doubt restarting any process on the RSP would help because the netflow records are  exported directly from line cards.

    Is the syslog logging still working?

    regards,

    Aleks

    Bronze

    Hi Aleks,

    syslog is working fine. Only netflow export from the ISM to the collector is not working.

    This is the config:

    external-logging netflow version 9
    server
    address 10.100.31.195 port 9995
    refresh-rate 60
    timeout 1
    protocol tcp
    session-logging

    Cisco Employee

    hi Smail,

    in that case I would first check whether the netflow config is good on the ISM and whether the FIB entry for 10.100.31.195 is correct on ISM. Commands required to check that are unfortunately not in XR, you would have to connect to the ISM console and run commands from the shell. Since these are debugging commands they are not documented externally. Would you mind opening a TAC SR for further investigation? We'll be in a position to assist you better. If you open a TAC SR please let me know the ID.

    regards,

    Aleks

    Bronze

    Hi,

    I will probably do that after we have ISM reload. We have a backup route to a PE with ISM and subscribers will probably not notice it. We will also do it in off-peak time.

    We have to get the netflow output because of Lawful intercept.

    Regarding FIB entry. I have checked it and it seems ok.

    [root@localhost ~]# show_fib_table -l 10.100.31.195 -v 17
    [fib_shmem_generic_allocator,141] oflag 0x2 mode 0x100
    mmap successful at address 0x7000000000.
    show_fib_table: Lookup: of 17:10.100.31.195 Pass leaf_idx=0x0000002e

    [root@localhost ~]# show_fib_table -l 10.100.31.195 -v 20
    [fib_shmem_generic_allocator,141] oflag 0x2 mode 0x100
    mmap successful at address 0x7000000000.
    show_fib_table: Lookup: of 20:10.100.31.195 Pass leaf_idx=0x0000004d

    [root@localhost ~]# show_fib_table -l 10.100.31.195
    [fib_shmem_generic_allocator,141] oflag 0x2 mode 0x100
    mmap successful at address 0x7000000000.
    show_fib_table: Lookup: of 0:10.100.31.195 Pass leaf_idx=0x00000648

    Cisco Employee

    Can you also check this:

    run attach 0/<n>/cpu0

    se_p2mp_show
    <...>
    Dump options:
    <...>
    13 -> Show Netflow V9 Configuration

    Bronze

    Enter dump option <0-30>: 13

    Enter the coremask in hex:

    Any idea what coremask means?

    Cisco Employee

    hi Smail,

    apologies for the delay. Core mask means from which application instance you want to read the data. In this case want all 8 cores, so the mask should be 'ff'.

    Aleks

    Bronze

    Hi,

    thanks for the info.

    Here is the output.

    Enter the coremask in hex: ff
    Dump_option: 13, core_mask : FF
    P2MP Client Test: Rcvd msg from server
    Response of Core 0 :
    Status : 0
    Response :
    Response of Core 1 :
    Status : 0
    Response :
    Response of Core 2 :
    Status : 0
    Response :
    Response of Core 3 :
    Status : 0
    Response :
    Response of Core 4 :
    Status : 0
    Response :
    Response of Core 5 :
    Status : 0
    Response :
    Response of Core 6 :
    Status : 0
    Response :
    Response of Core 7 :
    Status : 0
    Response :
    p2mp_srvr_callback: Got1 connection error notification from P2MP Server
    p2mp_srvr_callback: Got1 connection error notification from P2MP Server
    P2MP Client Test: Closed P2MP library.

    Cisco Employee

    hi Smail,

    it seems the netflow confg was not properly picked up by the ISM. I think from this point it's best to go through a TAC SR. If you open one please share the ID.

    Aleks

    Bronze

    Here you go.

    SR 638066101

    New Member

    Hi, 

    Does a CGSE+ have to be configured with the same partition restrictions as a VSM? Say: ServiceApp1+ServiceApp2, ServiceApp3+ServiceApp4 and so on?

    Thanks!

    c.

    Cisco Employee

    Hi Carlos,

    the CGSE/CGSE+ don't have these restrictions. A pair of ServiceApps is enough.

    Cheers,

    N.

    New Member

    Thank you very much, Nicholas. We have big deployments that don't use such specific pairings and work perfectly, but TAC is stating the opposite. I appreciate your help.

    New Member

    Hi,

    Regarding Bulk Port Allocation. Once a pool of ports has been asigned to a CGNAT subscriber (say 2048 for BPA), it means ALL those 2048 ports are in fact reserved for that customer and cannot be used by any other customer at that time until all sessions within that range are terminated. is this assumption correct?

    If in fact it is correct, say I have a public pool of 1 single ip address, for example. Does that mean that, at most, i could have around 32 customers for that IP address since 2048 ports would be reserved for each on of them as soon as one session is active?

    The reason I ask is that in my case I'm seeing a lot of resource depletion errors in a card where most users are using around (tops) 400 sessions (ports) at any given time, so I'm thinking since a bulk-port-allocation of 2048 was configured, instead of having customers use only 400 ports per public IP address, the CGSE is "reserving" 2048 ports that aren't getting freed for a while until those 400 sessions actually are terminated.

    Am I reading this right? Any advice/comment would be greatly appeciated.

    Thanks,
    c.

    Cisco Employee

    Hey Carlos,

    Regarding Bulk Port Allocation. Once a pool of ports has been asigned to a CGNAT subscriber (say 2048 for BPA), it means ALL those 2048 ports are in fact reserved for that customer and cannot be used by any other customer at that time until all sessions within that range are terminated. is this assumption correct?

    >>> this is the correct understanding.

    If in fact it is correct, say I have a public pool of 1 single ip address, for example. Does that mean that, at most, i could have around 32 customers for that IP address since 2048 ports would be reserved for each on of them as soon as one session is active?

    >>> Again this is correct understanding.

    The reason I ask is that in my case I'm seeing a lot of resource depletion errors in a card where most users are using around (tops) 400 sessions (ports) at any given time, so I'm thinking since a bulk-port-allocation of 2048 was configured, instead of having customers use only 400 ports per public IP address, the CGSE is "reserving" 2048 ports that aren't getting freed for a while until those 400 sessions actually are terminated.

    >>> There is no straightforward answer for it. Here you need to access the port utilization manually if not using tools/equipment's like Prime.

    BPA or port limit can't be concluded without practical data to scrutinize.  BPA/Port limit is based on type of services in use, type of subscriber base [mobile/broadband] and total subscriber base.

    Once way is to pick random value and then optimize it as per network need. Other way is :- 

    1. Get the list of ports in use by each subscriber IP during peak hours. Take couple of snapshots.
    2. show cgn nat44 <> outside-translation for tcp, udp and for each external address. This will help you to know the ports in use.
    3. Separate outside port numbers, summarize these and then divide number of ports with configured BPA value. That way you will get average BPA usage value to further decide the correct appropriate value.

    Thanks

    Nitin Pabbi

    New Member

    Hi, Nitin:

    Thank you very much for such detailed answer.

    One quick follow up if you don't mind. IF you experience resource depletion, like we certainly have experienced: shouldn't (is there?) there be at syslog message or snmp related trap as a reaction to such an important event? We certainly didn't see or haven't been able to find either.

    Are you aware of such log message?

    Thanks,

    c.

    New Member

    Dear Nicolas

    Hope this mail finds you well

    I am in the process of implementing ABF redundancy for ISM

    I have read the article but I am looking for deeper configuration lines

    Can you help?

    BR,

    Mohammad

    Bronze

    Hi Mohammad,

    there is a post here from Aleksandar Vidakovic. There is a link with a great example from Xander. IP SLA should do the trick for you. 

    https://supportforums.cisco.com/discussion/11727191/ios-xr-abf-behavior

    Please check it out and get back to us if you have any issues.

    New Member

    Hi Smailmilak

    Thanks for the kind update

    I have implemented ABF before , am looking for ISM

    When I create the access-list , I should apply it to the interfaces where the 3G traffic comes in ?

    BR,

    Mohammad

    Bronze

    Correct.

    You should put it on the interface from where the traffic comes from.

    Is your A9K the gateway?

    Bronze

    Hi,

    we have several ISM-500 in our network. I know that 5.3.3 was the latest version for ISM and no other versions will be released.

    What about IOS-XR? Will Cisco drop support for ISM-500 on future releases i.e. we will not be able to use it at all? 

    Cisco Employee

    hi Smail,

    5.3.3 is the last release that supports A9K-ISM-100. Future XR releases won't support it. There is a programme for ISM to VSM migration. Please check the details of this programme with your (or your end customer's) account team at Cisco.

    /Aleksandar

    Bronze

    Hi, 

    we are working on it but it will take some time.

    So you are saying that the ISM-500 will not work at all on future XR releases?

    For now it's working on 6.0.x

    http://www.cisco.com/c/en/us/td/docs/routers/asr9000/software/asr9k_r6-0/cg-nat/configuration/guide/b-cgnat-cg60xasr9k/b_cgnat_cg53xasr9k_chapter_011.html

    Cisco Employee

    Hi,

    So you are saying that the ISM-500 will not work at all on future XR releases?

    >> yes, future release don't have support of ISM-100 card.

    We have VSM-500 as n alternative for ISM-100 card on ASR9000 series routers [except AR9001 router]

    5.3.3 is the last release for engineering support of this card.

    ---------------------------------

    For now it's working on 6.0.x

    http://www.cisco.com/c/en/us/td/docs/routers/asr9000/software/asr9k_r6-0...

    >>> Thanks for highlight this miss. We will correct this document ASAP.

    Thanks

    Nitin Pabbi

    Bronze

    Thank you both for the information. I have to inform our customer right now about this.

    We are upgrading to 5.3.4 because of a bug and we have to stay on it until we have finished the migration from ISM-100 to VSM-500.

    Cisco Employee

    Few comments "show cgn nat44 <> statistics" in section 29.

    Number of active translations- Number of NAT DB (Static translation + Dynamic translation) entries. This value indicates the number of translations active in the database. NAT DB is also alternatively called Main DB in few places. NAT DB and Session DB are databases within the application.

    Number of sessions - Number of Session DB (Static translation + Dynamic translation) entries. If multiple translations (more than one) with same source IP:source port pair are created, such translations are entered in session db to ensure next translations get same translated source IP:source address pair. Note that the session value for a given source ip:port pair can increase from zero, only when more than one translation is getting created for the same pair. This value is not equivalent to number of active translations.

    Bronze

    Hi,

     

    is Paired-Address-Pooling supported on ISM-100 with CGN software? I know only about many-to-one but customer is still complaining that some IPTV services are not working because the same customer is reaching the services with different public IP's

     

    I see that it's supported on IOS-XE

    https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/ipaddr_nat/configuration/xe-16/nat-xe-16-book/iadnat-addr-pool.html

     

    9438
    Views
    5
    Helpful
    49
    Comments