High Memory Utilization due to NAT

Hitesh Vinzoda · ‎08-09-2009

NAT configured on 3845 with 12.4.24 T ADV ENT SERVICES

* Have got 64 /25 inside subnets to do the nat with 64 Live IP's. one each for /25 inside subnet.

* I checked the processes and memory on freshly loaded router which comes out to be 49 MB of free memory.

* started the NAT on router with 8 of /25 inside ip pool with policy NAT to 8 live IP's. The router withing 3 hours hanged due to no availability of free memory. Rebooted it and removed the NAT.

* Checked Cisco website for NAT it says 312 bytes per translation that gives us around 3 MB for 10000 translations. Checked the logs and found peak translation only to be 15000.

* Found that problem was NAT ACL with any statement in destination portion ( extended one). Changed it with standard ACL with no any statement.

* Reviewed and resumed the NAT on router. it works now but it uses around 20 MB of memory for just 10000 translation entries.

* Checked the UDP, TCP and ICMP timeout .... Limited UDP to 4 Mins. TCP to 25 Mins and ICMP- 5 Mins. was able to free only 2 MB of so from 20 MB.

* Changed the IOS from ADV ent services to IP base to get rid of unwanted processess and services as main AIM of this router is to run NAT.

* Freshly loaded router gave me 120 MB of free space and was happy now to test out the things.

* Againg started the NAT for 8 pools of /25 inside subnet with 8 live IP's ( Policy nat ).

* At 25000 translations it eats up memory of around 24 MB.

* Turned of Virtual Reassembly as it was reaching to thresold very often.

* Migrated another 8 pools of /25 which comes to total of 16 /25 Inside subnets and free memory left to 64 MB. with the peak translation upto 42000 and active translation to 15000 on an average.

* It often gives the I/O memory errors too ( with only 16 /25 Pools configured on it).

* All this stuff works fine with Netscreen firewall overloaded with only 4 IP's for all 64 /25 pools. ..... ( Is netscreen had an edge over cisco when it comes to NAT ...._?? ) I wonder..!

If Cisco says that only 312 bytes are required for storing a single translation Why i m not able to free my DRAM memory. Tried my luck with everything. Need some expert advice on this to figure out the High Memory usage of NAT....

NOTE : Only default router and no other services are used on router apart from Netflow

Edison Ortiz · ‎08-09-2009

You should give 12.4(15)T9 a try and it's more stable than 12.4(24)T.

If that fails, I recommend opening a TAC case for further troubleshooting.

HTH,

__

Edison.

Hitesh Vinzoda · ‎08-09-2009

I heard from someone that 3845 does the NAT in IOS and best solution will be selecting a router which does NAT in hardware.

I'm having 256 MB of DRAM in 3845. does 50000 translation with a peak will work on 3845 ? Because only 120 MB of RAM is left after the router is freshly booted up.

Edison Ortiz · ‎08-09-2009

We've seen memory leaks with NAT under the IOS you are using, can you please give 12.4(15)T9 a try and repost?

Also, NetFlow consumes a lot of memory so can you try turning off NetFlow and see if the memory consumption helps?

BTW, NAT runs in IOS in all routers.

HTH,

__

Edison.

Hitesh Vinzoda · ‎08-11-2009

Thanks for your advice Edison..

I ll give a try with 12.4(15) T9 IPbase and will let you know.

Regards

Hitesh Vinzoda

Hitesh Vinzoda · ‎08-16-2009

Hi Edison,

I have checked the memory leaks on 12.4(24)T IPbase with following commands and they show no memory leaks.

"show memory debug leaks"

"show memory debug leaks summary"

Do you still think the problem is with memory leaks?

Regards

Hitesh Vinzoda