DNS server process, high CPU usage

[aleksandar.ciric]

Hello,

I am having a problem with high CPU usage on DNS process. We have 1921 setup in network of about 100 workstations that use it as DNS server. It also performs split view functionality for couple of domains in order to avoid hairpin NAT. I know that a router is not fit for heavy server duty but such setup was unfortunately necessary.

In peak work hours we oftenly have CPU hitting 100%  with DNS server process being the root cause,and on several occasions we had DNS process stuck in such state for prolonged periods of time during which other router functions were affected (t's a default gateway, IPsec concentrator etc.). Sometimes shutting the process and restarting it would solve the problem, but once it required a full reload in order to restore functionality (such was the description that engineer on duty reported to me).

Forwarder queue statistics:

Current size = 0

Maximum size = 86

Drops        = 0

IOS upgrade did not help, we are currently running 15.2(2)T.

The question is ... is such behaviour (CPU load) to be expected on LAN with 100 workstations due to slow CPU on router or we have bugged IOS DNS server (requiring a TAC case)?

The previous solution seem to do this functionality quite nicely (even though it was also a router), so I am not inclined to think that we are dealing with someone DoS-ing the DNS (WAN access to DNS is of course forbidden)

P.S.

Since we moved servers off the routers DNS, we do not receive complaints, but we had a couple of unresolved messages a day while mail server was using router for DNS. I am suspecting that an old bug where IOS DNS servers occasionaly sends clients back empty DNS replies (properly formatted message but without A record) could be still around?

[jwbensley]

Hi Aleksandar,

Did you ever find a resolution to your problem? I have just experianed a sustined high CPU usage of 40% on a production 1841 for the past 16 hours, because of the DNS Server process. (I know 40% isn't that high but this router averages 4%~ so its high in context).

I have rebooted the device and all "seems" well. Its running  c1841-adventerprisek9-mz.124-25e.bin.

Did you simply reboot yours two, or find a cause and solution?

Thanks.

[lamnguyen2010]

hi,

i have the same problem on UC540 and 2911 on IOS 15.0.1. CPU high, router crashed. Reboot helps but until CPU usage spikes again. the only fix is to remove "ip dns server" from config and use a different DNS server

                    111                       1111    1111111111
          999999999900077777777799999777778888000099990000000000
    322222111119999900077776666622222111119999000088880000000000
100            ********                       ******************
 90       *************         *****     **********************
 80       ***************************     **********************
 70       ******************************************************
 60       ******************************************************
 50       ******************************************************
 40       ******************************************************
 30       ******************************************************
 20       ******************************************************
 10       ******************************************************
   0....5....1....1....2....2....3....3....4....4....5....5....6
             0    5    0    5    0    5    0    5    0    5    0
               CPU% per second (last 60 seconds)

after removed IP dns server

                                       111                      
                             99999999990007777777779999977777888
              33333333332222211111999990007777666662222211111999
100                               ********                      
 90                          *************         *****     ***
 80                          ***************************     ***
 70                          ***********************************
 60                          ***********************************
 50                          ***********************************
 40                          ***********************************
 30                          ***********************************
 20                          ***********************************
 10                          ***********************************
   0....5....1....1....2....2....3....3....4....4....5....5....6
             0    5    0    5    0    5    0    5    0    5    0
               CPU% per second (last 60 seconds)

[adepojutayo]

i cannot say I have the exact solution but I can tell u what worked for me today. you can check the configuration for your router and find out if it is acting as a dns server for both your internal and external network traffic. That was my issue. 

To resolve it, I denied dns inbound on the internet interface by adding a simple

deny tcp any any eq 53

deny udp any any eq 53

... this is because it seems your router is not only resolving internal dns queries but also responding to external dns queries as well. I really do hope it helps!! Good luck 

https://learningnetwork.cisco.com/thread/85489

[Petar Bajovic]

Hi,

Did anyone find a solution for this problem? I have similar problem on my VPN-NAS (Network Access Server) which is Cisco 1921 with 15.0(1r)M12 IOS. 

Problem: When I turn on "ip dns server" on my router, my CPU usage "hit the roof" (99%). I can not debug DNS then, because of high CPU usage. Is this some kind of a bug on this version of IOS?

If you need more info about this problem, I'm here. I really want to find a solution for this problem. 

Thank you,

Petar

[Daniel Ulreich]

Hi Petar,

I had the CPU load problem with an 2921 running IOS 15.5 in my lab too.

After enabling ip cef everything was fine and the load went down to a maximum of 5%

Daniel

[MaxUng80377]

Have you found a solution yet? 

I have a router is having the same issue. The DNS Server is at high CPU usage and I cannot switch it off because clients are using the router as the DNS server.