Time travel?

Answered Question
Aug 20th, 2010
User Badges:

Hi,

  Noticed something a little strange. We have 3 Ironport boxes running in same location, all taking time settings from the NTP server time.ironport.com, but all running on different times, i.e. 10:30, 10:38 and 10:46... anyone else notice this?

Correct Answer by Christopher Smith about 6 years 10 months ago

Greetings David,


There can be several reasons why this can occur. To understand the specifics of the issue I would recommend starting the the NTP logs on each of these appliances.



The  most common NTP issue is when the IronPort Appliance cannot communicate  with the NTP servers that were defined.  When this happens, you will  see timeouts in the NTP logs:

Tue May 11 13:48:03 2010 Warning: timeout while waiting for ntp response IP 10.1.123.10
Tue May 11 13:49:06 2010 Warning: timeout while waiting for ntp response IP 10.1.123.11
Tue May 11 13:50:09 2010 Warning: timeout while waiting for ntp response IP 10.1.123.10
Tue May 11 13:51:12 2010 Warning: timeout while waiting for ntp response IP 10.1.123.11

The timeout indicates that either the NTP servers defined are not  answering NTP queries on UDP port 123, there is no route to the NTP  servers from the Ironport, or there is a Firewall, or some other network  device, blocking UDP port 123.

Please check to see if your NTP  servers are accepting NTP requests.  Also check that your Network allows  UDP port 123 between the IronPort Appliances and your NTP servers or  the Internet Time servers you have configured.  

Cannot resolve Time Server name:
If your IronPort Appliance cannot resolve a DNS name for a time server, you will get a log entry like this:

Mon May 17 22:44:51 2010 Warning: DNS Failure looking up "tick.example": ('tick.example', 'A', (3, 'NXDomain'))

Make sure your fully qualified domain name is correct, and make sure the  IronPort Appliance can resolve that name by using the NSLOOKUP command.   

Dueling NTP servers:
If  you have more than one NTP server defined, and they are serving  different times (that is, the NTP servers themselves are not in synch),  you will see the IronPort logs switching the time back and forth:

Wed Mar 17 10:24:23 2010 Info: sntp query host 10.192.25.61 delay 236 offset -502564820
Wed Mar 17 10:24:23 2010 Info: time stepped:  -502564820 us 
Wed Mar 17 10:16:05 2010 Info: sntp query host 10.192.25.62 delay 259 offset 502648713 
Wed Mar 17 10:16:05 2010 Info: time stepped: 502648713 us 
Wed Mar 17 10:24:32 2010 Info: sntp query host 10.192.25.61 delay 225 offset -502663731
Wed Mar 17 10:24:32 2010 Info: time stepped: -502663731 us
Wed Mar 17 10:17:16 2010 Info: sntp query host 10.192.25.62 delay 277 offset 502601949
Wed Mar 17 10:17:16 2010 Info: time stepped: 502601949 us

To fix this issue, determine which of your NTP servers is providing the wrong time, and get it in synch.  


Time step too large:
If the time on the system is off by six months or more, you will get this message in the logs:

Wed Nov 11 21:26:04 2009 Info: sntp query host 10.92.151.132 delay 510 offset 16158559567723
Wed Nov 11 21:26:04 2009 Warning: NTP time step too large: 187 days  (server tock.example ip 10.92.151.132) [manual intervention required]

To fix this issue, manually set the time and date and then re-enable NTP:

vmw033-esa07.run> settime

WARNING: Changes to system time will take place immediately and do not require the user to run the commit command.

This machine is currently running NTP.  In order to manually set the time, NTP must be disabled.
Do you want to stop NTP and manually set the time? [N]>

Current time Wed Nov 11 21:27:35 2009 UTC.
Please enter the time in MM/DD/YYYY HH:MM:SS format. []> 11/30/2009 22:11:00 

Time set to Mon Nov 30 22:11:00 2009 UTC.
vmw033-esa07.run>

vmw033-esa07.run> ntpconfig 

Currently configured NTP servers:
No servers currently configured.

Choose the operation you want to perform:
- NEW - Add a server.
- SOURCEINT - Set the interface from whose IP address NTP queries should originate.
[]> new
Please enter the fully qualified hostname or IP address of your NTP server.
[]> tock.example 

Currently configured NTP servers:
1. tock.example 

Choose the operation you want to perform:
- NEW - Add a server.
- DELETE - Remove a server.
- SOURCEINT - Set the interface from whose IP address NTP queries should originate.
[]> 

vmw033-esa07.run> commit


Coming into synch:
Successful  NTP queries will also be logged.  Note: the longer the NTP process has  been running, the less frequent time checks are.  Do not be concerned  that there are time adjustments each for each update.  This is typical  of NTP:

Tue May 18 09:25:40 2010 Info: sntp query host 10.92.151.132 delay 526 offset -42979543404
Tue May 18 09:25:40 2010 Info: time stepped: -42979543404 us
Mon May 17 21:29:25 2010 Info: sntp query host 10.92.151.132 delay 505 offset 414
Mon May 17 21:29:25 2010 Info: adjust: time_const: 1 offset: 414us next_poll: 32
Mon May 17 21:30:01 2010 Info: sntp query host 10.92.151.132 delay 495 offset -14316
Mon May 17 21:30:01 2010 Info: adjust: time_const: 1 offset: -14316us next_poll: 32
Mon May 17 21:30:37 2010 Info: sntp query host 10.92.151.132 delay 492 offset -20855
Mon May 17 21:30:37 2010 Info: adjust: time_const: 1 offset: -20855us next_poll: 32
Mon May 17 21:31:13 2010 Info: sntp query host 10.92.151.132 delay 553 offset -21061
Mon May 17 21:31:13 2010 Info: adjust: time_const: 1 offset: -21061us next_poll: 32
Mon May 17 21:31:49 2010 Info: sntp query host 10.92.151.132 delay 525 offset -19240
Mon May 17 21:31:49 2010 Info: adjust: time_const: 1 offset: -19240us next_poll: 32
Mon May 17 21:32:26 2010 Info: sntp query host 10.92.151.132 delay 513 offset -16589
Mon May 17 21:32:26 2010 Info: adjust: time_const: 1 offset: -16589us next_poll: 32
Mon May 17 21:33:02 2010 Info: sntp query host 10.92.151.132 delay 494 offset -14659
Mon May 17 21:33:02 2010 Info: adjust: time_const: 1 offset: -14659us next_poll: 32
Mon May 17 21:33:38 2010 Info: sntp query host 10.92.151.132 delay 537 offset -12772
Mon May 17 21:33:38 2010 Info: adjust: time_const: 2 offset: -12772us next_poll: 64
Mon May 17 21:34:46 2010 Info: sntp query host 10.92.151.132 delay 470 offset -18918
Mon May 17 21:34:46 2010 Info: adjust: time_const: 2 offset: -18918us next_poll: 64


For more information about NTP, see the AsyncOS Configuration Guide on the IronPort Support Portal.




Christopher C Smith

CSE

Cisco IronPort Customer Support

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (1 ratings)
Loading.
Correct Answer
Christopher Smith Fri, 08/20/2010 - 06:53
User Badges:
  • Cisco Employee,

Greetings David,


There can be several reasons why this can occur. To understand the specifics of the issue I would recommend starting the the NTP logs on each of these appliances.



The  most common NTP issue is when the IronPort Appliance cannot communicate  with the NTP servers that were defined.  When this happens, you will  see timeouts in the NTP logs:

Tue May 11 13:48:03 2010 Warning: timeout while waiting for ntp response IP 10.1.123.10
Tue May 11 13:49:06 2010 Warning: timeout while waiting for ntp response IP 10.1.123.11
Tue May 11 13:50:09 2010 Warning: timeout while waiting for ntp response IP 10.1.123.10
Tue May 11 13:51:12 2010 Warning: timeout while waiting for ntp response IP 10.1.123.11

The timeout indicates that either the NTP servers defined are not  answering NTP queries on UDP port 123, there is no route to the NTP  servers from the Ironport, or there is a Firewall, or some other network  device, blocking UDP port 123.

Please check to see if your NTP  servers are accepting NTP requests.  Also check that your Network allows  UDP port 123 between the IronPort Appliances and your NTP servers or  the Internet Time servers you have configured.  

Cannot resolve Time Server name:
If your IronPort Appliance cannot resolve a DNS name for a time server, you will get a log entry like this:

Mon May 17 22:44:51 2010 Warning: DNS Failure looking up "tick.example": ('tick.example', 'A', (3, 'NXDomain'))

Make sure your fully qualified domain name is correct, and make sure the  IronPort Appliance can resolve that name by using the NSLOOKUP command.   

Dueling NTP servers:
If  you have more than one NTP server defined, and they are serving  different times (that is, the NTP servers themselves are not in synch),  you will see the IronPort logs switching the time back and forth:

Wed Mar 17 10:24:23 2010 Info: sntp query host 10.192.25.61 delay 236 offset -502564820
Wed Mar 17 10:24:23 2010 Info: time stepped:  -502564820 us 
Wed Mar 17 10:16:05 2010 Info: sntp query host 10.192.25.62 delay 259 offset 502648713 
Wed Mar 17 10:16:05 2010 Info: time stepped: 502648713 us 
Wed Mar 17 10:24:32 2010 Info: sntp query host 10.192.25.61 delay 225 offset -502663731
Wed Mar 17 10:24:32 2010 Info: time stepped: -502663731 us
Wed Mar 17 10:17:16 2010 Info: sntp query host 10.192.25.62 delay 277 offset 502601949
Wed Mar 17 10:17:16 2010 Info: time stepped: 502601949 us

To fix this issue, determine which of your NTP servers is providing the wrong time, and get it in synch.  


Time step too large:
If the time on the system is off by six months or more, you will get this message in the logs:

Wed Nov 11 21:26:04 2009 Info: sntp query host 10.92.151.132 delay 510 offset 16158559567723
Wed Nov 11 21:26:04 2009 Warning: NTP time step too large: 187 days  (server tock.example ip 10.92.151.132) [manual intervention required]

To fix this issue, manually set the time and date and then re-enable NTP:

vmw033-esa07.run> settime

WARNING: Changes to system time will take place immediately and do not require the user to run the commit command.

This machine is currently running NTP.  In order to manually set the time, NTP must be disabled.
Do you want to stop NTP and manually set the time? [N]>

Current time Wed Nov 11 21:27:35 2009 UTC.
Please enter the time in MM/DD/YYYY HH:MM:SS format. []> 11/30/2009 22:11:00 

Time set to Mon Nov 30 22:11:00 2009 UTC.
vmw033-esa07.run>

vmw033-esa07.run> ntpconfig 

Currently configured NTP servers:
No servers currently configured.

Choose the operation you want to perform:
- NEW - Add a server.
- SOURCEINT - Set the interface from whose IP address NTP queries should originate.
[]> new
Please enter the fully qualified hostname or IP address of your NTP server.
[]> tock.example 

Currently configured NTP servers:
1. tock.example 

Choose the operation you want to perform:
- NEW - Add a server.
- DELETE - Remove a server.
- SOURCEINT - Set the interface from whose IP address NTP queries should originate.
[]> 

vmw033-esa07.run> commit


Coming into synch:
Successful  NTP queries will also be logged.  Note: the longer the NTP process has  been running, the less frequent time checks are.  Do not be concerned  that there are time adjustments each for each update.  This is typical  of NTP:

Tue May 18 09:25:40 2010 Info: sntp query host 10.92.151.132 delay 526 offset -42979543404
Tue May 18 09:25:40 2010 Info: time stepped: -42979543404 us
Mon May 17 21:29:25 2010 Info: sntp query host 10.92.151.132 delay 505 offset 414
Mon May 17 21:29:25 2010 Info: adjust: time_const: 1 offset: 414us next_poll: 32
Mon May 17 21:30:01 2010 Info: sntp query host 10.92.151.132 delay 495 offset -14316
Mon May 17 21:30:01 2010 Info: adjust: time_const: 1 offset: -14316us next_poll: 32
Mon May 17 21:30:37 2010 Info: sntp query host 10.92.151.132 delay 492 offset -20855
Mon May 17 21:30:37 2010 Info: adjust: time_const: 1 offset: -20855us next_poll: 32
Mon May 17 21:31:13 2010 Info: sntp query host 10.92.151.132 delay 553 offset -21061
Mon May 17 21:31:13 2010 Info: adjust: time_const: 1 offset: -21061us next_poll: 32
Mon May 17 21:31:49 2010 Info: sntp query host 10.92.151.132 delay 525 offset -19240
Mon May 17 21:31:49 2010 Info: adjust: time_const: 1 offset: -19240us next_poll: 32
Mon May 17 21:32:26 2010 Info: sntp query host 10.92.151.132 delay 513 offset -16589
Mon May 17 21:32:26 2010 Info: adjust: time_const: 1 offset: -16589us next_poll: 32
Mon May 17 21:33:02 2010 Info: sntp query host 10.92.151.132 delay 494 offset -14659
Mon May 17 21:33:02 2010 Info: adjust: time_const: 1 offset: -14659us next_poll: 32
Mon May 17 21:33:38 2010 Info: sntp query host 10.92.151.132 delay 537 offset -12772
Mon May 17 21:33:38 2010 Info: adjust: time_const: 2 offset: -12772us next_poll: 64
Mon May 17 21:34:46 2010 Info: sntp query host 10.92.151.132 delay 470 offset -18918
Mon May 17 21:34:46 2010 Info: adjust: time_const: 2 offset: -18918us next_poll: 64


For more information about NTP, see the AsyncOS Configuration Guide on the IronPort Support Portal.




Christopher C Smith

CSE

Cisco IronPort Customer Support

Actions

This Discussion