NTP fails to synchronize

Unanswered Question
Feb 23rd, 2007

Greetings.

We have a 2811 in a remote location which connects to our core network using IPSEC tunnels over the Internet.

We have NTP configured pointing to four different servers. The servers are reachable from the router through the IPSEC tunnels no problem. We can trace to these NTP servers, and from these NTP servers without issue.

NTPO fils to synchronize with any of these servers. I have tried removing the confguration, confguring public NTP servers, but nothing works.

Debug NTP packets shows the packets being xmit'ted, but nothing returning, or rather nothing hitting the NTP listening port.

I'd appreciate any ideas here short of rebooting the router.

Regards,

Joe

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 3 (2 ratings)
Loading.
purohit_810 Fri, 02/23/2007 - 08:00

Hi,

First can you check on router 2811 hit, with filter NTP packet.

NTP is using 123 Port no. Former we were using NTP port 1024. Also please put and allow that port once and check.

CISCO HAS GIVEN IN ONE DOCUMENT PLEASE REFER:

{{

CSCed13205

SDM does not issue the ntp update-calendar IOS command on Cisco 7200 routers if there are no new settings to enter and if the Network Time Protocol (NTP) server was configured using the CLI and only one NTP server IP address was provided and no ntp update-calendar IOS command was present in the running configuration.

Workaround:

Use SDM to delete the NTP server configuration entry, click Refresh, and then recreate the entry, or make changes to the existing NTP server entry.

_____________________________________________

CSCed30721

Whenever any unconfigured interface contains the description $FW_INSIDE$, on a router configured with a firewall, adding a new NTP server will not modify the firewall ACLs to allow NTP passthrough traffic. Instead, when the user edits the firewall's outside interface in the Interfaces and Connections window, SDM prompts the user to add the NTP passthrough traffic.

Workaround: Use the CLI to manually remove the description $FW_INSIDE$ from the unconfigured interface.

}}

______________________________________________

Regards,

Dharmesh Purohit

ziutek Fri, 02/23/2007 - 08:06

There are no ACLs involved. All traffic goes to our core via IPSEC tunnels. There are no FWs between the IPSEC endpoints and the NTP servers.

debug NTP packet shows constant xmits, but no rcv's:

.Feb 23 16:04:35.379: NTP: xmit packet to 17.254.0.28:

.Feb 23 16:04:35.379: leap 3, mode 3, version 3, stratum 0, ppoll 64

.Feb 23 16:04:35.379: rtdel 0000 (0.000), rtdsp 0002 (0.031), refid 7F7F0701 (127.127.7.1)

.Feb 23 16:04:35.379: ref C9897FBB.6118DFD4 (14:59:07.379 GMT Fri Feb 23 2007)

.Feb 23 16:04:35.379: org 00000000.00000000 (00:00:00.000 GMT Mon Jan 1 1900)

.Feb 23 16:04:35.379: rec 00000000.00000000 (00:00:00.000 GMT Mon Jan 1 1900)

.Feb 23 16:04:35.379: xmt C9898F13.611922A0 (16:04:35.379 GMT Fri Feb 23 2007)

Working connection:

Feb 23 16:57:32.524: NTP: xmit packet to 128.127.3.189:

Feb 23 16:57:32.524: leap 0, mode 3, version 3, stratum 4, ppoll 128

Feb 23 16:57:32.524: rtdel 5636 (336.761), rtdsp 2030 (125.732), refid 807F03BD (128.127.3.189)

Feb 23 16:57:32.524: ref C9898CEC.AFD9BF6C (16:55:24.686 CET Fri Feb 23 2007)

Feb 23 16:57:32.524: org C9898CEC.9B42AA9F (16:55:24.606 CET Fri Feb 23 2007)

Feb 23 16:57:32.524: rec C9898CEC.AFD9BF6C (16:55:24.686 CET Fri Feb 23 2007)

Feb 23 16:57:32.524: xmt C9898D6C.864342FA (16:57:32.524 CET Fri Feb 23 2007)

Feb 23 16:57:32.688: NTP: rcv packet from 128.127.3.189 to 172.30.254.140 on Loopback1: 

It's almost as if the machine has stopped listening on the NTP port.

Richard Burts Fri, 02/23/2007 - 08:12

Joe

Is it possible that the head end routers which terminate the IPSec tunnels are not treating NTP as traffic that should be protected by IPSec?

My other questions would have to do with how NTP is configured:

- are you using authentication for NTP? If so is that possibly a fault in how authentication is configured for the remote router?

- are you using the option of access lists within NTP (access-group peer or access-group serve-only)?

HTH

Rick

purohit_810 Fri, 02/23/2007 - 08:17

Great... Your out put said good things.

I would require more two commands output. Can you give me.

sh ntp status

sh ntp associations detail

Waiting your reply.

Regards,

Dharmesh Purohit

purohit_810 Fri, 02/23/2007 - 08:09

Also Look on below link....

http://www.cisco.com/en/US/products/sw/iosswrel/ps5207/products_feature_guide09186a008028b448.html#wp1048527

System Clock and Network Time Protocol (NTP)

The Cisco 1800, 2800 and 3800 series modular routers include on-board hardware clocks. System software clocks are synchronized to the hardware clocks at boot time or in accordance with configuration parameters, such as NTP. The kron facility can be used for periodic synchronization of hardware and software clocks. The following configuration example shows how to enable the clocks to periodically synchronize:

kron occurrence CLOCK-SYNC5 in 5 recurring

policy-list CLOCK-SYNC

!

kron policy-list CLOCK-SYNC

cli clock read-calendar

!

NTP provides a synchronized time base and allows for the analysis of log data to understand and troubleshoot the time sequence of network events. Used in conjunction with the Cisco IOS Firewall, NTP enables the comparison of logs from different network devices and services, the comparison of which is essential for tracking security incidents and troubleshooting. Without precise time synchronization among all the various logging, debug output, management, and AAA functions in the network, you cannot make time comparisons.

For more information about Cisco IOS Network Time Protocol (NTP) service, see the following URL:

http://www.cisco.com/en/US/products/sw/iosswrel/ps1835/products_configuration_guide_chapter09186a00800ca750.html#41044

In addition to the URL listed above, note the following information regarding Cisco IOS NTP:

?All Cisco IOS routers that are certified are capable of running NTP from the Cisco IOS software.

?When using the no clock initialize nvram command, the router will always start with the reference time of March 1, 2002 00:00:00.

Regards,

Dharmesh Purohit

ziutek Fri, 02/23/2007 - 10:32

NTP is being sourced from a lo1. This loopback is advertised and reachable only through the IPSEC tunnel.

At the moment we have configured the machine with a public and private ntp server.

jer1rt01#sh nt ass de

128.127.51.189 configured, insane, invalid, unsynced, stratum 16

ref ID 0.0.0.0, time 00000000.00000000 (00:00:00.000 GMT Mon Jan 1 1900)

our mode client, peer mode unspec, our poll intvl 64, peer poll intvl 64

root delay 0.00 msec, root disp 0.00, reach 0, sync dist 0.000

delay 0.00 msec, offset 0.0000 msec, dispersion 16000.00

precision 2**5, version 3

org time 00000000.00000000 (00:00:00.000 GMT Mon Jan 1 1900)

rcv time 00000000.00000000 (00:00:00.000 GMT Mon Jan 1 1900)

xmt time C989B0B9.614361BC (18:28:09.379 GMT Fri Feb 23 2007)

filtdelay = 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

filtoffset = 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

filterror = 16000.0 16000.0 16000.0 16000.0 16000.0 16000.0 16000.0 16000.0

17.254.0.28 configured, insane, invalid, unsynced, stratum 16

ref ID 0.0.0.0, time 00000000.00000000 (00:00:00.000 GMT Mon Jan 1 1900)

our mode client, peer mode unspec, our poll intvl 64, peer poll intvl 64

root delay 0.00 msec, root disp 0.00, reach 0, sync dist 0.000

delay 0.00 msec, offset 0.0000 msec, dispersion 16000.00

precision 2**5, version 3

org time 00000000.00000000 (00:00:00.000 GMT Mon Jan 1 1900)

rcv time 00000000.00000000 (00:00:00.000 GMT Mon Jan 1 1900)

xmt time C989B093.6121CD7E (18:27:31.379 GMT Fri Feb 23 2007)

filtdelay = 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

filtoffset = 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

filterror = 16000.0 16000.0 16000.0 16000.0 16000.0 16000.0 16000.0 16000.0

jer1rt01#sh ntp stat

jer1rt01#sh ntp status

Clock is unsynchronized, stratum 16, no reference clock

nominal freq is 250.0000 Hz, actual freq is 250.0000 Hz, precision is 2**18

reference time is C9897FBB.6118DFD4 (14:59:07.379 GMT Fri Feb 23 2007)

clock offset is 0.0000 msec, root delay is 0.00 msec

root dispersion is 0.02 msec, peer dispersion is 0.02 msec

Neither server will synchronize.

Like I said, it appears that ntp has just stopped listening.

From debug:

jer1rt01#

.Feb 23 18:30:17.379: NTP: xmit packet to 128.127.51.189:

.Feb 23 18:30:17.379: leap 3, mode 3, version 3, stratum 0, ppoll 64

.Feb 23 18:30:17.379: rtdel 0000 (0.000), rtdsp 0002 (0.031), refid 7F7F0701 (127.127.7.1)

.Feb 23 18:30:17.379: ref C9897FBB.6118DFD4 (14:59:07.379 GMT Fri Feb 23 2007)

.Feb 23 18:30:17.379: org 00000000.00000000 (00:00:00.000 GMT Mon Jan 1 1900)

.Feb 23 18:30:17.379: rec 00000000.00000000 (00:00:00.000 GMT Mon Jan 1 1900)

.Feb 23 18:30:17.379: xmt C989B139.61214164 (18:30:17.379 GMT Fri Feb 23 2007)

.Feb 23 18:30:17.379: NTP: 128.127.51.189 unreachable

jer1rt01#

.Feb 23 18:30:43.379: NTP: xmit packet to 17.254.0.28:

.Feb 23 18:30:43.379: leap 3, mode 3, version 3, stratum 0, ppoll 64

.Feb 23 18:30:43.379: rtdel 0000 (0.000), rtdsp 0002 (0.031), refid 7F7F0701 (127.127.7.1)

.Feb 23 18:30:43.379: ref C9897FBB.6118DFD4 (14:59:07.379 GMT Fri Feb 23 2007)

.Feb 23 18:30:43.379: org 00000000.00000000 (00:00:00.000 GMT Mon Jan 1 1900)

.Feb 23 18:30:43.379: rec 00000000.00000000 (00:00:00.000 GMT Mon Jan 1 1900)

.Feb 23 18:30:43.379: xmt C989B153.6121E158 (18:30:43.379 GMT Fri Feb 23 2007)

jer1rt01#trace 128.127.51.189

Type escape sequence to abort.

Tracing the route to nyc1util01.routers.citco.com (128.127.51.189)

1 tun111.gen0rt06.routers.citco.com (172.31.20.97) 64 msec 68 msec 68 msec

2 tu2129.nyc1rt06.routers.citco.com (172.28.0.9) 168 msec 168 msec 172 msec

3 vlan110.nyc1l3-01.routers.citco.com (172.25.0.11) 168 msec 176 msec 200 msec

4 nyc1util01.routers.citco.com (128.127.51.189) 168 msec 172 msec 172 msec

jer1rt01#

NTP says the IP is unreachable, but I successfully traced to it.

Thanks for the input.

Joe

sundar.palaniappan Fri, 02/23/2007 - 11:47

Joe,

I see your trace from the 2800 worked fine but did you ping/trace to the server sourcing the loopback1 address?

HTH

Sundar

purohit_810 Fri, 02/23/2007 - 13:33

can you check one more thing, where From where NTP device is located (At remote end), Is there NTP protocol open or not.

This is only last posibility at network end.

Else server not responding.

Also check NTP authentication.

Regards,

Dharmesh Purohit

Danilo Dy Fri, 02/23/2007 - 16:23

Hi,

Check the following;

- NTP Service running in the server

- NTP Service version running in the server

- TCP/UDP 123 is open in the firewall or ACL (server or router)

- TCP/UDP 123 is allowed in VPN Tunnel

- NTP authentication?

- ACL for NTP sync (ntp access-group)?

- NTP source (loopback)?

Your traceroute shows Tunnel interfaces. Thus the Tunnel interface MTU is set correctly?

ziutek Sat, 02/24/2007 - 00:43

Trace sourced from the lo1:

jer1rt01#trace ip

Target IP address: 128.127.51.189

Source address: 172.30.254.65

Numeric display [n]:

Timeout in seconds [3]:

Probe count [3]:

Minimum Time to Live [1]:

Maximum Time to Live [30]:

Port Number [33434]:

Loose, Strict, Record, Timestamp, Verbose[none]:

Type escape sequence to abort.

Tracing the route to nyc1util01.routers.citco.com (128.127.51.189)

1 tun111.gen0rt06.routers.citco.com (172.31.20.97) 68 msec 68 msec 64 msec

2 tu2129.nyc1rt06.routers.citco.com (172.28.0.9) 172 msec 172 msec 172 msec

3 vlan110.nyc1l3-02.routers.citco.com (172.25.0.12) 168 msec 172 msec 172 msec

4 nyc1util01.routers.citco.com (128.127.51.189) 172 msec 172 msec 172 msec

jer1rt01#

NTP is running fine on the server as we have more than 500 devices using this for NTP.

No FW or ACL blocking.

All IP is allowed through the tunnel.

No NTP authentication.

No ACL for NTP. Have not needed or used this on other boxes.

MTU on the tunnels appears to be fine since OSPF adjacencies form without issue.

Just abour ready for a reboot.

Joe

Actions

This Discussion