NTP issues on 2800 router?

Cory Dryden · ‎12-06-2011

Hi guys,

Have some questions regarding NTP. I have two routers, one is synched the other will not to the same server. Both routers have the same static routes configured.

Also I have four NTP servers listed and only one appears sane. I have put the ouput of sh ntp associations detail below. I am new to NTP so maybe they are not suppose to become synched unless the first goes down? But I am not sure. Any help would be great.

203.221.91.147 configured, insane, invalid, unsynced, stratum 16
ref ID 0.0.0.0, time 00000000.00000000 (10:00:00.000 QLD Mon Jan 1 1900)
our mode client, peer mode unspec, our poll intvl 1024, peer poll intvl 1024
root delay 0.00 msec, root disp 0.00, reach 0, sync dist 187.500
delay 0.00 msec, offset 0.0000 msec, dispersion 16000.00
precision 2**5, version 3
org time 00000000.00000000 (10:00:00.000 QLD Mon Jan 1 1900)
rcv time 00000000.00000000 (10:00:00.000 QLD Mon Jan 1 1900)
xmt time D28948FC.5AD01680 (12:07:56.354 QLD Wed Dec 7 2011)
filtdelay = 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
filtoffset = 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
filterror = 16000.0 16000.0 16000.0 16000.0 16000.0 16000.0 16000.0 16000.0

203.161.12.165 configured, our_master, sane, valid, stratum 2
ref ID 130.95.179.80, time D28948E3.6AAEA5FB (12:07:31.416 QLD Wed Dec 7 2011)
our mode client, peer mode server, our poll intvl 512, peer poll intvl 512
root delay 2.01 msec, root disp 35.57, reach 343, sync dist 6076.355
delay 72.54 msec, offset 4.8177 msec, dispersion 6002.75
precision 2**23, version 3
org time D2894C5E.679EF542 (12:22:22.404 QLD Wed Dec 7 2011)
rcv time D2894C5E.6FAC90B3 (12:22:22.436 QLD Wed Dec 7 2011)
xmt time D2894C5E.5D18C2D9 (12:22:22.363 QLD Wed Dec 7 2011)
filtdelay =    72.54   65.96    0.00    0.00   65.48   64.73   66.02   65.52
filtoffset =    4.82    0.33    0.00    0.00    0.43    0.88    0.24    0.34
filterror =     0.02    7.83 16000.0 16000.0   39.08   42.98   50.80   58.61

203.221.91.146 configured, insane, invalid, unsynced, stratum 16
ref ID 0.0.0.0, time 00000000.00000000 (10:00:00.000 QLD Mon Jan 1 1900)
our mode client, peer mode unspec, our poll intvl 1024, peer poll intvl 1024
root delay 0.00 msec, root disp 0.00, reach 0, sync dist 187.500
delay 0.00 msec, offset 0.0000 msec, dispersion 16000.00
precision 2**5, version 3
org time 00000000.00000000 (10:00:00.000 QLD Mon Jan 1 1900)
rcv time 00000000.00000000 (10:00:00.000 QLD Mon Jan 1 1900)
xmt time D2894C75.5D8C2B8C (12:22:45.365 QLD Wed Dec 7 2011)
filtdelay = 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
filtoffset = 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
filterror = 16000.0 16000.0 16000.0 16000.0 16000.0 16000.0 16000.0 16000.0

194.35.252.7 configured, insane, invalid, unsynced, stratum 16
ref ID 0.0.0.0, time 00000000.00000000 (10:00:00.000 QLD Mon Jan 1 1900)
our mode client, peer mode unspec, our poll intvl 1024, peer poll intvl 1024
root delay 0.00 msec, root disp 0.00, reach 0, sync dist 1099.533
delay 0.00 msec, offset 0.0000 msec, dispersion 16000.00
precision 2**5, version 3
org time 00000000.00000000 (10:00:00.000 QLD Mon Jan 1 1900)
rcv time 00000000.00000000 (10:00:00.000 QLD Mon Jan 1 1900)
xmt time D28949D5.5B43B5CB (12:11:33.356 QLD Wed Dec 7 2011)
filtdelay = 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
filtoffset = 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
filterror = 16000.0 16000.0 16000.0 16000.0 16000.0 16000.0 16000.0 16000.0

Leo Laohoo · ‎12-06-2011

Can you post the NTP configuration please?

Can you also post the output to the command "sh ntp a"?

Cory Dryden · ‎12-06-2011

address ref clock st when poll reach delay offset disp

~203.221.91.147 0.0.0.0 16 - 1024 0 0.0 0.00 16000.

*~203.161.12.165 130.95.179.80 2 5 1024 77 65.3 0.44 376.2

~203.221.91.146 0.0.0.0 16 - 1024 0 0.0 0.00 16000.

~194.35.252.7 0.0.0.0 16 - 1024 0 0.0 0.00 16000.

* master (synced), # master (unsynced), + selected, - candidate, ~ configured

ntp clock-period 17180051

ntp server 203.221.91.147

ntp server 203.161.12.165 prefer

ntp server 203.221.91.146 prefer

ntp server 194.35.252.7

Leo Laohoo · ‎12-06-2011

This output shows that the appliance was able to synchronize with a stratum 2 NTP source, 203.161.12.165.

What abut the output from the one that doesn't work?

Cory Dryden · ‎12-06-2011

address ref clock st when poll reach delay offset disp

~203.161.12.165 130.95.179.80 2 23d 64 0 65.9 -6.00 16000.

~194.35.252.7 .GPS. 1 24d 64 0 360.1 -1.48 16000.

* master (synced), # master (unsynced), + selected, - candidate, ~ configured

#sh ntp stat

Clock is unsynchronized, stratum 16, no reference clock

nominal freq is 250.0000 Hz, actual freq is 249.9987 Hz, precision is 2**18

reference time is D2699634.9202AC75 (11:04:52.570 QLD Sun Nov 13 2011)

clock offset is -5.9951 msec, root delay is 360.06 msec

root dispersion is 5.23 msec, peer dispersion is 3.74 msec

Leo Laohoo · ‎12-06-2011

Do you have the config?

Cory Dryden · ‎12-06-2011

ntp clock-period 17179956

ntp server 203.161.12.165 prefer

ntp server 194.35.252.7

Is there anymore that you need? Routes are set on this router too and it was working just over a week ago.

Leo Laohoo · ‎12-06-2011

Can this router successfully ping 203.161.12.165 and 194.35.252.7?

How are they connected? Is there a FW involved?

Can you post a 2 minute of "debug ntp event" and "debug ntp packet"?

Cory Dryden · ‎12-06-2011

There is a firewall, however the rules are in palce to allow NTP through. Can see traffic from the router that is working, but not this router.

244452: .Dec 7 14:08:13.643 QLD: NTP: xmit packet to 203.161.12.165:

244453: .Dec 7 14:08:13.643 QLD: leap 3, mode 3, version 3, stratum 0, ppoll 64

244454: .Dec 7 14:08:13.643 QLD: rtdel 5C2D (360.062), rtdsp 0158 (5.249), refid C223FC07 (194.35.252.7)

244455: .Dec 7 14:08:13.643 QLD: ref D2699634.9202AC75 (11:04:52.570 QLD Sun Nov 13 2011)

244456: .Dec 7 14:08:13.643 QLD: org 00000000.00000000 (10:00:00.000 QLD Mon Jan 1 1900)

244457: .Dec 7 14:08:13.643 QLD: rec 00000000.00000000 (10:00:00.000 QLD Mon Jan 1 1900)

244458: .Dec 7 14:08:13.643 QLD: xmt D289652D.A4D65B48 (14:08:13.643 QLD Wed Dec 7 2011)

244459: .Dec 7 14:08:30.643 QLD: NTP: xmit packet to 194.35.252.7:

244460: .Dec 7 14:08:30.643 QLD: leap 3, mode 3, version 3, stratum 0, ppoll 64

244461: .Dec 7 14:08:30.643 QLD: rtdel 5C2D (360.062), rtdsp 0158 (5.249), refid C223FC07 (194.35.252.7)

244462: .Dec 7 14:08:30.643 QLD: ref D2699634.9202AC75 (11:04:52.570 QLD Sun Nov 13 2011)

244463: .Dec 7 14:08:30.643 QLD: org 00000000.00000000 (10:00:00.000 QLD Mon Jan 1 1900)

244464: .Dec 7 14:08:30.643 QLD: rec 00000000.00000000 (10:00:00.000 QLD Mon Jan 1 1900)

244465: .Dec 7 14:08:30.643 QLD: xmt D289653E.A4DEA88E (14:08:30.644 QLD Wed Dec 7 2011)

244466: .Dec 7 14:09:17.644 QLD: NTP: xmit packet to 203.161.12.165:

244467: .Dec 7 14:09:17.644 QLD: leap 3, mode 3, version 3, stratum 0, ppoll 64

244468: .Dec 7 14:09:17.644 QLD: rtdel 5C2D (360.062), rtdsp 0158 (5.249), refid C223FC07 (194.35.252.7)

244469: .Dec 7 14:09:17.644 QLD: ref D2699634.9202AC75 (11:04:52.570 QLD Sun Nov 13 2011)

244470: .Dec 7 14:09:17.644 QLD: org 00000000.00000000 (10:00:00.000 QLD Mon Jan 1 1900)

244471: .Dec 7 14:09:17.644 QLD: rec 00000000.00000000 (10:00:00.000 QLD Mon Jan 1 1900)

244472: .Dec 7 14:09:17.644 QLD: xmt D289656D.A4EA441B (14:09:17.644 QLD Wed Dec 7 2011)

244473: .Dec 7 14:09:34.644 QLD: NTP: xmit packet to 194.35.252.7:

244474: .Dec 7 14:09:34.644 QLD: leap 3, mode 3, version 3, stratum 0, ppoll 64

244475: .Dec 7 14:09:34.644 QLD: rtdel 5C2D (360.062), rtdsp 0158 (5.249), refid C223FC07 (194.35.252.7)

244476: .Dec 7 14:09:34.644 QLD: ref D2699634.9202AC75 (11:04:52.570 QLD Sun Nov 13 2011)

244477: .Dec 7 14:09:34.644 QLD: org 00000000.00000000 (10:00:00.000 QLD Mon Jan 1 1900)

244478: .Dec 7 14:09:34.644 QLD: rec 00000000.00000000 (10:00:00.000 QLD Mon Jan 1 1900)

244479: .Dec 7 14:09:34.644 QLD: xmt D289657E.A4F199CB (14:09:34.644 QLD Wed Dec 7 2011)

244480: .Dec 7 14:10:21.644 QLD: NTP: xmit packet to 203.161.12.165:

244481: .Dec 7 14:10:21.644 QLD: leap 3, mode 3, version 3, stratum 0, ppoll 64

244482: .Dec 7 14:10:21.644 QLD: rtdel 5C2D (360.062), rtdsp 0158 (5.249), refid C223FC07 (194.35.252.7)

244483: .Dec 7 14:10:21.644 QLD: ref D2699634.9202AC75 (11:04:52.570 QLD Sun Nov 13 2011)

244484: .Dec 7 14:10:21.644 QLD: org 00000000.00000000 (10:00:00.000 QLD Mon Jan 1 1900)

244485: .Dec 7 14:10:21.644 QLD: rec 00000000.00000000 (10:00:00.000 QLD Mon Jan 1 1900)

244486: .Dec 7 14:10:21.644 QLD: xmt D28965AD.A4FF2CE8 (14:10:21.644 QLD Wed Dec 7 2011)

244487: .Dec 7 14:10:38.644 QLD: NTP: xmit packet to 194.35.252.7:

244488: .Dec 7 14:10:38.644 QLD: leap 3, mode 3, version 3, stratum 0, ppoll 64

244489: .Dec 7 14:10:38.644 QLD: rtdel 5C2D (360.062), rtdsp 0158 (5.249), refid C223FC07 (194.35.252.7)

244490: .Dec 7 14:10:38.644 QLD: ref D2699634.9202AC75 (11:04:52.570 QLD Sun Nov 13 2011)

244491: .Dec 7 14:10:38.644 QLD: org 00000000.00000000 (10:00:00.000 QLD Mon Jan 1 1900)

244492: .Dec 7 14:10:38.644 QLD: rec 00000000.00000000 (10:00:00.000 QLD Mon Jan 1 1900)

244493: .Dec 7 14:10:38.644 QLD: xmt D28965BE.A50657AD (14:10:38.644 QLD Wed Dec 7 2011)

Leo Laohoo · ‎12-06-2011

That's wierd. Can this router ping the two addresses?

What is the IOS version of the router?

NTP: Common issues and troubleshooting

Cory Dryden · ‎12-06-2011

2801 Software (C2801-ADVSECURITYK9-M), Version 12.4(6)T, RELEASE SOFTWARE (fc1)

It can't ping becasue the firewall blocks it. From the firewall it can ping. The 194 address has not responded since we noticed this problem yesterday.

Cory Dryden · ‎12-07-2011

I have more information to add to this.

The routers can ping the desired time server. This is verified by looking at the firewall logs and seeing it deny an icmp entry with the correct source and dest addresses.

Internal routing is correct and heading to the internet.

It seems as though packets are not coming back from the desired time server.

tony.henry_2 · ‎12-07-2011

Cory,

To me it looks like the far end denying you. All the NTP systems you have should sync, the router then picks the lowest stratum. just for your own sanity point the router that isn't getting time at the router that is. The router that isn't working will pick sync and call it a stratum 3 source.

Depending how critical the time is, You might try shutting down the service on the one that is working and seeing if you cant get it going on the currently down one.

HTH

Tony

Cory Dryden · ‎12-07-2011

Thanks for the reply. The servers we are using have been used for a few months now. Also I forgot to add that the router that was synced yesterday is no longer synced, so I have nothing synced at present.

How do you shut down a service on a router?

Leo Laohoo · ‎12-07-2011

What kind of server are we talking about?