Solved: Re: NTP not working after memory and code upgrade, Cisco 2611

tleroy · ‎05-08-2008

I upgraded memory to 64 megabytes and code to version 12.3(26) on a Cisco 2611 Router, and now NTP will not synchronize. Here are some relevant parts of my configuration and debug commands and output:

Router1#show ver

Cisco Internetwork Operating System Software

IOS (tm) C2600 Software (C2600-IK9O3S3-M), Version 12.3(26), RELEASE SOFTWARE (fc2)

Technical Support: http://www.cisco.com/techsupport

Compiled Mon 17-Mar-08 15:23 by dchih

ROM: System Bootstrap, Version 11.3(2)XA4, RELEASE SOFTWARE (fc1)

Router1 uptime is 1 day, 14 hours, 23 minutes

System returned to ROM by reload

System restarted at 18:33:44 edt Tue May 6 2008

System image file is "flash:c2600-ik9o3s3-mz.123-26.bin"

!

interface Loopback0

description Loopback interface for service bindings

ip address 10.0.0.1 255.255.255.252

no ip redirects

no ip unreachables

no ip proxy-arp

no ip mroute-cache

!

access-list 21 permit 128.59.59.177

access-list 21 permit 132.236.56.250

access-list 21 deny any log

!

ntp clock-period 17208647

ntp source Loopback0

ntp access-group peer 21

ntp server 132.236.56.250 source Loopback0 prefer

ntp server 128.59.59.177 source Loopback0

!

Router1#show debug

NTP:

NTP clock adjustments debugging is on

NTP clock parameters debugging is on

NTP events debugging is on

NTP packets debugging is on

NTP clock synchronization debugging is on

NTP reference clocks debugging is on

NTP authentication debugging is on

Router1#

Debug output:

.May 8 13:01:48.826 UTC: NTP: xmit packet to 128.59.59.177:

.May 8 13:01:48.826 UTC: leap 3, mode 3, version 3, stratum 0, ppoll 64

.May 8 13:01:48.826 UTC: rtdel 0000 (0.000), rtdsp 10001 (1000.015), refid 00000000 (0.0.0.0)

.May 8 13:01:48.826 UTC: ref 00000000.00000000 (19:00:00.000 EST Thu Dec 31 1899)

.May 8 13:01:48.830 UTC: org 00000000.00000000 (19:00:00.000 EST Thu Dec 31 1899)

.May 8 13:01:48.830 UTC: rec 00000000.00000000 (19:00:00.000 EST Thu Dec 31 1899)

.May 8 13:01:48.830 UTC: xmt CBCD783C.D3BD3463 (09:01:48.827 edt Thu May 8 2008)

.May 8 13:01:52.308 UTC: %SEC-6-IPACCESSLOGS: list 7 permitted 64.80.10.94 1 packet

.May 8 13:01:56.827 UTC: NTP: xmit packet to 132.236.56.250:

.May 8 13:01:56.827 UTC: leap 3, mode 3, version 3, stratum 0, ppoll 64

.May 8 13:01:56.827 UTC: rtdel 0000 (0.000), rtdsp 10001 (1000.015), refid 00000000 (0.0.0.0)

.May 8 13:01:56.827 UTC: ref 00000000.00000000 (19:00:00.000 EST Thu Dec 31 1899)

.May 8 13:01:56.827 UTC: org 00000000.00000000 (19:00:00.000 EST Thu Dec 31 1899)

.May 8 13:01:56.831 UTC: rec 00000000.00000000 (19:00:00.000 EST Thu Dec 31 1899)

.May 8 13:01:56.831 UTC: xmt CBCD7844.D3ED7FC3 (09:01:56.827 edt Thu May 8 2008)

Show NTP Status and Associations results:

Router1#show ntp stat

Clock is unsynchronized, stratum 16, no reference clock

nominal freq is 249.5901 Hz, actual freq is 249.5819 Hz, precision is 2**16

reference time is 00000000.00000000 (19:00:00.000 EST Thu Dec 31 1899)

clock offset is 0.0000 msec, root delay is 0.00 msec

root dispersion is 0.00 msec, peer dispersion is 0.00 msec

Router1#show ntp as

address ref clock st when poll reach delay offset disp

~132.236.56.250 0.0.0.0 16 - 64 0 0.0 0.00 16000.

~128.59.59.177 0.0.0.0 16 - 64 0 0.0 0.00 16000.

* master (synced), # master (unsynced), + selected, - candidate, ~ configured

Router1#

The existing configuration has not changed regarding NTP. It worked on 12.1(19) code.

Any help would be much appreciated!

Richard Burts · ‎05-08-2008

Ted

I would give it a while. It has been my experience that sometimes NTP is actually pretty slow to sync, especially if the clock of the router is significantly different from the clock of the NTP source.

Another thing to try. Please do an extended ping from the router. In the extended ping specify the NTP server as the destination and specify your loopback as the source. This will check the connectivity that NTP will experience. Since it was working before, I would expect this to work. But I believe that it is worth checking.

Another question to ask: is there any possibility that something has changed on the path between your router and the NTP servers? especially is it possible that something is filtering (firewall or router or something) and not allowing the NTP packets?

HTH

Rick

HTH

Rick

View solution in original post

Richard Burts · ‎05-08-2008

Ted

Moving from 12.1(19) to 12.3(26) is a pretty big step and it is difficult to know which behaviors changed from the old config to the new config.

My guess at this point is that there is some issue with thee access list and the ntp access-group peer 21. I have seen some odd behaviors when an ntp access-group peer was configured without an ntp access-group serve-only. As an experiment I suggest that you remove the ntp access-group peer 21. Give it a try and let us know if the behavior changes.

HTH

Rick

HTH

Rick

tleroy · ‎05-08-2008

Hi Rick,

Thanks for the reply and suggestion. I removed the access-group peer 21 as suggested, but no luck so far. I'll leave it a bit and check back, but my suspicion is that if it were going to work, it would happen quickly.

Router1#show run | include ntp

ntp disable

ntp clock-period 17208647

ntp source Loopback0

ntp server 132.236.56.250 source Loopback0 prefer

ntp server 128.59.59.177 source Loopback0

Router1#

Router1#show ntp stat

Clock is unsynchronized, stratum 16, no reference clock

nominal freq is 249.5901 Hz, actual freq is 249.5819 Hz, precision is 2**16

reference time is 00000000.00000000 (19:00:00.000 EST Thu Dec 31 1899)

clock offset is 0.0000 msec, root delay is 0.00 msec

root dispersion is 0.00 msec, peer dispersion is 0.00 msec

Router1#show ntp as

address ref clock st when poll reach delay offset disp

~132.236.56.250 0.0.0.0 16 - 64 0 0.0 0.00 16000.

~128.59.59.177 0.0.0.0 16 - 64 0 0.0 0.00 16000.

* master (synced), # master (unsynced), + selected, - candidate, ~ configured

Router1#

Richard Burts · ‎05-08-2008

Ted

I would give it a while. It has been my experience that sometimes NTP is actually pretty slow to sync, especially if the clock of the router is significantly different from the clock of the NTP source.

Another thing to try. Please do an extended ping from the router. In the extended ping specify the NTP server as the destination and specify your loopback as the source. This will check the connectivity that NTP will experience. Since it was working before, I would expect this to work. But I believe that it is worth checking.

Another question to ask: is there any possibility that something has changed on the path between your router and the NTP servers? especially is it possible that something is filtering (firewall or router or something) and not allowing the NTP packets?

HTH

Rick

HTH

Rick

tleroy · ‎05-08-2008

Hi Rick,

The clock isn't very far off:

Router1#show clock

.13:28:19.254 edt Thu May 8 2008

Router1#

I think you have me on the right path with the ping command though:

!

ntp clock-period 17208647

ntp source Loopback0

ntp server 132.236.56.250 source Loopback0 prefer

ntp server 128.59.59.177 source Loopback0

!

end

Router1#ping 132.236.56.250

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 132.236.56.250, timeout is 2 seconds:

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 64/64/68 ms

Router1#ping 132.236.56.250 source loopback0

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 132.236.56.250, timeout is 2 seconds:

Packet sent with a source address of 10.0.0.1

.....

Success rate is 0 percent (0/5)

Router1#

I'll check my ACL's on the Internet Interface for the problem.

I'll let you know what I find.

Thanks!

tleroy · ‎05-08-2008

Hi Rick,

Thanks for the adept troubleshooting help! It feels a little silly that I hadn't tried that, but that what forum's are for.

I had removed a nat statement just before doing the code upgrade:

Router1#show run | include nat

ip nat inside source static tcp 10.0.0.1 123 2.8.2.2 123 extendable

ip nat inside source static udp 10.0.0.1 123 2.8.2.2 123 extendable

Router1#

2.8.2.2 is a fake in place of my Internet IP, but you get the picture. Port 123, NTP!

Richard Burts · ‎05-08-2008

Ted

I am glad that you got the problem resolved. The explanation makes a lot of sense.

Thank you for posting back to the forum to indicate that you had solved the problem. It is helpful to see problems and to see the explanation of the problem.

And thank you for using the rating system to indicate that your problem was resolved (and thanks for the rating). It makes the forum more useful when people can read about a problem and can know that they will read an explanation of the solution of the problem.

The forum is an excellent place to learn about Cisco networking. I encourage you to continue your participation in the forum.

HTH

Rick

HTH

Rick

tleroy · ‎05-09-2008

You're welcome Rick, and thanks for the help. It's working now:

Router1#show ntp stat

Clock is synchronized, stratum 3, reference is 132.236.56.250

nominal freq is 249.5901 Hz, actual freq is 249.5825 Hz, precision is 2**16

reference time is CBCEC60E.1F300C79 (08:46:06.121 edt Fri May 9 2008)

clock offset is 1.3031 msec, root delay is 88.18 msec

root dispersion is 50.05 msec, peer dispersion is 0.53 msec

Router1#show ntp as

address ref clock st when poll reach delay offset disp

*~132.236.56.250 128.118.25.12 2 25 256 377 65.2 -0.99 0.5

+~128.59.59.177 192.5.41.209 2 61 256 377 48.5 3.60 16.3

* master (synced), # master (unsynced), + selected, - candidate, ~ configured

Router1#