Some steps you might try to help you resolve this.
1. Make sure that configured NTP server is reachable by pinging the server
IP address(es).
2. Issue the command show ntp status on the server(s) to make sure that NTP server
has synched itself. If it is not synched, verify the server NTP configuration.
The show ntp status of the server should display a status of 'sync' before
clients can sync their clock with the server.
3. Verify that the 'stratum' value configured on the server is valid for your setup.
Configure a high stratum number to ensure that this router does not override
the clock on another system with a lower stratum number. The lower stratum number
indicates a more reliable clock.
4. If authentication is configured between the server and the client, make sure
that authentication-key number and md5 key value matches. Also, verify that
the command ntp server {server_address} key {key} is included in the client
configuration.
5. Make sure that NTP packets are not blocked by the access-list statement.