Berthold
If I understand correctly what you have configured I believe that you are configuring ntp to go to the shared (virtual) address of HSRP. I think it would be better to not do this and to configure the physical interface address.
You do not need to configure your backbone switches as servers. As long as they have learned authoritative time from the unix machine then they will respond to ntp querries and will give ntp time to other devices.
Sometimes it is helpful to configure the ntp source-address, especially if there are multiple layer 3 interfaces through which the ntp packets may be sent. Configuring the ntp source-address helps assure that the ntp packets will have the same source address no matter which interface they are sent through.
And one further comment: your post indicates that you are configuring ntp peer. I think it might be better to configure ntp server. Configuring ntp server indicates that you are going from a less authoritative time source to a more authoritative time source. Configuring ntp peer is appropriate when the machines are equally authoritative ntp sources. I believe that it will work either way that you configure it (server or peer) but believe that server is a bit more appropriate.
HTH
Rick
HTH
Rick