Keeping the clock right
A clock that drifts by a few minutes produces bugs that look like anything except a clock problem.
What a wrong clock breaks
- TLS. Certificates have a validity window. A machine whose clock is a day behind rejects perfectly good certificates and cannot issue its own.
- Authentication. TOTP two-factor codes, Kerberos tickets and signed API requests all fail outside a tolerance measured in seconds.
- Logs. Correlating events across two machines is impossible if their clocks disagree. This is the one that wastes whole afternoons.
- Scheduled work. Jobs run twice, or not at all. See cron jobs and timers.
- Databases. Replication and conflict resolution rely on timestamps being comparable.
Check it
timedatectl
Look for System clock synchronized: yes and an active NTP service. If
it says no, nothing is keeping the clock honest and it is drifting right now.
chrony, for preference
apt install chrony or dnf install chrony, then
systemctl enable --now chronyd.
Chrony handles intermittent connectivity and large initial offsets better than the alternatives, and corrects by slewing — speeding the clock slightly — rather than jumping it. Time never goes backwards, which matters to anything keeping a monotonic sequence.
Check what it is doing:
chronyc tracking — offset and drift
chronyc sources -v — which servers it is using
A healthy machine shows an offset in milliseconds or better. Anything in whole seconds means something is wrong.
systemd-timesyncd is installed by default on many systems and is fine
for a machine that does nothing time-critical. Run one or the other, never both.
Keep servers on UTC
timedatectl set-timezone UTC
Local time on a server buys you nothing and costs you an hour twice a year. With summer time, 02:30 happens twice in autumn and not at all in spring — a job scheduled then runs twice or never. Logs from machines in different zones become impossible to line up.
Display local time in your application, where you know the user. Store and log in UTC.
On our network
Outbound NTP works normally; you do not need to ask for anything. If you run many machines, pointing them at one or two of your own servers that in turn sync upstream keeps them tightly in step with each other, which is usually what you actually care about. See connecting your own servers.
Do not leave an open NTP server reachable from the internet. Badly configured NTP has been used for amplification attacks for years, and our acceptable use policy is explicit about running amplification sources.
Still stuck? Mail support@novogara.com — an engineer answers, at any hour. Back to the knowledge base