NTP: keeping clocks aligned

Time is a load-bearing dependency you only notice when it is gone. DNSSEC signatures, TLS certificates, log correlation, Kerberos, and audit trails all assume that "now" is the same on every system. NTP is what makes that true.

The Network Time Protocol is defined in RFC 5905 (NTPv4), with operational guidance in RFC 8633 (NTP Best Current Practices). The NTP Pool Project operates the most widely used public time source.

Why this matters#

A small list of things that break when clocks drift:

Drift	What breaks
±60 s	Some TLS handshakes begin failing ("certificate not yet valid" / "expired")
±1 m	TOTP / 2FA codes stop matching (RFC 6238 uses 30 s time steps, and validators typically allow only about one step of drift)
±5 m	Kerberos tickets refuse to issue (krb5/AD default `clockskew` tolerance is 300 seconds; configurable)
±5 m	RRSIG records may validate as expired or not-yet-valid; zones go bogus (see dnssec-troubleshooting). The actual threshold depends on how tightly the zone operator set the RRSIG validity window, not a hard protocol constant.
±10 m	Cross-host log correlation becomes nonsensical
Random	Cron schedules, backups, and monitoring alerts all skew

Time is a silent failure mode. By the time you notice, it has usually already broken something downstream.

How NTP works#

NTP exchanges timestamps between client and server to compute two values:

Offset: how far the local clock is from the server's clock.
Round-trip delay: how long the exchange took, used to discard high-latency measurements.

The client averages multiple servers, discards outliers, and steers the local clock toward consensus gradually (slewing) rather than jumping. This keeps monotonic time monotonic, which matters for any software that relies on timestamps increasing. One caveat: a very large offset is corrected with a one-time step instead. ntpd steps when the offset exceeds a threshold (128 ms by default); chrony slews only, unless makestep is configured (commonly limited to the first few updates after boot).

NTP runs on UDP/123. A client maintains an association with each configured server, polling at increasing intervals as confidence grows. The protocol minimum poll interval is 16 s (minpoll 4), but most implementations default to a starting interval of 64 s (minpoll 6); the range under default settings is typically 64 s → 1024 s, reaching the floor of 16 s only if explicitly configured.

Stratum levels#

Time servers are organized in tiers:

Stratum	What it is
0	Reference clock: atomic clock, GPS receiver, radio clock. Not directly queryable on the network.
1	A server directly attached to a Stratum-0 source.
2	A server synchronizing from a Stratum-1.
3+	Each hop adds one stratum.
16	Unsynchronized or connection lost.

For most purposes, Stratum 2 or 3 sources are entirely adequate. Over a decent network path, wall-clock error versus UTC is typically well under 50 ms.

Picking good sources#

Bad source selection is the most common cause of "clocks look fine but aren't." The basic rules:

Use at least four sources. NTP needs a quorum to detect a faulty server. Fewer than four and you cannot distinguish truth from a slow or lying peer. RFC 8633 section 3.2: operators "SHOULD use at least four independent, diverse sources of time."
Choose geographically and administratively diverse sources. Four servers from the same operator share failure modes. Mix regional pool.ntp.org pools with vendor pools (Cloudflare time.cloudflare.com, Google time.google.com, Apple time.apple.com), subject to the leap-second rule below.
Never mix leap-smearing and non-smearing sources. Around a leap second, Google smears (spreads the extra second across many hours) while Cloudflare explicitly does not, and ordinary NTP servers (which is what pool.ntp.org typically hands you) apply the full second at once. During the smear the two camps legitimately disagree by up to a second, which wrecks source selection. RFC 8633 section 3.7.1 is blunt: clients "MUST NOT be configured to use a mixture of smeared and non-smeared servers." Pick one family.
Prefer low jitter, low offset, stratum ≤ 3. Run chronyc sources -v and favor servers with sub-millisecond jitter.
Use a pool, not a single hostname. pool.ntp.org rotates DNS answers so clients land on different members (each lookup returns a fresh set of member addresses with a short TTL). A single-host configuration converts one Stratum-1 outage into a drifting clock.
Use NTS (Network Time Security) where supported. NTS (RFC 8915) adds TLS-based authentication to NTP (similar in spirit to DNS-over-TLS), preventing on-path tampering. Key establishment runs over TLS on TCP/4460, so that port must be open outbound too. Cloudflare (time.cloudflare.com), Netnod, and several other public servers support it.

A reasonable chrony configuration for a general-purpose host (all non-smearing sources; do not add time.google.com to this mix):

# /etc/chrony/chrony.conf
pool 0.pool.ntp.org iburst
pool 1.pool.ntp.org iburst
pool time.cloudflare.com iburst nts

Checking NTP health#

chrony#

Check overall sync status, including the selected reference, stratum, and current offset:

chronyc tracking

List every configured source with per-source stratum, offset, and jitter:

chronyc sources -v

A healthy chronyc tracking output looks like:

Reference ID    : A29FC801 (time.cloudflare.com)
Stratum         : 3
Ref time (UTC)  : ...
System time     : 0.000123456 seconds slow of NTP time
RMS offset      : 0.000123456 seconds
Frequency       : 3.567 ppm slow
Leap status     : Normal

Sub-millisecond system time offset is healthy. Watch for:

Leap status : Not synchronised: not synchronized. In this state chronyc tracking shows Stratum : 0 and Reference ID : 00000000 (no source selected). Note the display difference: stratum 16 is the on-wire "unsynchronized" value from RFC 5905 and is what ntpq -p shows, but chrony's tracking output reports 0 instead.
System time offset in whole seconds: the clock has not synced since boot or drifted severely.

systemd-timesyncd#

timedatectl status
timedatectl timesync-status

The output shows the current NTP server, last sync time, and offset.

ntpq (classic ntpd)#

ntpq -p

Each peer line shows stratum, jitter, and offset. A * prefix on a line indicates the currently selected source.

Failure modes#

Symptom	Likely cause
Certificate validation suddenly fails after a long-uptime host reboots	Hardware (RTC) clock drifted while powered off; NTP has not yet re-synced
DNSSEC zones go `SERVFAIL` across multiple hosts simultaneously	Time skew large enough to invalidate all RRSIGs at once
Log aggregation shows events "before they happened"	One host's clock is ahead of the others
Cron jobs run at unexpected wall-clock times	Timezone vs. UTC confusion, or a drifted clock
NTP sync fails after re-IP or firewall change	Outbound UDP/123 is blocked by the new policy (and TCP/4460 for NTS key establishment)
Host with a badly wrong clock cannot even resolve its NTP server names	Chicken-and-egg: DNSSEC validation needs correct time, and the NTP hostnames need DNS. If the local resolver validates, a wildly wrong clock can make every signed answer bogus, including the answer for the time servers. Mitigations: configure at least one time source by IP address, or use a resolver escape hatch such as dnsmasq's `--dnssec-no-timecheck`, which defers the RRSIG time-window check until the clock is trusted

Hygiene#

Always configure at least four sources. One source gives you no protection against a falseticker.
Do not query stratum-1 servers directly without permission. Most operate under acceptable-use terms that prohibit unauthorized direct queries. The public services (pool.ntp.org, time.cloudflare.com, time.google.com) are explicitly open for general public use (the NTP Pool has separate rules for vendors shipping it as a product default).
Watch for NAT or firewall rate-limiting on UDP/123. Some routers throttle outbound port 123 to limit NTP reflection amplification abuse. This throttling affects legitimate clients too.
Log NTP status periodically. A daily chronyc tracking snapshot in your normal log stream catches drift before it causes failures.