← Workstreams

Workstream: HierarchicalClock

Status: Planned · Component: Maximize developer productivity · Docs: project page

Goal

Make read-after-write consistency across independent services a solved primitive, so a developer never has to reason about "did my read see my own write?" by hand. When a client writes to one service and then reads from another (often a service that caches the first), the platform should let them prove the read reflects their write — without polling, sleeps, or bespoke versioning.

Vision: a decentralized global network of clocks

The end state, designed in the module's DESIGN.md, is a decentralized global network of clocks so lightweight that every application runs its own — a ClockId, a monotonic counter, and a bounded memory of recent entanglements. Leaf clocks need not be reachable at their url:// address; their timestamps gain global meaning through the entanglements they record with reachable clocks. An application that witnesses an event records the event's timestamp in its ClockPad, and any later read — from any service — can be required to prove it incorporates that event.

Aggregator clocks are what keep this tractable at global scale. There is deliberately no single root: clocks entangle their timestamps upward into one or more well-known aggregators (and aggregators into each other), forming a dynamic multi-layer DAG that converges organically — so no code path can grow an implicit single-root or single-parent assumption. Anyone holding a timestamp can roll it up into an aggregator, not just the clock that minted it, so pads never depend on another clock's cooperation to stay small: a ClockPad that would otherwise track thousands of leaf clocks compacts to a handful of aggregator entries, trading granularity for size as events age.

Current state

This exists as community.kotlin.clocks.hierarchical — a distributed network of hierarchical logical clocks that "entangle" with each other so clients can verify a data owner's view is sufficiently up to date. The core concepts:

  • ClockId — a unique clock identity, addressed as a url:// protocol URL.
  • HierarchicalTimestamp — monotonically increasing values issued by a clock.
  • Entanglements — causal relationships established between clocks.
  • ClockPad — a client-side record tracking the latest timestamps from every service it has interacted with; passed along on subsequent calls so a receiving service can verify (and, if stale, refresh) before answering.

The module's DESIGN.md records the full design and its decisions: the aggregator DAG (no single root), opportunistic third-party rollup, the evidence-driven compaction surface, the retention/GC contract, fail-closed three-valued verdicts, and the phased W3Wallet trust model. As of core 0.0.6 that surface is implemented: pure evidence-driven ClockPad.compact(evidence, victims), the read-only coverageFor query, the knownTimestampFor(remote, asOf) decomposition query, and three-valued fail-closed verify (ClockPadVerdict/ClockLag). A filesystem-backed implementation exists as community.kotlin.clocks.hierarchical.filesystem, the canonical in-memory aggregator ships as community.kotlin.clocks.hierarchical.memory (InMemoryHierarchicalClock), and community.kotlin.clocks.simple's SystemClock/ManualClock serve as in-process wall-clock roots.

Adoption is further along than greenfield. It first landed on the observable path and now generalizes to all plain RPCs:

  • Read side: Observable ships the integration — DerivedObservable.calculateCurrentValue(requiredClockPad) answers from cache when the cached value is known to satisfy the pad, and otherwise recomputes with the pad available to the calculator via RequiredClockPad, so the requirement propagates to the services the calculator queries.
  • Transport: UrlResolver already marshals pads over RPC, forwards RequiredClockPad into server-side invocations, answers satisfied-pad queries, and auto-advances a per-node session pad when a live-projection command returns a HierarchicalTimestamp — with a passing read-your-writes test proving the loop end to end. Generalizing that ambient contract to all plain RPCs — a ClockPadAwareServiceHandler wrapping every invocation in withRequiredClockPad, opt-in serving clocks that stamp clock-timestamp and attach clock-receipts on responses, and session-pad threshold compaction fed by those receipts — is landing in PR #679, atop the additive RpcResponse metadata channel in UrlProtocol 0.0.329 (PR #334).
  • Reference example: CqrsObservableExample demonstrates the CQRS write-timestamp/read-pad pattern with an in-memory hierarchical clock.

The CQRS architecture already mandates that services return Hierarchical Clock entries with write responses so later reads can request data current as of the client's last write, and the Event Log plan relies on it for cross-service ordering. So the module is foundational; with the aggregation surface now published and ambient threading generalizing from the observable path to all plain RPCs (landing in PR #679), this workstream's core mechanism is largely in place — the remaining push is production adoption, non-blocking stale reads in Observable, and the W3Wallet trust phase.

Why it accelerates developers

  • No more "eventually consistent, so add a retry." The two canonical pain cases — multi-server mutation coordination and cache-staleness verification — become library calls (ClockPad.update(...), serverB.query(params, ClockPad.of(tsA))).
  • It composes with the rest of component 1. It is the consistency glue under Event Log projections and Observables caches, and the ordering backbone of EventSourcingStreams: every stream member embeds a clock and every published event carries a timestamp, so cross-member ordering and catch-up are causal by construction.

Plan / roadmap

  • [ ] Generalize ambient ClockPad threading to the whole standard service contract. Decided: threading is ambient by default — Api signatures stay pad-free; the required pad rides RPC metadata (client resolver attaches the session pad, the server handler exposes it via RequiredClockPad), and every response carries the serving clock's current timestamp, auto-merged into the caller's session pad. That yields automatic session read-your-writes and monotonic reads with zero per-service plumbing across the standard layered architecture. Explicit HierarchicalTimestamp returns remain, per CQRS, for handing the token to another party (on a remote-observable proxy the write's timestamp is surfaced through the RemoteCommandDerivedObservable the command returns — see REMOTE_OBSERVABLES.md §5.1). Delivered, pending merge: the mechanism UrlResolver ships on the observable/projection path is now generalized to all plain RPCs — server-side ClockPadAwareServiceHandler + withRequiredClockPad, opt-in per-registration serving clocks that stamp clock-timestamp and attach clock-receipts (no default serving clock — no proof, no claim), session-pad threshold compaction, and the response-piggybacked coverage receipts (now real, not merely transport sugar) — in PR #679, atop the additive RpcResponse metadata channel in UrlProtocol 0.0.329 (PR #334). The PR is green and awaiting merge.
  • [x] Aggregation network: rollup and pad compaction. Delivered — the evidence-driven compaction/verification surface ships in core 0.0.6 (ClockPad.compact(evidence, victims)/size, the read-only coverageFor query, the knownTimestampFor(remote, asOf) decomposition query with a future-asOf fail-closed guard, three-valued fail-closed verify → ClockPadVerdict/ClockLag, and annotation-only Entanglement.wallTimeMillis) (PR #21), with the canonical in-memory aggregator (multiple independent aggregators from day one — never a single root; opportunistic third-party witness rollup) in the new community.kotlin.clocks.hierarchical.memory (PR #1) and the same surface implemented over the filesystem clock (PR #12) and the simple roots (PR #29), per DESIGN.md. Artifacts are published; the source PRs are green and awaiting merge approval.
  • [ ] Non-blocking stale reads in Observable. Decided (policy updated by the remote observables design): the clock reports facts (what is required, what is known) and never dictates read policy — that is the consuming framework's contract with its users. Observable's chosen policy: a read whose value does not yet satisfy the required pad throws a structured RemoteObservableDataUnavailable (reason CONSTRAINT_NOT_MET) after registering the read as a dependency, so the satisfying value arrives through the normal invalidation/onNewValue path and re-runs the reader; presentation boundaries (e.g. Compose) keep showing the last-known-good value, dimmed, until it does. The blocking calculateCurrentValue(requiredClockPad) exists today; the non-blocking mode is built by the Observables workstream.
  • [ ] W3Wallet-signed timestamps and receipts. The second phase of the trust model (the network assumes a trusted fabric until then): clocks hold W3Wallet identities bound to their ClockId addresses, minted timestamps and entanglement receipts are signed, and incorporation proofs become verifiable chains. Gated on W3Wallet identity infrastructure; record shapes stay extensible in the meantime.
  • [ ] Reference adoption. Wire one real production read-after-write path (e.g. an Event Log projection) end-to-end and document the pattern — CqrsObservableExample covers the example tier already.
  • [ ] End-to-end tests. Delivered, pending merge: a hermetic suite over two in-process services on isolated P2P nodes proves a read reflects a prior write through the ambient contract (read-your-writes), through compaction + decomposition (a client-driven aggregator rollup that a caching service verifies by asking the aggregator knownTimestampFor(A, asOf)), and that a lagging service whose owning clock is unreachable surfaces an honest not-satisfied verdict rather than a false satisfied — in PR #679, per the testing standards, extending the read-your-writes coverage UrlResolver's projection path already had. Lands with the ambient-threading PR above.

Graduation

When the ClockPad flows automatically through the standard contract and at least one production read-after-write path depends on it, this graduates from a workstream to a first-class project; its Documentation Repository project page already exists and stays the canonical product documentation, alongside the module's DESIGN.md.