Workstream: GithubProxy Freshness — cached reads, ClockPad-driven staleness, observable delivery

Status: In progress (started 2026-08-13) — phase 1 merged, phase 2 served-but-not-consumed · Parent: GithubProxy workstream · Cuts across: Ambients, Observables, HierarchicalClock, UrlResolver

Where it stands (verified 2026-08-17): the foundational gaps are closed upstream — required-pad propagation (UrlResolver PR #960) and effect delivery to projection clients (UrlResolver PR #962) — the url://realtimeclock/ clock exists as a published library plus service, GithubProxy caches on disk and evaluates freshness against the ambient pad, and it serves the repository/pull-request tranche as live projections. What remains is the consumer half: the CLI and WUI establish pads and handle the freshness effects, but still read through the existing operations rather than resolve(), so no non-blocking notify-on-arrival read runs end to end yet.

A deep-dive on the read path of GithubProxy, orthogonal to that workstream's wallet/capability axis: however a caller is authorized to read GitHub data, this workstream decides how fresh the answer must be, where it comes from (a persistent cache vs. a live GitHub fetch), and how it is delivered (as observables rather than blocking RPC). It is also the deliberate reference adoption the ambients plan calls for: the first service where the required-freshness ClockPad, algebraic-effect warnings, and observables all flow end-to-end from a CLI/WUI entry point through url:// to a remote read — with the expectation that first-integration gaps will surface across several repositories and be fixed at their source.

Goal

GithubProxy caches GitHub data on disk. Reads are served from a persistent, account-partitioned cache when the cached copy is fresh enough, and from GitHub otherwise; every successful fetch refreshes the cache. (Today the proxy has no cache — every read costs a live GitHub round trip.)
"Fresh enough" is defined by the ambient required-freshness ClockPad. The caller does not thread freshness arguments through signatures; the required-freshness ambient carries the requirement down the call chain and across the url:// wire. Data is re-fetched from GitHub whenever the cached copy cannot be proven to satisfy the pad.
A missing requirement is loud, never silent. When no required-freshness pad is specified, GithubProxy assumes up-to-date (real-time) data is required — it re-fetches — and additionally tosses an algebraic effect complaining that no ClockPad was specified, raised so the actual caller can observe it.
Reads are exposed as observables. GithubProxy data is served as live projections: resolving is non-blocking, a read that cannot yet satisfy the ambient constraint reports unavailability through the standard contract, and the client is notified automatically when data satisfying its requirement arrives. Writes remain commands.
Cross-repo integration gaps get fixed at their source. This is one of the first full integrations of observables, effects, and ambients across a UrlResolver service; gaps discovered in UrlResolver, Observable, or the clocks libraries are fixed upstream (per the upstream-bug-fix rule), never papered over inside GithubProxy.

Requirements — settled decisions (2026-08-13)

Pad entries are logical timestamps of arbitrary clocks, and GithubProxy owns no clock of its own. Any pad entry is a freshness requirement GithubProxy must honor: for each entry it must work out whether its cached data is at least as fresh as that entry, which in general means relating the entry to the wall time at which the cache was fetched.
A global real-time clock anchors wall-time requirements. Some clocks expose real-world time as their logical timestamp, so requiring actual real-time freshness is just a pad entry for such a clock. No global url:// address for one existed, so this workstream creates it: the well-known ClockId url://realtimeclock/, whose logical timestamp value is epoch milliseconds — delivered as a library (community.kotlin.clocks.hierarchical.realtime) plus a hosted service (RealtimeClockServerService). "No older than five minutes" is the entry url://realtimeclock/@(now − 5min); "real-time" is @now.
"ClockPad unavailable" means an empty pad. A pad with any entry counts as a specified requirement and is evaluated per entry; only a pad with no entries triggers the assume-real-time default plus the complaint effect.
Unproven freshness fails closed. An entry whose relationship to wall time cannot be established is never claimed satisfied: GithubProxy still re-fetches (best-effort freshness) but does not assert satisfaction, and it tosses a warning effect naming the entry — the read reports the constraint as unmet rather than lying.
The complaint effect is raised on both sides. The client-facing layer tosses it into the caller's effect scope, and the server independently tosses it for pad-less requests (covering callers that never established the ambient); server-raised effects must surface on the consuming side as effects, and making that true across the wire is in scope.
Scope: representative tranche first. Repository metadata, the pull-request list, and pull-request details + checks go observable end-to-end (server → CLI/WUI) before the remaining read surface converts.

Freshness semantics

For a cached item fetched from GitHub at wall time F and a required pad entry X@c:

Real-time clock entries (X = url://realtimeclock/): satisfied exactly when F ≥ c.
Any other clock: satisfied only with evidence bounding the wall time at which X minted c (entanglement wall-time metadata and clock-network queries are the intended evidence sources); without evidence the entry is unverifiable and fails closed as above.
After a re-fetch, exactly the entries the fetch provably satisfies join the served freshness watermark, so a client's availability check passes precisely when satisfaction was proven — never optimistically.
Cache hits change no other contract: authorization, tenancy isolation, guardrails, and rate-limit decisions behave as if the read had gone to GitHub; a cache hit must never leak one account's data to another principal, and only real GitHub fetches consume GitHub-facing budgets.

Known integration gaps this workstream closes

Verified 2026-08-13; these are requirements on the foundational layers, not GithubProxy patches:

The projection wire must carry and consume the required pad. Today a subscriber's freshness pad is recorded and echoed but not consumed, the subscribe path does not include the ambient requirement, and a requirement that grows after subscribing is never communicated. A projection server needs a first-class way to learn a subscriber's requirement (initially and on change) so it can refresh its data and advance its watermark.
Server-raised effects must reach projection consumers. Effects already propagate to clients on the sandboxed-connection path, but on the persistent-RPC/live-projection path server-captured effects are silently dropped — and their presence corrupts the command-result envelope. Warnings raised while serving observable reads must surface as effects on the consuming side.
A dereferenceable real-time clock must exist (created by this workstream, above).

Plan / roadmap

Phase 1 — foundations (merged 2026-08-14/15):

[x] Required-pad propagation and consumption for live projections (gap 1), proven by an end-to-end scenario where an unsatisfied read heals automatically — without re-resolving or polling — once the server refreshes. Landed in UrlResolver PR #960; serveRemoteObservable now takes a watermarkProvider and a requiredPadListener, so a server learns each subscriber's requirement initially and as it grows. A follow-up defect pair — pad recomputation on unsubscribe and refresh-failure delivery — is still open in UrlResolver PR #968.
[x] The real-time clock: library + url://realtimeclock/ service, published so any consumer can import the well-known ClockId; timestamps remain monotonic under wall-clock regression and concurrent minting. Delivered by community.kotlin.clocks.hierarchical.realtime PR #1 and RealtimeClockServerService PR #1, with the monotonicity properties covered by monotonicitySurvivesWallClockRegression and concurrentMintsAreUniqueAndMonotonic.
[x] On-disk caching + freshness evaluation in GithubProxy on the existing read path for the tranche operations, including the two warning effects (missing pad, unverifiable entry) observable by a real client, and cache persistence across server restarts. Landed in GithubProxyServerService PR #36.

Phase 2 — observable delivery (projections served 2026-08-15/16; consumption outstanding):

[x] Effect propagation for projection consumers (gap 2). Landed in UrlResolver PR #962: server-captured effects now reach persistent-RPC projection clients and no longer corrupt the command-result envelope. One known limitation carried forward — concrete effect types arrive downgraded to the base NotificationEffect (the message text survives), so typed reconstruction on the consumer remains a resolver-side follow-up.
[x] Tranche projections: repository metadata, PR list, PR details + checks served as live projections whose watermarks reflect proven freshness, refreshed on subscriber demand. Landed in GithubProxyApi PR #21 (the extracted contract) and GithubProxyServerService PR #37.
[ ] CLI/WUI consumption: entry points establish the ambient pad (a max-age flag / query parameter becomes a realtime-clock entry), handle the warning effects with specific handlers, and demonstrate non-blocking notify-on-arrival reads end-to-end. Half done. The pad and effect-handler half landed in GithubProxyCli PR #19 and GithubProxyWui PR #2. The tranche is served but not yet consumed: nothing outside GithubProxyApi references GithubTrancheProjectionUrls, and the WUI's pages still read through the existing operations rather than resolve(). Rendering those pages as live projections is the remaining work, and it is also what the Observables workstream needs for its own graduation.

Phase 3 — deferred follow-ups:

[ ] Arbitrary-clock verification: a clock-network-backed evidence source so foreign-clock entries verify instead of failing closed.
[ ] Full read surface converts to projections; conditional-request support (ETags) to make re-fetches cheap; cache size/eviction bounds.
[ ] Tenancy-scoped projections, following the parent workstream's capability trajectory.
[ ] Promote the freshness warning effects to a shared home once a second service adopts the same contract.

Relationship to other workstreams

GithubProxy — the parent; it governs who may read (wallet capabilities, guardrails), this workstream governs how fresh the read is and how it is delivered. The cache and freshness gate sit behind whatever authorization model the parent lands.
Ambients — this is the plan's "reference adoption" item made concrete: pad and effect ambients flowing CLI/WUI → url:// → service → GitHub, with the contract telling the whole story.
Observables / UrlResolver — supply the live-projection machinery consumed here; gaps 1–2 are their deliverables and fold into their plans.
HierarchicalClock — the freshness model (pads, watermarks, entanglement wall-time evidence) is that workstream's; url://realtimeclock/ becomes the network's first well-known public clock and a natural witness/aggregator participant.

Graduation

This workstream graduates when: a CLI or WUI request carrying an ambient url://realtimeclock/@T entry is served from the on-disk cache with no GitHub call when the cache provably satisfies T, and triggers a re-fetch with a non-blocking observable update when it does not; an empty pad yields real-time data plus the complaint effect observed in the caller's effect scope across the wire; the tranche projections serve repository/PR/checks data end-to-end with constraint-unmet reads healing automatically; and the foundational-layer gaps are merged upstream with tests.

The foundational-layer clause is satisfied and the serving side is in place; the outstanding clause is the end-to-end one — a CLI or WUI page that reads the tranche through resolve(), so the non-blocking update and the automatic heal are exercised by a real consumer rather than only by the resolver's own tests.