← Workstreams

Workstream: EventLog

Status: Planned · Component: Maximize developer productivity

Goal

Make state management a solved primitive: a developer records what happened as immutable domain events and derives every queryable view from them, instead of inventing bespoke persistence and worrying about keeping caches, databases, and UIs in sync. The Event Log is the authoritative source of truth; if any derived state is lost or corrupted, it is rebuilt by replaying the log.

Current state

This exists as the Event Log project — the write side of the CQRS pattern, an append-only, immutable record of domain events:

Layer Repository What it provides
Core community.kotlin.eventlog EventLog interface, data types (Event, StoredEvent, EventBatch, CommitStatus), streaming API (EventStreamIterator)
Impl (local) community.kotlin.eventlog.localfilesystem SQLite-backed local storage
Impl (test) community.kotlin.eventlog.naivetoy In-memory implementation for unit tests
Service community.kotlin.eventlog.service.server HTTP server exposing the EventLog API over local-filesystem storage
Client community.kotlin.eventlog.service.client HTTP client for the EventLog service server

The core API has two paths: an append path for recording events and a streaming path for consuming them to build projections. It is deliberately distinct from Logging: the Event Log holds facts that drive state; logging holds commentary. This same core contract is also what the planned EventSourcingStreams decentralized streams implement — the Event Log is that contract's durable, single-node, retention-forever implementation (the system of record), and a stream is the same contract in motion between parties. So the building blocks exist; this workstream is about making it the default way services hold state.

Why it accelerates developers

  • Persistence is no longer a per-service design problem — append events, derive read models, done.
  • Rebuildable state removes a whole class of "the cache is wrong and I can't fix it" incidents: invalidate and replay.
  • Pairs with the rest of component 1: read-after-write ordering comes from HierarchicalClock; efficient read models from Observables; cross-cutting auth from W3Wallet.

Plan / roadmap

These are proposed and need sharpening before work starts. Two design decisions are already settled in EventSourcingStreams and taken as given here: events are immutable and never rewritten or migrated — schema evolution happens in versioned, event-sourced projection definitions installed with an explicit effective-as-of log position, so a bad or new event type is repaired retroactively by a backdated projection version while retention still covers it — and ordering across multiple appenders is causal, via the HierarchicalClock timestamp every event carries, with no stronger total-order mechanism imposed.

  • [ ] Expose the Event Log over url:// (not just HTTP) so it is a first-class P2P service consumable by sandboxed clients, consistent with the rest of the ecosystem.
  • [ ] Deploy a shared, durable hosted instance on ContainerNursery with storage that survives container restarts.
  • [ ] Projection toolkit. A documented, reusable pattern (and helper library) for subscribing to the stream and materializing read models, with the invalidate-and-rebuild discipline built in — shared work with the EventSourcingStreams projection bridge and its versioned, event-sourced projection definitions.
  • [ ] Snapshots to bound replay cost. Replay-from-zero is the safety net, but a large retention-forever log needs snapshots so a rebuild does not reread the whole log. Deciding where snapshots live — including whether they build on SimpleFileSystem or BlobStorage — is part of this item's design; the projection checkpoints from EventSourcingStreams' versioned-definitions design are the likely building block.
  • [ ] HierarchicalClock integration so appends return timestamps and projections expose read-after-write guarantees — see HierarchicalClock.
  • [ ] Reference adoption. Migrate one existing service to event-sourced state and document the before/after.
  • [ ] End-to-end tests using the in-memory naivetoy impl plus a real service-server round trip, per the testing standards.

Graduation

The Event Log already has a Documentation Repository project page. This workstream graduates when a url:// endpoint and durable hosted instance are live and a production service is event-sourced on it; at that point it is no longer tracked as a workstream here.