← Workstreams

Workstream: CockroachDb

Status: Shipped · Deferred milestone: W3Wallet-gated accounts, quota, and billing · Component: Maximize developer productivity · Docs: project page

Goal

Give every service and agent a SQL database on demand: ask url://cockroachdb/ for a database and receive a handle speaking the ecosystem's established sql.Database contract — parameterized SQL execution, typed getters, row queries, atomic transactions, batch inserts — so creating, updating, and deleting databases, tables, and rows is a proxy API call rather than an exercise in provisioning clusters, distributing JDBC credentials, and hand-wiring TLS.

This is the SQL Database mechanism of the approved storage taxonomy made concrete as a hosted service — the tabular counterpart to Blobstore (BlobStorage) and SimpleFileSystem (FileStorage). It is equally a third-party service proxy in the GithubProxy mold: the service is the credential-containment boundary for CockroachDB cluster credentials and CockroachDB Cloud API tokens, keeping them out of adopting consumers.

Shape

The service follows the standard layered architecture as a CockroachDb-branded family — CockroachDbApi, CockroachDbEmbedded, CockroachDbServiceServer (at url://cockroachdb/), CockroachDbCli — and its data plane is the canonical instance of the multi-level layering pattern in the API architecture doc (which uses a CockroachDB-style managed-database proxy as its worked example):

Tier Interface Role
Org CockroachDbAccountManager Root entrypoint. Creates, lists, and addresses accounts — the multi-tenant chokepoint where ownership, quota, and (later) W3Wallet capabilities are enforced.
Account CockroachDbAccount One tenant. Carries its children's management methods: createDatabase / listDatabases / database deletion.
Database sql.Database The vended handle is the existing sql.Database contract. All table DDL and row DML flow through it as parameterized SQL.

Decisions that define the shape:

  • The data-plane contract is sql.Database, not a new API. The ecosystem's SQL surface already exists and every current consumer builds on it; the proxy serves that same interface over url:// rather than inventing a structured table/row API beside it. Consumers keep their programming model whether the database is local JDBC or remote proxy — the same deployment-spectrum indirection every service gets. Each vended database is URL-addressable (e.g. url://cockroachdb/accounts/<id>/databases/<name>).
  • Two account kinds from day one. An account is either a shared-cluster tenant — the operator configures the ServiceServer with one CockroachDB cluster (JDBC URL + TLS via CockroachDbPsqlSslFactory) and resells it, with per-account SQL users isolating tenants — or a bring-your-own CockroachDB Cloud account registered by its Cloud API token, whose databases are provisioned in the caller's Cloud account through the control-plane client cockroachdb.v1api. Both kinds vend the same sql.Database handles; per the proxy doctrine, a proxy supports bring-your-own-key and resells a shared one.
  • Capability granularity is the database handle. Holding a sql.Database handle is the authority over that one database (resource-handle semantics); finer-than-database attenuation is deliberately not offered, since arbitrary SQL inside a database cannot be meaningfully sub-scoped. Cross-tenant policy (who may create accounts and databases, quotas) lives on the manager tiers.
  • Remote-only members have explicit transport behavior. The transaction-lambda execute member runs over server-held sessions. getResultSet honors its callback contract by materializing rows server-side and replaying them client-side; true streaming is a future optimization. getConnection(): Connection is inherently in-process and is the one member the proxy does not serve (it fails with a descriptive error directing callers to the parameterized-SQL surface).
  • Authoritative state lives in the cluster itself. The account registry, database-ownership records, and BYO Cloud tokens are rows in the service's own metadata database inside the operator's shared cluster, keeping the ServiceServer ephemeral as required.
  • W3Wallet gating is a hardening milestone, not v1. v1 ships with plain ownership bookkeeping; W3Wallet-verified capabilities, quota, and billing bolt onto the manager tiers afterward — the same sequence Blobstore and SimpleFileSystem follow. The tiers are already capability-shaped, so gating does not reshape the contract.
  • Tests run against the real things. End-to-end tests (testing standards) exercise a real single-node cockroach process (downloaded and cached by the test harness) for the data plane — not a Postgres stand-in — and an in-process fake HTTP server implementing the handful of CockroachDB Cloud v1 endpoints the service uses for the control plane, the approved fallback for a dependency that cannot be self-hosted.

Current state

The layered family is shipped with green CI on main: CockroachDbApi, CockroachDbEmbedded, CockroachDbServiceServer, CockroachDbCli, CockroachDbHealthCheck, and CockroachDbTestHarness. The service is deployed on ContainerNursery at url://cockroachdb/, its URL_BIND_DOMAIN cold-start path has been verified, and its registered ProductionHealth check reports SUCCESS in the current state verified on 2026-07-17. See the CockroachDb project page for the shipped architecture, artifact versions, deployment, transport behavior, and known limitations.

Adoption and consolidation are also complete: the chatgpt cache uses CockroachDbApi without raw cluster credentials; cockroachdb.manager is a CockroachDbApi GUI client; cockroachdb.v1api verifies its HTTP control-plane client against in-process Jetty fakes, while CockroachDbTestHarness drives the published client through its shared fake with database and SQL-user effects backed by a real single-node CockroachDB; and cockroachdb.storage and cockroachdb.storage-tests run CI against real CockroachDB rather than a PostgreSQL substitute. The remaining W3Wallet gating milestone is deferred by user direction on 2026-07-17, not a graduation blocker.

Why it accelerates developers

  • A database becomes one API call. A new service or agent gets a real SQL database — creatable, queryable, transactional — by asking the proxy, instead of provisioning a cluster, minting SQL users, downloading CA certificates, and wiring DBCP2.
  • One credential boundary instead of credential sprawl. The proxy contains the shared-cluster credential and BYO Cloud tokens the way GithubProxy contains GITHUB_TOKEN; the migrated chatgpt cache now uses CockroachDbApi without raw cluster credentials.
  • The right tool for tabular data. The storage decision guide routes filterable, joinable, aggregatable structured data to SQL Database; this workstream makes that row of the taxonomy as effortless as Blobstore made BlobStorage.
  • Agent-ready. Reachable over url:// with URL-addressable databases, an agent swarm can create a scratch database, work in it transactionally, and hand its URL to another agent.

Plan / roadmap

  • [x] Split the sql contract from its implementation. The contract is published as sql:sql-api:0.0.1, so CockroachDbApi depends on the interfaces without the JDBC implementation.
  • [x] CockroachDbApi. The published contract provides CockroachDbAccountManager / CockroachDbAccount tiers vending sql.Database handles, both account kinds, URL-addressable resources, and descriptive domain exceptions.
  • [x] CockroachDbEmbedded. The shipped implementation provides lazy metadata bootstrap, per-account shared-cluster SQL users through CockroachDbPsqlSslFactory, and BYO Cloud accounts through cockroachdb.v1api.
  • [x] End-to-end test harness. CockroachDbTestHarness provides real single-node CockroachDB bootstrap with a concurrency-safe binary cache and Clock-driven waits, plus an in-process fake Cloud v1 API verified against the real Ktor client.
  • [x] CockroachDbServiceServer. The service hosts the Embedded at url://cockroachdb/, implements transaction lambdas over server-held sessions, honors the getResultSet callback contract by server-side materialization, and supports ContainerNursery lazy-start. True getResultSet streaming remains a future optimization.
  • [x] CockroachDbCli. The CLI provides account/database lifecycle commands, parameterized SQL execution and querying, --json, health, and embedded mode against a directly configured cluster.
  • [x] Deploy and monitor. The ServiceServer is deployed on ContainerNursery, CockroachDbHealthCheck is registered with ProductionHealth, and the deployment has a page in ServiceAtlas.
  • [x] Adoption and consolidation. The chatgpt cache uses CockroachDbApi, cockroachdb.manager is a CockroachDbApi GUI client, cockroachdb.v1api exercises its HTTP control-plane contract against in-process fake servers, and CockroachDbTestHarness verifies the published client against its shared fake while a real single-node CockroachDB backs database and SQL-user mutations. CI in cockroachdb.storage and cockroachdb.storage-tests exercises real CockroachDB.
  • [ ] W3Wallet-gated accounts, quota, and billing. The post-v1 hardening milestone: verified capabilities on the manager tiers, per-account database/size quotas, and metered billing for shared-cluster tenants — the W3Wallet "secure my service" path. (deferred — W3Wallet not ready; user direction 2026-07-17)

Graduation

Graduated (2026-07-17). The layered family, deployment, monitoring, and adoption criteria are met; CockroachDb is now a first-class project documented on the CockroachDb project page. The deferred W3Wallet milestone remains visible above as future work.

When the Api / Embedded / ServiceServer / CLI family exists with green CI, the hosted instance is deployed at url://cockroachdb/ with a registered health check, and at least one platform consumer has migrated onto it, this graduates to a first-class project: a project page in the Documentation Repository and a row in ALL_PROJECTS.md.