← Workstreams

Workstream: UrlResolver HTTPS Transport

Status: Planned · Component: Maximize developer productivity

Goal

Let a url:// service be hosted behind plain HTTPS — optionally upgraded to WebSocket — so it can run anywhere an HTTPS endpoint can: on Cloud Run and other request-driven platforms that scale to zero, or behind any HTTPS reverse proxy, load balancer, or CDN. Every resolver reaches such a service directly, either by its ordinary mesh name (url://my-service/) or by an explicit address that needs no mesh at all: url://protocol.https.icann.com.mydomain.www/.

The motivating gap: the only production transport that reaches another machine beneath UrlProtocol/UrlResolver is libp2p over raw TCP, so hosting a url:// service means owning a machine that can hold a long-lived socket. A platform that admits only inbound HTTPS on one port, terminates TLS at its own front end, and runs an instance only while a request is in flight cannot accept a libp2p dial, and cannot keep the standing relay registration a NAT-ed node relies on either — an instance that is scaled to zero is registered nowhere, and no request arriving over the mesh can wake it. The cheapest, most widely available hosting there is — "give me an HTTPS URL and bill me per request" — is therefore closed to url:// services today.

Current state

  • One production mesh transport. Libp2pNetworkProvider — TCP + Noise + stream multiplexing — is the default behind the NetworkProvider seam that the Pluggable Network Layer workstream delivered. The seam itself takes addresses as opaque strings, so a second link type fits beneath it; but the peer records, announcements, and peer management above it still read every address as a libp2p multiaddress, and that assumption is what a second link type has to lift.
  • One address-is-the-transport precedent. url://localhost.tcp:PORT/ already bypasses libp2p for a container reached through ContainerNursery's url:// facade — the address itself says how to connect. It is a single pattern-matched special case, not a general scheme.
  • Hierarchical names are walked from the left. In the shipped hierarchical name verification, the leftmost segment of a dotted name is its top-level domain (secure.service belongs to the TLD secure), a TLD's authority comes from a root-signed descriptor, and a TLD with no descriptor is free to claim. Every level follows one fixed rule — its children are whoever its key signs — so no level can yet run its own logic, and no TLD resolves a name by leaving the mesh.
  • An application-level web binding is planned separately. The Observables web transport serves projections to non-resolver clients (a browser, curl) as an HTTPS snapshot plus a wss delta stream. That makes one kind of data web-readable; it does not carry url:// itself.

The shape

Two independent pieces, each useful without the other (the second also depends on per-level resolution code in the UrlResolver workstream): an HTTPS link type beneath the network seam (how bytes reach a service), and a protocol top-level domain in the name hierarchy (how an address can say "reach this over HTTPS at this DNS name"). The plan stops at the contracts below; the wire-level design is written in UrlProtocol's own documentation as part of the work.

HTTPS as a link type

HTTPS joins libp2p-over-TCP as a second way two nodes exchange the same RPC frames beneath NetworkProvider. It is a second binding of one protocol, not a parallel stack: the sandbox model, W3Wallet authorization, and every API a service author writes are unchanged, and a node may listen on both link types at once under its one peer identity.

  • HTTPS is the envelope; the session inside it is end to end. The frames travel inside a session secured between the two parties' own keys, exactly as on a libp2p link, and each side's identity is whatever that session authenticates. A front end that terminates TLS merely forwards the session: it can read or alter it only if it could also supply the key the session is secured to. What vouches for the service's key is the one thing that differs between the two ways of addressing a service, and so decides whether a front end is such a party.
  • Any HTTPS intermediary works. The binding asks nothing of what sits in front of the service beyond forwarding ordinary HTTPS (and the standard WebSocket upgrade, where offered). TLS may terminate at the platform's front end or a reverse proxy, and the client may itself be dialing out through a forward proxy.
  • One identity, many instances. A platform may run several interchangeable instances behind one endpoint and send consecutive requests to different ones. All instances serve under the service's single identity, and every exchange is secured to that identity on its own, so it can land on any instance.
  • Delivering an exchange twice gains nothing. An intermediary may retry a request, and a hostile one may replay it. The reply is readable only by the original caller, and a state change is a command, which a command-outcome store shared across the instances runs once however many times its exchange arrives.

With WebSocket, and without it

Two bindings carry the frames. A request/response exchange — one HTTPS request, with the call's frames in its body and the reply's frames streamed back in its response — is the baseline every HTTPS path supports, and the unit a request-driven platform wakes on, bills for, and load-balances. A WebSocket on the same endpoint is the optional upgrade: one long-lived, two-way connection per client that multiplexes every call, push, and subscription that client has with the service — the HTTPS counterpart of today's persistent RPC connection.

The client learns which bindings an endpoint offers from the endpoint itself and upgrades when it can. When it cannot — the host does not offer WebSocket, or an intermediary on the path strips the upgrade — everything in the middle column still works, and the caller can always see which binding a connection is using.

What a call needs Request/response only With WebSocket
Plain RPC, client-bytecode fetch One exchange Multiplexed on the one socket
Streamed parameters Sent as the request body, before the reply begins Interleaved with the reply
Streamed results; one-way effects (warnings, notifications) raised to the caller Frames in the exchange's response. An intermediary that buffers responses delays them; it cannot reorder or drop them Delivered as they happen
Cancellation Closing the exchange A frame; the socket stays up
Live projection, observed command A held request the server answers when the state moves — the level-triggered path the push transport already falls back to, without a timer Pushed deltas
An effect that needs the caller's answer; an interactive two-way stream; a push-only API Not available: the call fails with an error naming the missing binding Yes
Mesh participation — gossip, link-state adjacency, relaying No: the node is a leaf Yes

The gap in the middle column is deliberate. Anything that needs the client to speak again in the middle of a call would have to arrive as a second request, and a second request may reach a different instance than the one holding the call.

A platform also bounds how long any one connection may live, a WebSocket included. That bound is routine, not a failure: the client replaces a connection ahead of it — new calls move to the replacement while calls in flight finish on the old — and subscriptions re-establish exactly as they do after any reconnect. A single call cannot outlive the bound, so long work is always a command that is observed and resumed, never one blocking call.

Staying awake: the hold contract

A request-activated host makes progress only while a request is open on the instance doing the work — Cloud Run, in its default billing mode, allocates CPU only during request processing and ends any request at its timeout. Left implicit, that turns "the response was sent" into "the work silently stopped." The transport instead makes it an explicit agreement that neither a service author nor a caller writes any code for:

  • The server states its terms. On first contact an endpoint tells the client three things: that it is request-activated, how long one exchange may live on its platform, and which bindings it offers.
  • The client holds for exactly as long as it is interested. While a resolver has live interest in such a service — a call not yet answered, an observed command not yet terminal, a live subscription — it keeps an exchange open to it, and lets go when the last interest ends, leaving the instance free to sleep. An instance is therefore awake exactly while someone is waiting on it, and what that costs is attributable to who is waiting.
  • One hold per client per service. Everything a client is watching on a service — its subscriptions and observed commands — shares one held exchange; a call in flight is its own exchange. Without WebSocket the held request names what is being watched, so the client replaces it when that set changes. Holds occupy the platform's per-instance concurrency, so they are never one per subscription.
  • The hold reaches the work. Over WebSocket the socket is the hold and stays with one instance. Without it, a call's own exchange stays open while the call runs, and a watched command advances on whichever instance receives the client's held request, resuming from durable state if it was running elsewhere. Ending an exchange releases the hold but does not prove the previous instance has stopped — a platform may let a handler run on after its request is closed — so the command's authoritative state admits one instance at a time and fences out any other.
  • Silence is survived. A held exchange stays usable through intermediaries' idle limits. Over WebSocket that is the protocol's existing ping frames. Without it the client cannot speak again on an open request, so the server keeps an open reply alive from its side, and a held request with nothing to report is answered and re-issued rather than left silent.
  • Unwatched work is paused, never lost. Work nobody is holding stops advancing when the instance loses its CPU and continues the next time the service is awake. That is safe only when a command's in-flight progress is authoritative in durable state, per Long-Running Operations and Authoritative Progress — a requirement on every command a request-activated service offers, and distinct from the shared command-outcome store in the Observables plan, which makes a command's terminal result replayable across instances but records no progress. A service whose work must finish with nobody watching is deployed with CPU always allocated, which makes it an ordinary always-on peer — a declared deployment choice, not something discovered in production.

What crosses with a call

Every ambient that accompanies a call today rides the RPC frames, and the frames cross HTTPS intact. Nothing ambient is translated into HTTP — no header, cookie, or query string — because intermediaries log, rewrite, and size-limit those, and because one carrier with one codec is what keeps the two link types from drifting apart.

What crosses Over HTTPS
The caller's identity Not an ambient but a property of the connection: the key the end-to-end session authenticates, as on a libp2p link. Never an HTTP header, and never what a front end reports about the client. It identifies a peer; what the caller may do is still decided by the wallet capability it presents.
Required freshness — the ClockPad ambient Unchanged: the pad rides every request — in its metadata for a plain call, in its parameters for an observable read — and the reply's metadata returns the serving clock's timestamp and coverage receipts for the resolver to merge. Every exchange carries the whole required pad, which is what makes interchangeable instances safe — an instance that has not yet seen the caller's writes must satisfy the pad or report that it cannot; it can never answer stale.
The wallet ambient The same capability carrier the W3Wallet plan defines for url://, in the same frames: a capability reference, or an audience-bound, holder-bound token. Where that plan binds a session key to its channel, the channel is the end-to-end session — under either way of addressing — never the TLS connection that ends at a front end. This workstream adds no second carrier.
Effects Raised to the caller in the reply's frames; an effect that needs an answer requires the WebSocket binding (above).
Observation scope Process-local, as today; invalidations cross as projection updates.

Token capabilities verify offline, so they suit a request-activated service best — no dial is needed to honor one. A reference capability still works: the service dials the grant's rendezvous address while the request is open, when it has CPU. Either way a service that sleeps wakes with cold caches, so anything it caches — a verifier key, a peer list — must be re-derivable.

The browser-facing mechanisms are a different surface and are not reused here: the Authorization header of the Observables web transport and the X-W3Wallet-Daemon-Url header exist for clients that have no resolver.

Two ways to address an HTTPS-hosted service — and two trust anchors

Mesh name — url://my-service/ Explicit address — url://protocol.https.icann.com.mydomain.www/
Who may serve it The peer key the name's verified delegation chain authorizes Whoever legitimately serves HTTPS for that DNS name
What vouches for the service's session key The delegation chain names it — the TLS terminator is never trusted for identity The hostname's WebPKI credential authenticates the key. A front end that holds that credential is therefore itself a trusted endpoint of the service; one that does not forwards ciphertext
Who could impersonate the service Nobody without the delegated key; a front end sees only ciphertext Whoever controls HTTPS for the hostname, its TLS-terminating front end included — precisely the trust a browser extends
How the endpoint is found A location hint (gossip, or the name's signed record) DNS
Needs the mesh Yes No — no bootstrap peer, no gossip, no peer ID

For a mesh name, an HTTPS address is only ever a hint, exactly as a multiaddress is today: the dial is pinned to the expected peer identity, so a forged hint pointing a name at someone else's HTTPS endpoint wastes a dial and grants nothing.

An explicit address is walked from the left like every other url:// name:

url://protocol.https.icann.com.mydomain.www/
      │        │     │     └─ the DNS name www.mydomain.com, root-first like the rest
      │        │     └─ naming authority: the public DNS hierarchy resolves what follows
      │        └─ transport: HTTPS
      └─ TLD: names reached at an external address rather than through the mesh

protocol is an instance of exactly what verified naming anticipates — a name node whose implementation consults an outside authority. That makes it dependent on resolution as code per level from the UrlResolver workstream, which is not built yet: the verifier that has shipped can only check that each child was signed by its parent's key, and cannot express a level that checks DNS answers and certificate chains. Both are needed eventually; the HTTPS link type, by contrast, does not depend on it and can be built first, reached by mesh names. From the icann level down, authority passes out of the url:// hierarchy into an existing delegation system adopted wholesale: the public DNS hierarchy says where the host is, and WebPKI says who may speak for it. The two levels beneath protocol exist because transport and naming authority vary independently.

The plan

  • [ ] Carry url:// frames over HTTPS beneath the seam. An HTTPS link type alongside the libp2p default, selected by the address being dialed, with a node able to listen on both. This includes lifting the multiaddress assumption above the seam, so peer records, announcements, and peer management carry an HTTPS location as readily as a libp2p one.
  • [ ] Both bindings. Request/response and WebSocket each carry their column of the binding table; a client discovers what an endpoint offers, keeps working when the upgrade is unavailable or stripped in transit, exposes which binding is in use, and replaces a connection ahead of the platform's bound without failing a call.
  • [ ] The end-to-end session. As described under HTTPS as a link type: the same session under both ways of addressing, each exchange secured on its own, and a twice-delivered exchange harmless. An HTTPS address learned from gossip is identity-pinned like any other dial hint.
  • [ ] The hold contract. As described under Staying awake.
  • [ ] Ambients cross in the frames. As described under What crosses with a call. Shared with the url:// propagation items in AMBIENTS.md: this workstream carries whatever those items define, and defines no carrier of its own.
  • [ ] Request-activated hosting: asleep is not offline. A service hosted this way holds no connection while idle and need not be running at all — the dial is the wake-up. Three rules make that work with the rest of the network:
    • Its location is durable, not connection-derived. A sleeping service cannot announce itself, so where to dial it is published as a standing fact: in the service's signed name record under verified naming, and — for names that are still free to claim — as a standing announcement from an always-on node its operator runs, the role ContainerNursery's url:// facade already plays for its own sleeping containers. A published location says that the service is request-activated, so every node that learns it knows not to probe it. Whoever publishes the location never carries the traffic. A service with neither is reachable only by its explicit address.
    • It is never probed, and never treated as offline for being asleep. A synthetic health check against a request-activated endpoint wakes and bills it, defeating the point. Its reliability and reachability evidence accrues from real traffic only, which the peer-management design already records exactly like a scheduled check, and its location is shared onward without the reachability check that peer exchange applies to connection-held peers — that check applies to the always-on node publishing it instead. That design and UrlProtocol's peer-sharing rules gain this exemption as part of this item.
    • It is a leaf. It serves its own names and dials what it needs while handling a request, but it never relays, never acts as a guard peer, and never appears as transit in the link-state graph.
  • [ ] An HTTPS endpoint is a public address. Any resolver can make an outbound HTTPS request, so an HTTPS-hosted service is dialed directly by every client and never through a relay — the direct-connection preference in its simplest case.
  • [ ] Join the mesh over HTTPS. A continuously running node behind an HTTPS front end is a full mesh peer over WebSocket — gossip, link-state adjacency, relaying — and the public bootstrap nodes offer a WebSocket endpoint, so a resolver whose network permits only HTTPS egress still joins the mesh and stays reachable through that link.
  • [ ] The protocol top-level domain. url://protocol.https.icann.<dns-labels-root-first>[:port]/ (so www.mydomain.com is …icann.com.mydomain.www) resolves with DNS and connects over HTTPS, authorized by the hostname's WebPKI certificate. Unlike every other TLD, whose resolver is distributed over signed gossip, this one ships pinned inside the resolver, because these names must resolve for a resolver that has never met a peer. Names beneath protocol are never satisfied by a mesh announcement: a peer that gossips a claim to protocol.https.icann.example.bank is not routed to.
  • [ ] One endpoint, shared with the web. The url:// binding lives at a reserved well-known path on its host, so one HTTPS origin serves a service's url:// endpoint, its WUI, and its Observables web transport side by side.
  • [ ] Write the wire-level design where it belongs. The framing of an exchange, the end-to-end session, the terms an endpoint states, and the hold are specified in UrlProtocol's documentation alongside its network architecture, before the first of them is built.
  • [ ] End-to-end tests. Per the testing standards, fully offline — every test supplies its own DNS answers and its own certificate authority, never the public internet — and through real resolvers and a real HTTPS listener behind a local front end that behaves as a request-driven platform does.
    • Addressing and trust. A resolver with no bootstrap peers opens a typed connection to an explicit protocol.https.icann.… address, and is refused when the endpoint's certificate does not cover the hostname. A mesh name behind the front end is reached directly with zero relay hops; a gossiped hint pointing that name at an endpoint holding a different key is rejected; a front end that alters a frame in transit is detected; and a mesh announcement for a name under protocol is never routed to.
    • Bindings. Every row of the binding table is exercised over each binding it claims. With the upgrade stripped in transit, plain calls, streamed results, one-way effects, and a live projection all still work, and a call that needs the caller's answer fails with the error naming the missing binding.
    • Holding. An observed command keeps advancing for as long as its handle is observed and completes across several forced connection replacements without a failed call; a held exchange outlives the front end's silence cutoff; a command whose observer goes away stops advancing and, when observed again, resumes from its durable progress on a different instance, with the first instance fenced out if it is still running.
    • Sleeping. A request-activated service receives no request across an idle period, remains resolvable afterwards, and is started by the first real call.
    • Ambients. A write served by one instance followed by a read served by another reflects the write or reports that it cannot — never a stale answer. A caller identity asserted in an HTTP header is ignored. A wallet capability presented through the front end is honored, and a command whose exchange the front end delivers twice runs once.
    • Mesh over HTTPS. In a NetLab topology whose firewall permits only outbound HTTPS, a resolver joins the mesh through a public node's WebSocket endpoint and reaches a NAT-ed service.

Decisions

  • Explicit addresses are spelled root-first all the way through: url://protocol.https.icann.com.mydomain.www/ (decided 2026-10-05). Every segment, the DNS labels included, reads from the root outward, so a name never changes direction partway. The fully reversed alternative, url://www.mydomain.com.icann.https.protocol/, would be equally consistent on its own, but it would make protocol the rightmost segment while every existing url:// name (secure.service, vpn.myservice) has its TLD on the left — giving the namespace two walk directions chosen by the last segment. Which segment is the TLD decides whether a name is verified or free to claim, so exactly one rule may answer it: the leftmost segment, as shipped. Beneath icann each label is one more level of delegation — com, then mydomain, then www — the same walk DNS itself performs, written in the same direction as the rest of the name.
  • HTTPS is a link type beneath NetworkProvider, not a gateway service. No node sits between a client and an HTTPS-hosted service translating protocols; a relay in the data path is the thing the UrlResolver workstream is removing.
  • WebSocket is not emulated over paired requests. Pairing a held download with separate uploads only works when consecutive requests reach the same instance, which a load-balanced platform does not promise. Calls that need a two-way conversation require the real upgrade and say so when it is missing.
  • One session, two anchors. The end-to-end session is the same under both ways of addressing, so caller identity, wallet channel binding, and frame protection have one definition. Only what vouches for the service's key differs: a mesh name's delegation chain, or — for an explicit address and nothing else — the hostname's WebPKI certificate. A mesh name's authority remains its delegation chain even when the service happens to be hosted over HTTPS.
  • protocol is reserved now. No mesh service may be named protocol or anything beneath it. No production service is; one UrlResolver stress test announces synthetic protocol.close.… names and is renamed as part of this work. The reservation takes effect with this plan rather than with the first resolver that pins the TLD, so no name can be caught between the two.
  • Ambients never become HTTP headers. One carrier and one codec serve both link types, and capability material stays out of everything an intermediary logs.
  • Progress on a request-activated host is tied to an open exchange, and the transport says so. The hold is automatic and derived from the client's live interest; the alternative — letting work quietly stall after a response — is the failure this contract exists to prevent. Unattended work is a deployment choice (always-allocated CPU), never an accident.
  • localhost.tcp:PORT is left as it is. It is the same idea — an address that names its transport — and protocol is the general home for that idea, but respelling the existing local mode is not part of this workstream.
  • A request-activated service keeps no state on its own disk. An instance may vanish between any two requests, so state lives behind url:// in the storage primitives — Event Log, SimpleFileSystem, Blobstore, CockroachDb — the same separation the plan's storage todo asks of ContainerNursery-hosted apps so they can move to other hosts.

Why it accelerates developers

  • Hosting becomes a commodity. Any platform that hands out an HTTPS URL can host a url:// service; idle services cost nothing, and a service can leave the shared ContainerNursery host without changing its API, its clients, or its name.
  • It removes a relay, not adds one. An HTTPS endpoint is dialable by everyone, so services hosted this way take load off the public relay nodes instead of leaning on them.
  • url:// works from locked-down networks. A developer or agent behind an HTTPS-only corporate network can still join the mesh and call every service.
  • A service is reachable before any mesh exists. An explicit address needs only DNS and HTTPS, so a resolver with no peers, no bootstrap list, and no gossip can still open a typed connection — the smallest possible first step onto the fabric, and a natural door for the open ecosystem contender for component 3.

Graduation

This workstream graduates when:

  1. A url:// service hosted behind a TLS-terminating HTTPS front end is reached by mesh name, directly and with its peer identity verified end to end.
  2. url://protocol.https.icann.com.mydomain.www/-style addresses resolve and connects for a resolver with no peers configured.
  3. Request-activated hosting holds: an idle service receives no traffic from the network and is still resolvable and callable, and work on it advances for exactly as long as a client holds it — pausing and resuming, never lost.
  4. Both bindings carry what the binding table claims for them — including the caller's identity, the ClockPad, and the wallet carrier — and the WebSocket binding carries mesh participation, including joining the mesh from an HTTPS-only network.
  5. The end-to-end tests above gate UrlProtocol and UrlResolver CI.
  6. At least one production url:// service runs on Cloud Run, scaled to zero when idle, and is called both by its mesh name and by its explicit address.

At that point, document the transport and the protocol TLD on the existing UrlResolver project page and add HTTPS hosting to the deployment model — no new project page.