Stream legacy bytecode replies from UrlProtocol's ordinary TCP reply path so the manager's 32 MiB guard holds for published clients

Design decisions in force (recorded 2026-10-27 04:50 UTC)

• Owner direction, verbatim (2026-10-27): "Please decide on a good solution for https://www.handoff.wasmserver.com/handoffs/hf-2026-10-26-fix-published-client-compatibility-and-bounded-bytecode-replies-for-the-manager-server and delegate the work to gpt 6.1 sol, drive it to completion." This supersedes, for this effort, the 2026-10-26 constraints "No changes to UrlProtocol or UrlResolver" and "No merge, enqueue, deploy or production changes": the only implementation left is an upstream UrlProtocol change, and the owner asked for the effort to be completed. Interpretation recorded by the orchestrator: green, independently reviewed pull requests are merged by the orchestrator; a production deploy still waits for an explicit request.
• Design chosen: UrlProtocol's ordinary TCP reply job (TcpServer.encodedResponseWriteJob, the writer used for every non-streaming reply) streams the JSON reply to the socket in two passes over the same serializer: pass one counts the exact UTF-8 byte length for the 4-byte length prefix without accumulating text; pass two writes the prefix and the JSON through a fixed-size buffer. Wire bytes stay identical, so published legacy clients are unaffected. The response-outbox accounting keeps charging the reply's wire size, so allowance and read-pause behaviour is unchanged; only the physical copies (whole text, whole UTF-8 array, whole frame) disappear. The manager service then bumps its foundation.url:protocol pin and gains a fail-first legacy-client test under the unchanged 32 MiB guard. Nothing about the streamed path changes.
• Still in force from 2026-10-26: streamed clients keep the shared prepared copy and the 32 MiB guard; legacy clients get byte-identical replies and types; no heap, timeout or retention limit is raised; no assertion weakened; upstream fix, never a downstream workaround.

State at 2026-10-28 08:45 UTC (session resumed; delegate quota restored)

• UrlProtocol fix is complete and reviewed, being rebased. https://github.com/CodexCoder21Organization/UrlProtocol/pull/967 (branch fix/legacy-json-reply-streams-to-socket, reviewed head 2ae01d8a57a9e11fe1f6e88eb503480f68c208e8 on old main 391ae3940). Content: prepare-once immutable JSON form (String leaves referenced, everything else rendered into 64 KiB chunks), exact UTF-8 count for the length prefix, single 64 KiB frame buffer with direct writes at or above 64 KiB, attachments copied inside the encoding lease, prefix and body in one buffered write. Five test-comprehensiveness reviews, four adversarial reviews and the orchestrator's final review are clean on this content; CI was green on that head. Main moved under it overnight (971, 974, 945 = libp2p snapshot-41 pin bump across build.kts and ~2,700 test scripts, 979 = TCP_NODELAY on accepted sockets, 976, 963), so it is unmergeable until rebased: a mechanical rebase is in progress (conflicts expected only in build.kts coordinates, README.md, and the one TcpServer.kt line from 979; the 31 new test scripts move their libp2p pin to snapshot-41 as a separate commit so the reviewed commits stay intact). After the push, the before/after git diff <base>...<head> -- src tests comparison (ignoring only the pin lines) must be empty before it is enqueued.
• Release coordinates changed to foundation.url:protocol:0.0.1461-legacyjson-20261028 (was 0.0.1455). Reason: other sessions published 0.0.1455-rank-20261027, 0.0.1456/1457/1458/1459 with suffixes and a plain 0.0.1460 while this effort was paused, and main now declares 0.0.1457-rank-20261027; a suffixed, dated number cannot collide. The manager pin bump targets this version.
• The flake that evicted 967 is gone from main. testRejectedDrainLoggingDoesNotHoldOutboxMonitor has zero first-attempt failures in four consecutive main runs since https://github.com/CodexCoder21Organization/UrlProtocol/pull/945 landed (22:08 UTC 2026-10-27; e.g. https://github.com/CodexCoder21Organization/UrlProtocol/actions/runs/40639149474, 3,602 tests), and so does testOrdinaryGossipFailureLoggingFailureIsIsolated. The lost close completion was in the pinned libp2p snapshot-25; snapshot-41 appears to fix it. A lane is measuring the deterministic held-full-queue control against snapshot-41 to decide whether https://github.com/CodexCoder21Organization/UrlProtocol/pull/978 (this effort's fixture fix) is still needed (then rebase it) or superseded (then close it with a linking comment). https://github.com/CodexCoder21Organization/UrlProtocol/pull/975 and https://github.com/CodexCoder21Organization/UrlProtocol/pull/977 belong to other sessions and are not touched. The fresh-context reviews of 975 and the second-flake fix lane planned on 2026-10-27 are dropped: nothing in this effort depends on them any more.
• Other first-attempt failures seen on main since then (not this effort's): stressTestPeerExchangeHostShutdownCancelsQueuedAndDrainsRunning (3/3 in PR 963's first queue run, 1 retried on main's latest run) and testRelayWithdrawalRegistryWaitDoesNotDelayRootClose (1 retried). If 967's queue run is evicted by one of these, it gets root-caused and fixed as its own pull request before re-enqueueing, per the standing rule.
• Manager side: branch fix/legacy-bytecode-reply-bounded-heap on https://github.com/CodexCoder21Organization/AiCliSupervisorManagerServiceServer holds the fail-first legacy-client test (commit https://github.com/CodexCoder21Organization/AiCliSupervisorManagerServiceServer/commit/7d8672d067007f78af3a1da0110427448de23c3f, red against protocol 0.0.1434). Its pull request bumps the pin to 0.0.1461-legacyjson-20261028 and is cut only after that version is published.
• Publishing rule for this effort: publish foundation.url:protocol:0.0.1461-legacyjson-20261028 from merged main, built locally (--local, -Xmx2g) and uploaded with the community-kotlin-maven-artifact-publishing CLI against https://api.kotlin.directory/upload; CI never publishes. Probe https://kotlin.directory/foundation/url/protocol/0.0.1461-legacyjson-20261028/protocol-0.0.1461-legacyjson-20261028.pom (must be 404) before publishing; the correct path is foundation/url/protocol/<version>/protocol-<version>.pom (group foundation.url, artifact protocol).
• Infrastructure records from 2026-10-27: https://github.com/CodexCoder21Organization/PlanRepository/blob/main/challenges/2026-10-27-1322-filedrop-service-unreachable-wui-and-url-filedrop-on-2026.md, https://github.com/CodexCoder21Organization/PlanRepository/blob/main/challenges/2026-10-27-1718-handoff-service-url-handoff-and-https-www-handoff.md and https://github.com/CodexCoder21Organization/PlanRepository/blob/main/challenges/2026-10-27-1735-kotlin-directory-maven-downloads-timed-out-inside-a.md. PR 967's native evidence archive (SHA-256 f52b5a766dbc13ee0581c89b3812aefffd92d8ba4c6b68b41b828b5eaf7235c5) could not be uploaded to filedrop; the counts in the description come from the runner's own XML.

Next steps, in order

1. Finish the mechanical rebase of PR 967 (in progress); verify the src/tests patch comparison; enqueue with build-watchman --to-merged; audit the queue run's first attempts; confirm the change on main. Any eviction is diagnosed from the run's records before any re-enqueue.
2. Settle PR 978 from the snapshot-41 measurement (in progress): rebase and land it if the fixture ordering still fails deterministically; otherwise close it with the explanation and links.
3. Publish 0.0.1461-legacyjson-20261028 per the rule above; open the manager pin-bump pull request from the existing branch (description leads with why: the fail-first test and the owner's request), run fresh-context test-comprehensiveness and adversarial reviews and CI, merge.
4. Complete this handoff. Separately decide whether to open a follow-up handoff for the router-side libp2p response writer in ContainerNursery, which still aggregates whole legacy frames (production trace evidence: https://github.com/CodexCoder21Organization/UrlProtocol/tree/wip/c1-manager-transport-trace); the orchestrator's default is a follow-up handoff rather than widening this effort.

The remainder of this document is the situation brief written 2026-10-27 04:30 UTC, kept as background.

Legacy bytecode replies exceed the manager service's memory bounds inside UrlProtocol's ordinary TCP reply path

Written 2026-10-27 ~04:30 UTC by opus-code-2026-10-27, at the owner's request for a situation brief.

RE-VERIFY: everything below is a write-time snapshot. Recheck UrlProtocol main (here 1228066b), AiCliSupervisorManagerServiceServer main (here c1bbd461), the PR states and https://kotlin.directory/foundation/url/protocol/maven-metadata.xml (latest release here 0.0.1441) before acting.

What this brief is for. It describes the situation, the evidence, the constraints and the references. It does not propose a design or implementation; choosing one is the first job of whoever picks this up. Earlier versions of this handoff named an approach ("stream encodeMessage"), and that wording is deliberately withdrawn. Prior attempts are described below only so their results aren't rediscovered, not as recommendations.

The situation

The AiCli supervisor manager service (AiCliSupervisorManagerServiceServer) serves its client implementation as bytecode over UrlProtocol. Clients fall into two groups:

• Streaming-capable clients advertise the released metadata bytecodeResponseFormat=streamed. Since AiCliSupervisorManagerServiceServer PR 45 (merged 2026-10-27 01:12:42Z), they get a streamed reply backed by one shared prepared copy, and its memory guards pass.
• Legacy clients (any published client that doesn't advertise the capability) must keep getting exactly the reply format they already understand: a JSON message whose fields carry the implementation and stdlib as base64 strings, byte-identical to main.

For legacy clients the reply is built and sent by UrlProtocol's ordinary TCP reply path, not by the service. That path materializes the whole reply before writing it:

• JsonWireEncoding.kt:37-42: jsonWireBytes / jsonWireString produce the complete JSON text, then its UTF-8 bytes (through StringBuilderWriter, line 402, and JSONObject.quote).
• Libp2pRpcProtocol.kt:3468: encodeMessage allocates the complete length-prefixed frame from those bytes.
• TcpServer.kt:592-607: encodedResponseWriteJob calls encodeMessage and retains the frame until the write finishes. Ordinary replies reach it at lines 2397 and 2432.

Measured sizes (exact public-encoder measurements, from the L190 report)

A single legacy frame exceeds the retained ceiling by 1,490,123 bytes, independently of heap size, so the problem appears with one client, not only under concurrency.

Observed failure

Under the service's unchanged 32 MiB heap guard, the first legacy reply fails with OutOfMemoryError during encoding. Stack: Arrays.copyOf ? AbstractStringBuilder.ensureCapacityInternal ? StringBuilder.append ? StringBuilderWriter.write (JsonWireEncoding.kt:312 in the published 0.0.1434) ? JSONObject.quote ? jsonWireString ? jsonWireBytes ? Libp2pRpcProtocol.encodeMessage ? TcpServer.encodedResponseWriteJob ? TcpServer.ConnectionHandler.buildResponseWriteJob. The OOM doesn't report the failed array length, so none is claimed.

These existing service tests reproduce it. In L190 both failed their first attempt and every automatic retry:

• tests/testBytecodeReplyFitsBoundedHeap.kts
• tests/testPausedBytecodeClientsShareStorageUnderConstrainedHeap.kts

Related tests on main: testConcurrentBytecodeRepliesRetainOnePayloadAndReleaseOnDisconnect.kts, testLargeRelayedBytecodeResponseSurvivesConstrainedHeap.kts, testBytecodeReplyPreservesClientBytes.kts, testStreamedBytecodeCapabilityAndOwnership.kts.

Baseline retained bytes on the original PR head: 1 client 4,863,435; 2 clients 9,726,870; 8 clients OOM at 32 MiB.

Constraints in force (owner decisions, verbatim where quoted)

From the owner on 2026-10-26:

• "Clients that advertise the released bytecodeResponseFormat=streamed capability get the streamed/attachment reply, built from one shared prepared copy, with memory bounded regardless of how many such clients connect."
• "All other clients get exactly main's current legacy reply (byte-identical behaviour and types). No regression for them; their memory cost is unchanged from main."
• "The PR's own new memory guards apply to the streamed path (keep the 32 MiB limit and the byte-identity checks)."
• "No changes to UrlProtocol or UrlResolver." Still in force. Any work in UrlProtocol needs this lifted explicitly by the owner, and as of this brief it hasn't been.
• "No merge, enqueue, deploy or production changes."
• "If after this you can show the 32 MiB guard still cannot pass without a protocol change, stop and report the exact allocation (stack + sizes); do not change the limit." That stop condition was reached (L190).

Standing rules: don't raise heap, timeout or retention limits; don't cut iteration counts; don't weaken assertions; keep the legacy wire schema and exact bytes; fix upstream rather than working around it downstream (Upstream Bug Fixes).

What has been tried, and what it showed (context, not recommendations)

• Caching one immutable shared copy of the legacy base64 strings in the service (L190, branch wip/hd22-L190-legacy-allocation at 0925d83a). This removes the service's own preparation copies, but still OOMs on the first single-client reply, because UrlProtocol's encoder makes its own whole-JSON and whole-frame copies.
• Limiting concurrency with a permit in the service. This can't help, because the allocation happens with one client. Also, returning from the application handler happens before the transport allocates and writes the frame, so a permit released at handler return doesn't cover the transport's allocation.
• Existing streaming markers produce final jarBytes / implClassName and can't carry the legacy string schema that published clients require.
• UrlProtocol.bind exposes no writer injection or completion hook to the service.

Related open UrlProtocol PRs (neither resolves this as-is)

• UrlProtocol PR 696, "Stream large legacy bytecode replies through bounded response writes". OPEN/DIRTY, idle since 2026-10-13, head b804211c. Its change targets the StreamingBytecodeResponse writer in Libp2pHostFactory, refuses frames above 4,194,304 bytes, and doesn't change the ordinary encodeMessage path used here.
• UrlProtocol PR 765, "Bound UrlProtocol reply buffering for slow readers". OPEN/DIRTY, idle since 2026-10-21, head 21a2088c, with a recorded remote check failure. It touches reply buffering and frame production.

Whoever picks this up should read both, to know what already exists and where it conflicts with current main.

Who is affected and how much (known and unknown)

• The advertiser of streamed on bytecode forwards is ContainerNursery's router, not the end resolver. It first appeared in CN 0.0.140 with resolver 0.0.595 and an explicit protocol 0.0.334 (ContainerNursery commit ae48a399). The deployed router source identified in L182 (d84bf644) advertises it. So CN-routed manager bytecode fetches are expected to take the streamed path once PR 45 is deployed. Direct requests without the capability stay legacy.
• Unknown: which binary is deployed for the manager route (container logs showed no build identity), the traffic-weighted share of legacy requests, and the deployed versions of each consumer. Known consumers by source: WUI (resolver 0.0.1337) and HandoffServiceServer (resolver 0.0.1263).
• Nothing was deployed. Whether a deploy is needed at all is unestablished.

References

• Owner-scope history, measurements and stop condition: L190 report (with cached-legacy-paused.xml, frame-sizes.txt and the public-encoder probe alongside).
• Compatible streamed path, its tests and the 59/59 suite: L193 report.
• Reproducer checkpoint: branch wip/hd22-L189-fail-first at cd6776fe.
• UrlProtocol documentation: README sections "Inbound RPC memory safety", "Complete frames declined before admission", "Bytecode Fetching" and "Forwarding a bytecode response"; docs/NETWORK_ARCHITECTURE.md.
• Negotiation constants: BytecodeProtocol.METADATA_KEY_BYTECODE_RESPONSE_FORMAT = "bytecodeResponseFormat", BYTECODE_RESPONSE_FORMAT_STREAMED = "streamed". Missing or unknown values mean legacy.
• Testing guidance: TESTING.md; engineering principles: PHILOSOPHY.md.

Branches and state

Nothing in this effort is deployed or published. Nothing exists only locally: every branch above is pushed.

Open decisions for the owner

1. (Decided 2026-10-27: the restriction was lifted and the streaming design chosen; see the top of this document.)
2. Whether the router-side libp2p response writer in ContainerNursery should be folded into this effort or get its own handoff (default taken: its own handoff).