Handoff: Choose an outer pull policy, then finish the ContainerNursery resolver upgrade

Written 2026-10-23 23:42:21 UTC by lane L41. This updates the same L37 stream-upgrade effort after L39 isolation and L41 feasibility checks.

RE-VERIFY: This is a write-time snapshot. Check remote branch heads with git ls-remote; use gh pr list for the L41 source branch before assuming no PR; inspect the published resolver API and rerun the unchanged stream scenario before changing anything. L41 did not probe or change ContainerNursery production.

Design decisions in force

The 2026-10-23 L41 owner brief supersedes L37's instruction to stop without a consumer bridge adaptation:

• "Decision taken: the upstream change is a deliberate, documented stream contract (short non-EOF reads are legal)."
• "ContainerNursery's own bridge contract ("fills the requested outer pull") must therefore be met by ContainerNursery itself"
• "when bridging, keep reading from the resolver stream until the requested byte count is reached or EOF/error, without unbounded buffering (respect the existing per-pull size limit and backpressure; no extra copies beyond what is needed; correct behaviour on cancellation/close mid-fill; propagate errors, never swallow)."
• "Do NOT change UrlResolver/UrlProtocol, do NOT edit or loosen the existing stream test, do NOT raise timeouts."
• "Show the existing stream test fails before your change and passes after (first attempt from XML), both gossip guards still pass, the 162 affected-path tests, then the full suite once under the lock."
• "Open a PR whose description opens with the outage and links the analysis, UrlResolver PRs 1463 and 1431, and L39's findings; get bld-build green with build-watchman (no --to-merged). Stop at green."
• "Use only gpt-6.1-sol; no sub-agents on other models."

Common-rules.md also requires local tests, one complete suite under /tmp/claude-501/hd22-suite.lock, no merge/enqueue/deploy/publish/restart, no kompile/buildtest-CI edits, and no completing or releasing handoffs. The upstream ban and unchanged-test requirement have NOT been relaxed. User input is pending on the outer-policy scope choice.

Mission and reason

The owner asks to adapt ContainerNursery's URL bridge and adopt the published resolver admission fix after production stalled on 2026-10-23 21:02–21:47 UTC. L34 analysis attributes the retained refresh backlog to UrlResolver's unaccounted first-contact replay path. Admission fix ships in resolver0.0.1334; that release also has adaptive stream buffers. L39 findings identify resolver1333 as the first buffer-policy release and show no published admission-fix release without that change.

The remaining work is a scope choice at the outer pull boundary, followed by implementation and the original validation/PR gates. L41 has not implemented a production fix or opened a PR.

What was found and done

1. L37 aligned the consumer closure to resolver1334/protocol1435 and POM-required Netty/effects/observable versions while keeping existing libp2p snapshot38. Both builds and pinned test closures agree. Its two accepted public gossip guards fail old1328/1429 and pass new1334/1435; old0/2 first attempts, new2/2. Its affected selection has162 scenarios from161 selected files,161/162 first-attempt passes with only the unchanged outside-fill scenario failing.
2. L39 isolated the stream behavior to the resolver change, not protocol1435. The additive consume-through-EOF probe preserves every3145851 bytes, so this is short chunking rather than truncated data. The upstream stream PR deliberately performs one source read per pull.
3. L41 fresh-cloned into its lane workspace, branched from L37's4d1456b57, and rebased current main2f822c50f. Rebased source head is2e40a5a6a04c04bb36356bd98dcf9e1e72efbab6. Existing stream test is unchanged versus L37; no production source patch remains.
4. Exact-published resolver1334 javap and read-only PR source establish the outer barrier: registerServiceInternal always builds StreamAwareServiceHandler around the CN delegate. Both UrlProtocol2 and the wrapper are final, stream maps/pull method private, registration handler private final, no public wrapper/pull-policy hook. The wrapper intercepts literal __stream_pull before calling CN. It allocates min(maxBytes,max(available,initial8192)) and calls read once.
5. The bridge reports honest initial local availability0. Client demand1048576 therefore becomes an8192-byte array before bridge.read runs. A conforming inner fill loop can fill only that offered length. It cannot enlarge the outer array.
6. L41 ran unchanged test baseline:0/1 first-attempt pass; all3 automatic attempts fail expected1048576/actual8192 (1630/1732/1722ms). It then changed ONLY bridge.read(b,off,len) to a fill loop, leaving available and test untouched:0/1 first-attempt pass; all3 attempts fail identically (1833/1729/1624ms). Patch is preserved as DIAGNOSTIC ONLY text, source restored after control. This is an experiment proving insufficiency, not a proposed cleanup-complete fix.
7. L41 reran both gossip guards on restored rebased source:2/2 first-attempt passes, no attempt histories,2315/2333ms. No guard/test assertion or timeout changed.
8. A separate gpt-6.1-sol review enumerated public registration, marker/nested-wrapper/transport/client and bounded-prefetch alternatives. It agrees the conjunction cannot be met through this API. Crucial qualification: eager prebuffering can pass the fixed eager-source test; that alone is not the requested general contract. Synchronous future top-up waits for bytes beyond satisfied demand; asynchronous top-up races an immediate next pull; blocking RPC in available changes its ordinary nonblocking meaning; constant availability would misstate source readiness.
9. Deterministic distinguishing slow-source scenario for a future implementation: source provides exactlyN demanded bytes and withholds byteN+1 until that outside pull returns. Current-demand fill must return without waiting for byteN+1. Top-up before return cannot do so. A finite larger prebuffer moves this example to the next buffer boundary.
10. No affected162/full-suite run or PR/CI was started by L41 for a known-failing candidate. The user question is pending: authorize an opt-in resolver outer fill hook (recommended), or preserve upstream and allow short outside pulls. Neither scope change is assumed. No production mutation, publish, merge or enqueue occurred.

Observed and inferred

OBSERVED: Both baseline and inner-fill control fail with exact same8192 limit; both gossip guards pass on rebased closure. Exact published API has no supported policy hook. Full XML, all stacks and patch are archived.

INFERRED: An inner bridge loop is insufficient to preserve exact outside demand with correct slow-source behavior. The supported place for coalescing is at/above the actual outer pull boundary; an upstream opt-in per-stream fill hook can do that without inventing InputStream availability or reading beyond satisfied demand. This needs an owner scope change because upstream edits are forbidden. No claim of lost bytes or new upstream accidental defect is made.

Relevant PRs, branches and evidence

Remote heads were verified by git ls-remote during L41. Source branch gh pr list --state all returned[] at23:41UTC; no L41 PR exists.

Evidence index, L41 findings, independent review, baseline XML, loop control XML, gossip XML.

The affected161-file selection remains preserved on L37's evidence branch. Build outputs, compiled JARs/classes and dependency caches are deliberately excluded from evidence. No local-only source edit or stash remains.

Deployed or published but not merged

None from L41. No deploy, publish, restart, merge or enqueue commands were run. Handoff health/get/claim/update/report only record effort state; no production service-health claim is made. Do not deploy as part of completing this handoff without an explicit owner request.

Reproduce and next steps

1. Re-verify remote heads, published API and any newer owner direction. Get the scope answer first; do not change upstream or the existing test under the current brief.
2. On the L41 source branch run scripts/test.bash --local --test testUrlFacadeRpc_bridgeFillsRequestedOuterPull --log baseline.xml. For the diagnostic control apply investigations/L41-20261023/DIAGNOSTIC-inner-fill-only.patch.txt from the evidence branch, rerun same test and restore source. It is not a proposed fix.
3. If the owner authorizes a supported outer hook, first define lifecycle/ownership invariants and fail-first public facade tests for short chunk fill, earlyEOF partial thenEOF, errors mid-fill, close/cancel while blocked, slow source completing exactly current demand, and no read beyond satisfied request. Prove the unchanged test flips too. If owner instead changes outside semantics, preserve full-byte-through-EOF and cleanup assertions explicitly; do not quietly loosen tests.
4. Verify both gossip guards, unchanged stream behavior under the chosen policy, then L37's162 affected scenarios plus new tests. Run exactly one full local suite under flock /tmp/claude-501/hd22-suite.lock scripts/test.bash --local --test . --log suite.xml; report first-attempt counts and retries distinctly.
5. Rebase current main, finish independent test/code reviews, create why-first outage PR linking L34 analysis, upstream PRs1463/1431 and L39 findings. Use build-watchman for bld-build without --to-merged and stop at green. Never merge/enqueue/deploy/publish/restart under this brief.

Reusable knowledge

A consumer read loop cannot exceed the array size a mandatory wrapper already chose. Do not equate an inner read length to the client's outer request ceiling. A fixed eager payload can make prebuffering look correct; the source that withholds byteN+1 until the N-byte response returns distinguishes demand-bound fill from waiting beyond demand. Read XML first-attempt/history nodes: runner final pass banners can conceal retries.