Finish snapshot-41 targeted qualification and update the consumer PR

Written 2026-10-27 08:00 UTC by fable-W4-1027.

RE-VERIFY: This is an active-work snapshot. Re-read PR945/965 state, remote refs and exact native attempt records. Source checkpoints are not passing claims. Do not infer a verdict from a queued run or from final counts without attempt ordinals.

Mission and direction in force

The owner asked to work through the handoff queue and drive items to completion, using best judgment without blocking questions. This item owns the published snapshot-41 consumer pin and compatibility changes in https://github.com/CodexCoder21Organization/UrlProtocol/pull/945 . The duplicate https://www.handoff.wasmserver.com/handoffs/hf-2026-10-26-complete-urlprotocol-snapshot-41-suite-verification-and-update-the-open-pr is absorbed only when this PR is ready.

The former BuildTest-evidence blocker is cleared. The latest orchestrator ruling requires work on the89 main-pass/head-fail cases with --local-and-remote targeted runs; do not wait on BuildTest. The PR's own bld-build is the full-suite gate. Worker does not merge, enqueue, approve, publish, deploy, restart services or change rulesets. No timeout, iteration count, assertion or test may be weakened. Main failures are listed below and do not create new handoffs. Blanket owner best judgment resolves parked test-edit decisions when the correction is verified.

What is established

• Exact native run572c0800 ended FAILED by watchdog at02:06:54Z:2420 completed rows, all attempt1,2298 passed/122 failed;1097 selected tests never attempted. Complete evidence: https://github.com/CodexCoder21Organization/UrlProtocol/blob/a4e58d34603bb5e965675f7ff99d01f90744a29b/investigations/run572c0800-W4/README.md .
• Exact main1228066bfd53af6e7661f68e237710ab33176aa8 run466b6723 has2513 distinct attempt1 records:2472 passed/38 failed/3 canceled. All122 head failures have a main attempt:89 passed/33 failed. Main also ended without completing its suite, OUTPACED after its180-minute overall deadline while TESTING. The broad incomplete-suite class is therefore not unique to the dependency upgrade. Complete paired errors and immutable attempt IDs: https://github.com/CodexCoder21Organization/UrlProtocol/blob/wip/W4-snapshot41-evidence/investigations/snapshot41-W4/failure-classification.json . This continuation audited all122 identity matches and ordinals. Main PASS by itself is not a causal explanation of a differing timing failure.
• Circuit helper has assertion-level proof: old helper0/1 pass, expected1048576 actual32768; corrected concrete-session fallback passes; product-only removal after green again0/1 with the same watermark assertion. The real three-host circuit fixture reaches the intended non-Netty ConnectionOverStream/Yamux path. All controls remain unchanged tests.
• Six-target run on159f91df finished2/6 first-attempt passes,4 assertion failures. Circuit and existing overlapping-borrow budget case pass. Close callback sees initiatedBy=not observed; cause reentry/concurrent reads similarly see unfinished diagnostics; Mplex admission wrongly acknowledges a sibling while server pause remains active. Complete native XML/console and circuit reversal: https://github.com/CodexCoder21Organization/UrlProtocol/tree/wip/W4-snapshot41-evidence-073425/investigations/snapshot41-W4/continuation-073425 .
• Source checkpointe44c4abbd contains the terminal publication and read-guard correction plus imported public tests. It preserves main934's ownerless-effect early return. Do not wholesale overwrite that file from closed snapshot40 PR621: its fallback into arbitrary callback scope is obsolete. Two imported fixtures were corrected to attempt every resource cleanup while retaining primary failures.
• Independent read-only code and coverage review checked exact pinned Netty bytecode and refuted a proposed channelInactive ordering concern: closeFuture listeners run before channelInactive; no speculative fix was made. The read guard and terminal publication follow the existing ownership model. Actual-head final review is underway; no new green is claimed yet.
• Started a111-function targeted racing run on exacte44c4abbde42fa845d1b479586a381a8e2e4a85d. Includes every89 original main-pass failure (two names share a source file, so function selectors are used),15 imported public scenarios, circuit, ownerless scopes,200-child immutable inventory and attachment handoff neighbors. Results pending; never call this PR ready from the checkpoint alone.
• Existing CI upgrade965 is OPEN. Main already pins kompile-cli0.0.125 and requires bld-build plus bld-all-tests.965 adds a full racing suite command to bld-build without changing its60-minute limit or removing the separate test job. Its change is carried as a separate commit on this source checkpoint so the functional PR exercises that gate too. No ruleset change is needed.
• kotlin.directory metadata freshly reports snapshot-41 as latest/release; no artifact was published or production action performed by this lane.

Ownership invariants

Connection owns negotiated session and physical close; read timeout handler owns identity pause tokens on the child loop. A decoder/caller read must not bypass any active server token; final release resumes, Yamux siblings stay independent and Mplex pauses its root. The recorder owns one terminal root record, render callbacks run outside the publication monitor, close-callback readers see final classification/inventory, and child teardown after root end preserves that inventory. Returned snapshots remain immutable. Ordinary close retains its first result; admission explicitly requests retryClose after a settled failed retirement while pending attempts remain single flight. Tests exercise these through public APIs.

Remaining steps

1. Reverify live refs and the active targeted run. Read the complete native XML, including attempt histories; require every snapshot-caused case pass on first attempt. Diagnose any remaining failure locally against main as needed, never blindly retry unchanged code.
2. Keep main's newer contracts while salvaging the closed compatibility work. The existing attachment-turn-handoff test currently expects follower bodies before owner release; the snapshot40 branch has stronger pause/header/no-body-before-release/full-delivery assertions. It is included unchanged in this run for diagnosis, not silently weakened.
3. Finish source-only removal controls for newly carried terminal/read-pause changes if needed, and independent actual-head review. Update README contracts after verified changes. No second worker full-suite submission: own bld-build is the full-suite gate under latest ruling.
4. Fetch/rebase; check destination945 remains OPEN at the expected head before any update. Push qualified changes to that existing PR branch with lease protection and rewrite description around snapshot41, exact evidence, baseline33 failures and the verified test-edit reasoning. Keep CI965 independent; do not wait on CI.
5. Once ready, report PRS_READY, update title/summary to remaining review/landing only, and complete the absorbed duplicate with the destination PR link. Orchestrator owns review and landing. No deploy/publish step is required for this item.

Relevant PRs / refs

Failures also observed on exact main

Of these33,19 share the first error line,10 share failure class with variable progress, and4 differ in precise symptoms. Do not waive a new different error by test name alone. Full paired errors and attempt IDs remain in the evidence JSON linked above.

Operational notes

Use ~/bin/coursier; PATH's binary has the wrong architecture. All report/body files must be ASCII. handoff-cli update does not accept --agent; agent is passed to claim/report/heartbeat/release where supported. report --no-usage avoids aggregating unrelated shared-supervisor transcripts; the active worker/model history is recorded explicitly in report text. An initial metadata call with --agent was rejected; subsequent valid update cleared the blocker. Some metadata clients hit their outer bounds during measured shared memory reclaim; the later report succeeded. No limit was increased.

Current local paths are disposable: fresh source/code clone /code/ws/W4/workspace/snapshot41-073425, evidence worktree /code/ws/W4/workspace/snapshot41-evidence-073425, logs and target manifest under /code/ws/W4. Source and completed evidence are pushed above. Build outputs, launchers, dependency caches and compiled classes are intentionally omitted. No stashes or unpushed completed source remain at this snapshot; active test results will be pushed at completion.

The earlier record follows for provenance only; its blocked/pending claims are superseded by the observations and latest ruling above.

Run the queued snapshot-41 regressions, then finish the consumer PR

Written 2026-10-27 by fable-W4-1027. Final blocked checkpoint around 05:08 UTC.

RE-VERIFY: This is a write-time snapshot. Check the destination PR with gh pr view, all branch heads with git ls-remote, native BuildTest run/attempt records, and current main. A queued run or a final-result total is not a first-attempt passing verdict.

Design decisions in force

The owner asked on 2026-10-27: "Work through all open handoffs ... Drive them to completion" and "Regarding questions, use your best judgement, do not ask blocking questions." This item's goal is to finish the published snapshot-41 consumer pin in https://github.com/CodexCoder21Organization/UrlProtocol/pull/945 .

The orchestrator ruling requires comparing the 122 failures and watchdog result of run572c0800 against main, fixing snapshot-caused regressions test-first, and recording failures already on main in this handoff and the PR. Do not open new handoffs for those. The owner's blanket best-judgment direction covers the parked existing-test edit. Do not merge, enqueue, approve, deploy, or change production. Do not raise timeouts, reduce iterations, skip tests, or weaken assertions. No second full suite is authorized. Final worker bar is first-attempt targeted remote passes for snapshot-caused failures and the updated destination PR; the orchestrator owns CI waits and landing.

Mission and current state

Current blocker

Both required execution clients reached their unchanged 30-minute wait limits with PENDING and tests 0/0. Native attempt pages for adapted run 4fd9324b and circuit run ac64f8d3 contained zero completed rows at the last successful reads. Later native connect returned STARTING: recovering local state with elapsedMs=167243 and then 469538; it did not return newer run state. Remote execution and readable status must recover before the required circuit red/green/reversal and adapted first-attempt passes can be obtained. The earlier queue's precise cause is not established. No production intervention is authorized. Existing useful runs are preserved for inspection/attachment after recovery; do not blindly resubmit them.

All clients, watchers and delegates started here are stopped. Partial source and all evidence are committed and pushed. The qualified pin is NOT pushed into the destination PR and the PR is NOT declared ready. The claim is released with the blocked report so another worker can resume after the service recovers.

The exact-main first-attempt comparison is complete. Obtain a genuine assertion-level red/green/reversal for the newly found circuit-path regression, complete the required targeted passes, then update the destination PR using an expected-current-head check. The prepared pin and all partial adaptations are remote and recoverable. They are not verified green.

The former blocked reason has been cleared. The previous evidence dependency is done: https://www.handoff.wasmserver.com/handoffs/hf-2026-10-27-obtain-complete-suite-evidence-for-run-572c0800 . Its complete record is https://github.com/CodexCoder21Organization/UrlProtocol/blob/a4e58d34603bb5e965675f7ff99d01f90744a29b/investigations/run572c0800-W4/README.md .

What was found and done

1. Re-verified main 1228066bfd53af6e7661f68e237710ab33176aa8 still pins snapshot-25; the destination PR remains OPEN at 468c245819c91eb5505263fb4831e527939d427c. Snapshot-41 is published: 1890830 bytes, SHA256 2482ff751bd14b929b3e6b9ad6161ce7a645eaff1c1bad7d43423a731edbfe3d. The upstream source is merged at https://github.com/CodexCoder21Organization/jvm-libp2p/commit/25efb1996bd79e59e6eef28d2d44d9a0405fdd78 .
2. Run 572c0800 ended FAILED by watchdog at 02:06:54Z. All 2420 completed attempt rows are attempt 1: 2298 passed, 122 failed; 1097 selected tests have no completed attempt row. This is not complete-suite success evidence. It is not 1219 observed assertion failures.
3. Recovered ALL completed attempts for exact-main run https://buildtest.kotlin.build/run?id=466b6723 . Native gitContext confirms main 1228066bfd53af6e7661f68e237710ab33176aa8. There are 2513 unique attempt/test identities, all ordinal 1: 2472 PASSED, 38 FAILED, 3 CANCELED. Every original head failure has a main first attempt: 33 FAILED, 89 PASSED, none missing. Complete paired errors and an executable audit are on the evidence branch. Of the 33 overlaps, 19 have the same first error line, 10 the same failure class with variable progress/count/stack, and 4 different precise symptoms. Those four remain mechanism-unassigned; one main pass alone does not prove causality for the 89 differing timing/transport cases. Main itself is OUTPACED after its CI worker's 180-minute overall deadline while TESTING: https://github.com/CodexCoder21Organization/UrlProtocol/runs/121846725335 . The broad incomplete-suite/deadline class therefore also occurs without snapshot-41; its exact infrastructure cause is not known. The documented reader CLI 0.0.126 recovered all rows after earlier 0.0.125 reads hit their unchanged response deadline; no limit was increased.
4. Submitted narrow main/head comparison runs, then canceled ONLY those two redundant queued runs once the historical exact commits supplied all requested identities. Initial adapted 124-target run 4fd9324b stayed PENDING through the client's existing 30-minute wait, which exited 2 with tests 0/0; native attempt count is zero. Circuit red run ac64f8d3 also remained PENDING with zero attempts at the last bounded read. These are scheduling observations, not red/green code verdicts. No full-suite selector was submitted. The native testsTotal is archive inventory; excluded testSpecs rows are separately FILTERED_OUT, as confirmed in BuildTestEmbeddedService source.
5. Source-confirmed consumer adaptations are checkpointed in separate commits: actual Yamux detection despite the stream-session wrapper; explicit retryClose for retirement after a settled close failure; skipping new NettyChannelCloseKt wrapper frames when finding the application close caller. The original failing public tests in572c0800 exercise these paths. No passing result is claimed for the adaptations.
6. Separate fixture commits preserve exact expected muxer, assertions and timing while inspecting actual root pipeline handlers, update two runtime jar guards to41, model retryClose in the one admission decorator, and use raw public Channel.close when a fixture deliberately needs independent physical close attempts. The retirement-failure diagnostic fixture explicitly retries its injected failed close during cleanup.
7. Independent test and code review found one important gap in the first Yamux helper: genuine RelayTransport.ConnectionOverStream is not P2PChannelOverNetty, though its session is concrete Yamux. The helper would lose prior bounded sender behavior. A standalone three real hosts circuit test is committed BEFORE changing that helper. It must fail at the 1 MiB / 2 MiB sender watermark assertions, not during compilation or setup. The proposed correction is to preserve concrete Yamux Session handling and also inspect the physical pipeline for wrapped direct connections. This correction is not applied yet.
8. Removed the destination PR's separate gossip fixture approval hold under the owner's ruling and recorded why: padding only the selected message above 256 KiB forces the documented real-refusal path. All assertions/timing remain; original red held-write, padding reversal and warning-order product-mutation controls remain in its description. The destination product head is unchanged. The old 46 second upload failure remains an infrastructure result: zero uploaded sessions/chunks and No space left on device.

Relevant PRs and remote branches

No artifact was published and no production deployment, upload or restart was performed. No PR was closed, merged, approved or enqueued by this worker.

An existing compatibility PR remains valuable and was inspected read-only: https://github.com/CodexCoder21Organization/UrlProtocol/pull/621 at b8acfe5bff4c18c8c8eccfb2be78ec84d7f84cc0. Its linked handoff had no active claim. Five separate fixture commits were salvaged here: three strong registered-root callback expectations and two original held-promise cleanup corrections. No assertion or limit was relaxed. A final independent review found a fixture cleanup error path that could skip host shutdown; commit 035c82c0d puts every release/barrier and host stop in one failure accumulator. Re-review confirms the fixture finding is resolved. This was private test cleanup; no test of private fixture internals or production API was added.

The broader pause/terminal-diagnostic source work in that PR is NOT integrated here. Reuse its final reviewed form, including the cause-message re-entry correction, rather than recreating an earlier partial version. Relevant source: PauseAwareReadTimeoutHandler.read, TerminalRootRecord, ROOT_CLOSE_PUBLISHER, and preserving final root child inventory. Its old snapshot-40 test claims are not relabeled as verification of this snapshot-41 branch.

Targeted run records

Next steps

1. Re-verify PR/branch/native run state. Inspect the existing queued runs 4fd9324b and ac64f8d3 before resubmitting. The complete historical main/head comparison is done; do not repeat it or the full suite.
2. Obtain assertion-level red for the real circuit test, preserve concrete Yamux Session support alongside direct Netty pipeline lookup, run the same test green and perform its product-only reversal. A compile/setup failure is not the required reproducer.
3. Finish the remaining snapshot-related compatibility changes test-first, reusing valid work from https://github.com/CodexCoder21Organization/UrlProtocol/pull/621 . Qualify against snapshot-41 and current main. Diagnose any remaining differing failures; do not guess from a timing result. Preserve all existing limits and strong assertions.
4. Use the complete paired errors to keep main failures distinct from new symptoms. Do not open other handoffs for baseline failures. Four overlapping test names have different precise errors, so do not automatically waive a new failure by name alone.
5. Once all snapshot-caused failures pass first-attempt on targeted remote runs, rebase, check the destination PR is still OPEN and has the expected head, and update it with the qualified pin and accurate evidence. If it has merged, use a new consumer PR. The orchestrator owns CI waits and landing. No publication or deploy is needed here.

Baseline failing identities

The complete native evidence, full paired errors and audit are at https://github.com/CodexCoder21Organization/UrlProtocol/blob/wip/W4-snapshot41-evidence/investigations/snapshot41-W4/failure-classification.md . All original 122 identities have main first attempts. These 33 also failed on main:

Operational notes

Cold-resume evidence and native Java read helper sources: https://github.com/CodexCoder21Organization/UrlProtocol/blob/wip/W4-snapshot41-evidence/investigations/snapshot41-W4/README.md . Use ASCII handoff/report files. Use ~/bin/coursier; PATH's binary has the wrong architecture. The authenticated gh pr edit command fails on a deprecated projectCards GraphQL lookup; GitHub REST PATCH with an exact JSON body file successfully edits the description instead. The report-challenge CLI auto-merges, so it cannot be run under this item's no-merge rule. All execution clients are tracked with 2700 second outer deadlines; no long CI watcher remains.

Build jars, compiled helper classes and caches are omitted intentionally; Java helper sources are published. Broad unrelated run-list samples and repetitive connection-close console output were not copied. Complete attempt errors, structured client results, zero-attempt pages and the recovery-health stack needed to resume are preserved on the evidence branch.