PR 616: option A applied (liveness kept), close failures surfaced; CI on new head, then peer-retention work

RE-VERIFY - written 2026-10-04 ~22:45 UTC by lane L35b. Snapshot only; recheck claims, PR head/checks and main before acting. No merge, enqueue, deploy, publication or handoff completion has been done.

Where it stands

• https://github.com/CodexCoder21Organization/UrlProtocol/pull/616 head f3a75df706691ff0a38ff5568a54e34f7be1d5bc on main 7b3f99062226f5108cce2e267ee4f58cbc0bb918: the original three commits + version (0.0.606, unpublished), then a14407aa (two fail-first close-failure tests) and f3a75df7 (fix). Checkpoint also at https://github.com/CodexCoder21Organization/UrlProtocol/tree/wip/L35c-close-failure.

• Supervisor ruling (option A): main's PR-619 attachment liveness contract stands. The receive-ownership correction and its two tests were dropped. README records the bounded under-report (at most one frame per stalled reader whose close does not unwind) as deliberate.

• A failed input.close() during a guard/watchdog trip is now reported to the reader as a suppressed exception (was swallowed). Fail-first locally 0/2 at exactly that assertion; head 12/12 locally (new tests + main's five attachment liveness tests + seven neighbours).

• testProtocolRootCloseCallerRacingEventLoopShutdown: isolated local 3/3 on main and 3/3 on the head (runs 2-3 may be replayed results).

• Remote buildtest timed out on every submission 22:00-22:12 UTC; all runs were local and memory-gated.

• Branch CI on f3a75df7: bld-all-tests 2353/2354; the only failure is testProtocolRootCloseCallerRacingEventLoopShutdown ("Iteration 25: the root is closed"), which also failed on the previous head (iteration 15) and passes on main's and other branches' CI. Treated as this PR's defect; mechanism not found (likely a test interaction in the shared CI JVM, since isolated runs pass). kotlin.build (remote) run ff8518b1 was still running.

• Root-close test mechanism (2026-10-05): every test has its own JVM, so this PR's tests cannot leak into it. After the loop terminates Netty 4.2 leaves the registered root open, and closeRootWithoutLoop's rootChannel.close is rejected by the terminated loop, so a close that loses the race to termination leaves the root open (forced: 31/300). Main pins that failed, root-left-open result in testProtocolRootCloseClaimedOnLiveLoopSettlesAfterLoopTermination (and two others), while testProtocolRootCloseCallerRacingEventLoopShutdown requires a closed root: an inconsistency in main's #605 code, not this PR. A draft fix (close via unsafe after termination) plus a public-API test is at https://github.com/CodexCoder21Organization/UrlProtocol/tree/wip/rootclose-after-loop-termination (b02cb316), never executed; it changes three tests' pinned contract and needs a decision and its own PR.

Next steps

1. Decide the root-close contract (close the root after loop termination vs keep main's failed/left-open result); land that separately on main, then re-run PR 616 CI. Watch bld-all-tests and kotlin.build (remote) on f3a75df7 to green (re-request kotlin.build at most once with the run record if it concludes FAILURE while its run is still executing).
2. Independent test-comprehensiveness and adversarial reviews on the final head, then supervisor merge decision.
3. Left for a later invocation: the UrlResolver prune-retention registry-inflation fix, then reviving https://github.com/CodexCoder21Organization/UrlProtocol/pull/617 (historical body below). Nothing done on it yet.

Historical handoff body ? preserved for the remaining peer-retention work

id: hf-2026-10-02-fix-the-four-review-findings-on-urlprotocol-pr-616-and-land-it-fix-urlresolver-prune-retention-registry-inflation-before-reviving-pr-617 url: url://handoff/handoffs/hf-2026-10-02-fix-the-four-review-findings-on-urlprotocol-pr-616-and-land-it-fix-urlresolver-prune-retention-registry-inflation-before-reviving-pr-617 title: Re-review and land UrlProtocol PR 574, PR 632, PR 634 and UrlResolver PR 1164; let PR 604 and PR 1165 merge on green; revive PR 617; then bump consumer pins summary: Lane hfUP574 checkpoint: 604 ready, 1165 ready with untouched-test exception; remaining candidates need verification created: 2026-10-02T11:04:13.698Z completed: null blocked-reason: Lane up634: main advanced during verification; rebased candidate needs replacement gates under the one-full-suite brief before PR push. dependencies:

Lane hfUP574 final checkpoint

OBSERVED: Final lane checkpoint preserves complete registry and relay candidate patches at https://github.com/CodexCoder21Organization/UrlProtocol/tree/wip/hfUP574-candidate-checkpoint . Registry source commit 05415ac9 is local and full run https://buildtest.kotlin.build/run?id=c03aeb02 remains TESTING. Its original cycle and failed-batch regression passed the corrected targeted gate. The separate cross-peer baseline at https://github.com/CodexCoder21Organization/UrlProtocol/tree/wip/hfUP574-registry-other-peer-cycle still needs to be ported and verified against the candidate. OBSERVED: Relay finalization regression selected results passed in five fresh runs (908a3aad, 33afd5be, 861ce10f, b43910dd, 016f2248), but several full manifests remain incomplete. Intent candidate run ab45c00f reports four passes and one failure; both new five-iteration stale-outcome scenarios passed, but the failed row is on an unavailable page and its name and stack have not been established. Do not treat the candidate as verified. OBSERVED: Ready for supervisor: https://github.com/CodexCoder21Organization/UrlProtocol/pull/604 (2323/2323 passed, required checks green), and https://github.com/CodexCoder21Organization/UrlResolver/pull/1165 under the lane's untouched-test exception (1898/1899 API rows passed; untouched testPruneVerificationRequiresUrlRpcProtocol timed out after 30000ms in https://buildtest.kotlin.build/run?id=ecbb1e2d ; all 13 changed scenarios passed). No PR was merged, queued, dequeued, or closed. OBSERVED: Remaining: https://github.com/CodexCoder21Organization/UrlProtocol/pull/574 has a reproducible invalid stop/start rejection fixture; no replacement contract was guessed. https://github.com/CodexCoder21Organization/UrlProtocol/pull/632 awaits full suite, cross-peer candidate verification, review and source push. https://github.com/CodexCoder21Organization/UrlProtocol/pull/634 awaits the remaining targeted failure investigation and review. https://github.com/CodexCoder21Organization/UrlResolver/pull/1164 has 22 selected client passes but incomplete API paging and historical failures not reliably reproduced. https://github.com/CodexCoder21Organization/UrlProtocol/pull/617 remains dependent on supervisor landing 632 and still needs its test/build/probe findings addressed. OBSERVED: Skipped: other-session https://github.com/CodexCoder21Organization/UrlProtocol/pull/616 and https://github.com/CodexCoder21Organization/UrlProtocol/pull/594 ; listed BuildTest*, kompile*, ContainerNursery 609 and UrlResolver 1146 ownership exclusions; all deployment, publication and pin-bump steps.

Previous handoff record

Lane hfUP574 live update ? 2026-10-04T06:11:55.107049+00:00

No merge, queue, deployment, service restart, Maven publication, or prohibited repository changes are authorized in this lane. The original snapshot's merge/publishing steps below are for the supervisor, not this worker.

Current work: all nine linked PRs were OPEN at start. Two clean PRs received one remote-check re-request each and are watched without merge mode. The TCP restart fixture failure is independently reproduced and remains unresolved. A new PeerRegistry publication/lifetime/evidence wait cycle is reliably forced through public calls. The candidate fix is local and its first targeted batch reports one failure, awaiting full result details. Resolver's 22 added prune scenarios passed together, but two historical intermittent failures remain unexplained. Relay finalization regression is permanent on a lane branch (first two fresh gates successful); the new registration-intent reproducer has no verdict yet. No PR is certified ready.

Durable lane branches:

• https://github.com/CodexCoder21Organization/UrlProtocol/tree/wip/hfUP574-registry-publication-cycle ? failing-first registry test and two current-build import corrections, commit https://github.com/CodexCoder21Organization/UrlProtocol/commit/377c20f1.
• https://github.com/CodexCoder21Organization/UrlProtocol/tree/wip/hfUP574-relay-finalization-test ? unchanged deterministic acknowledged-close regression, first gate checkpoint https://github.com/CodexCoder21Organization/UrlProtocol/commit/e2eed220.

Skipped: other-session PRs, deployment/publication steps, prohibited BuildTest*/kompile* pin changes. Draft https://github.com/CodexCoder21Organization/UrlProtocol/pull/617 still depends on the supervisor landing https://github.com/CodexCoder21Organization/UrlProtocol/pull/632.

Detailed incremental evidence is in the lane's RUNNING reports. The original snapshot follows for historical branches and provenance.

Land the UrlProtocol / UrlResolver registry-inflation and lifecycle PRs: re-review and land PR 574, PR 632, PR 1164, PR 634; let PR 604 and PR 1165 merge on green; revive PR 617; then bump consumer pins

Written 2026-10-02 21:42 UTC. RE-VERIFY before acting: every PR state, check state, and branch head below is a write-time snapshot. Authoritative re-checks: gh pr view <n> --repo CodexCoder21Organization/<repo> --json state,mergeStateStatus,headRefOid,statusCheckRollup; remote check-runs via gh api repos/<owner>/<repo>/commits/<sha>/check-runs; coordinator in-flight list at https://githubci.kotlin.build/api/builds; published versions at https://kotlin.directory/foundation/url/protocol/<version>/.

Mission summary

Original request (2026-09-24): "What's up with the PRs on https://github.com/CodexCoder21Organization/UrlProtocol/pulls - are they all red because of old issues we were having with the remote build infra? Let's try re-running the remote builds and rebase/fix the PRs to get them green if needed. Do work in parallel." Then: "Investigate the lost responses, use TDD to reproduce locally in hermetically sealed e2e tests and then fix." On 2026-10-02 the operator said: "Drive this to completion, use your best judgements, delegate heavy lifting to codex/gpt", and at 21:30 UTC: "Update the handoff and stop everything." All delegate lanes were stopped at 21:38 UTC; this handoff records exactly where each piece stands.

What remains (do these, in rough order)

1. https://github.com/CodexCoder21Organization/UrlProtocol/pull/574 (TCP serving lifecycle, branch wip/r71-tcp-serving-lifecycle, head 2b8e8c30 = content head 7db55108 + one empty re-trigger commit). Fourth fix round landed (13 failing-first regressions, README invariant table; fix comment https://github.com/CodexCoder21Organization/UrlProtocol/pull/574#issuecomment-5959543058). The fourth review (lane rv574d) was interrupted mid-way; its partial notes are at https://github.com/CodexCoder21Organization/UrlProtocol/tree/wip/rv574d-handoff-2026-10-02 (review-record/). Known PR-introduced suite failure: the branch's full Actions suite on 7db55108 fails testReview529ExecutorRejectionReturnsAdmission with "TcpServer on port 0 cannot start after its request executor has stopped; use a new TcpServer" (2285/2286 pass). The new terminal-start ownership check prevents that pre-existing fixture from reaching its deliberately terminated-executor rejection scenario. Decision needed by the next implementer: adapt that fixture to the new contract (a server whose executor has stopped is terminal; the rejection scenario must be exercised through a live server) rather than removing the terminal-start rule; then re-run the fourth review (invariant list in the brief p-rv574d.md in the session scratchpad, reproduced in "Reusable knowledge" below) and the manager's final review, then enqueue via GraphQL enqueuePullRequest (UrlProtocol has no auto-merge).
2. https://github.com/CodexCoder21Organization/UrlProtocol/pull/632 (PeerRegistry conditional removal with a total lock order, branch fix/Lexp2-conditional-evidence-removal, head 50a754f2 = content b3e1c044 + empty re-trigger). Third fix round landed; published as foundation.url:protocol:0.0.601 (proof with hashes: https://github.com/CodexCoder21Organization/UrlProtocol/pull/632#issuecomment-5959689041; fix record https://github.com/CodexCoder21Organization/UrlProtocol/pull/632#issuecomment-5959634196). Third review (lane rv632c) was interrupted while its baseline waited on the local build lock; partial notes at https://github.com/CodexCoder21Organization/UrlProtocol/tree/wip/rv632c-handoff-2026-10-02 (review/rv632c/). It had confirmed the published 0.0.601 bytes match the proof hashes. Remaining: finish the review (build the full lock-acquisition graph; hostile-caller cycle experiments for legacy re-entry under L, evidence writer under E vs active action under L, callback re-entry, restart mid-removal, equal-valued snapshots, replaced lifetime/descriptor, and the E-held entry the README says is now rejected), manager final review, enqueue.
3. https://github.com/CodexCoder21Organization/UrlResolver/pull/1164 (prompt expiry of never-advertised peers, branch fix/prompt-expiry-for-serviceless-peers, head b7831c87). Re-pinned to protocol 0.0.601 by lane C1164a (commit "Re-pin protocol to 0.0.601 for prompt peer expiry"; its seven-scenario verification batch had not produced a verdict when stopped, so run it: the scenario list is out/L632f-consumer-scenarios.txt in the session scratchpad, or the seven testPrune* scenarios the PR adds). Two of the 15 prune-policy scenarios failed once in a full batch run under heavy host load (testPruneKeepsServicelessPeerAfterIncompleteShortAgeVerification: "An unfinished shared handshake is INCOMPLETE and must retain the serviceless peer. Expected <0>, actual <1>"; testPruneKeepsActiveQueryOnlyProviderAtOneExchangeInterval after a query-stream open timed out) and have NOT reproduced in isolation (3/3 and 2/2 green under a build lock); lane F1164's diagnostic checkpoint and resume note: https://github.com/CodexCoder21Organization/UrlResolver/tree/wip/F1164-e30 (its hypothesis: the fixture accepts one socket while the original run logged two independent root closures, so a second physical dial may turn a shared INCOMPLETE observation into an exclusive negative verdict; unproven). Next: reproduce under the original condition (the full 15-scenario batch, genuine concurrent callers), never a timing tweak; then a re-review of the whole PR against 0.0.601 (previous review notes: https://github.com/CodexCoder21Organization/UrlResolver/pull/1164#issuecomment-5954651215 and the session's out/rv1164b-findings.md), manager final review, then gh pr merge 1164 --merge --auto (UrlResolver accepts auto-merge). Also: the branch build on 93614ebb failed once on the unrelated intermittent testEagerJoinHostCreationCompletesWithOccupiedCarrier ("joinNetwork did not reach the controlled pending host-start future"); lane E1164 started on it (branch fix/eager-join-host-creation-flake, partial notes https://github.com/CodexCoder21Organization/UrlResolver/tree/wip/E1164-handoff-2026-10-02; it found the CI failure is the pending-boundary assertion, not a host-start exception, and had prepared a virtual-thread stack collector). Fix it as a separate PR off main.
4. https://github.com/CodexCoder21Organization/UrlProtocol/pull/634 (relay recovery reset on acknowledged registration, branch fix/pending-relay-registration-recovery-flake, head e2eb21ef, opened 15:25 UTC 2026-10-02). This fixes the main-branch flake testPendingRelayRegistrationPreservesSameRelayFailuresUntilAcknowledged (seen red on https://github.com/CodexCoder21Organization/UrlProtocol/actions/runs/36753001122 and https://github.com/CodexCoder21Organization/UrlProtocol/actions/runs/37032383545). Lane F574flake2 built a deterministic reproducer of the underlying defect (a stream closure after the acknowledgement commits doubles the recovery delay from 5 to 10 s; fails 2/2 on main) at https://github.com/CodexCoder21Organization/UrlProtocol/commit/a7e58589034241292add1f78750766e8d67728d2 (branch fix/pending-relay-registration-recovery-flake-f574flake2, review-record/ has the resume note) and noted PR 634 records an outstanding intent-guard gap (an older monitor result must not publish into a newer registration intent). Lane F634 (partial notes https://github.com/CodexCoder21Organization/UrlProtocol/tree/wip/F634-handoff-2026-10-02) found PR 634's red remote check is infrastructure (run 98bf14af: 2241/2241 passed, ten shards released at the provisioning deadline, then a result-fetch EOF). Next: carry the reproducer onto PR 634 as a permanent test, add a failing-first test for the intent guard and fix it, gate 5/5, review, land. Check PR comments for another active claimant before pushing to that branch.
5. https://github.com/CodexCoder21Organization/UrlProtocol/pull/604 (close drains active service sends, head ca45f8b4). Reviews CLEAN; Actions green; the required kotlin.build (remote) check has failed three times for infrastructure reasons only (provisioning deadline; twice "SSH session is not connected" on all 10 shards, challenge filed at https://github.com/CodexCoder21Organization/PlanRepository/blob/main/challenges/2026-10-02-1711-kotlin-build-remote-reports-a-run-whose-10-10-shards.md); re-requested at 21:26 UTC. When it is green: enqueue via GraphQL enqueuePullRequest. If red again with no executed tests, re-request the check-run (gh api -X POST repos/CodexCoder21Organization/UrlProtocol/check-runs/<id>/rerequest).
6. https://github.com/CodexCoder21Organization/UrlResolver/pull/1165 (prune classification + relay candidate registration, head bc9b25d0). Review CLEAN; auto-merge armed since 16:28 UTC; the remote run has failed twice on provisioning deadlines (infrastructure); re-requested at 21:26 UTC. It lands by itself when the remote check goes green; if it fails on infrastructure again, re-request.
7. https://github.com/CodexCoder21Organization/UrlProtocol/pull/617 (draft, parked). Revive as the minimal change once 632 lands: its stress-test failure was a test defect (it checked observation records through a stale post-add membership list after the background trimmer ran; the contract no longer guarantees post-add membership; the test must use the public awaitCapacityTrimQuiescence). It auto-merges on PeerRegistry.kt against the 632 head but conflicts in build.kts. Also pending upstream: a typed reason for relay connect-probe failures so UrlResolver's prune classifier in PR 1165 stops matching exception message text.
8. After 1164/1165 land: publish UrlResolver, then bump pins in the buildtest coordinator and ContainerNursery. Production deploys only on the operator's explicit request.
9. Other session's PRs (not ours): https://github.com/CodexCoder21Organization/UrlProtocol/pull/616 (required checks red) and https://github.com/CodexCoder21Organization/UrlProtocol/pull/594 (green).

Relevant PRs / refs (write-time snapshot 2026-10-02 21:42 UTC)

Deployed or published but not merged: foundation.url:protocol 0.0.599, 0.0.600 and 0.0.601 are published at https://kotlin.directory/foundation/url/protocol/ from the unmerged PR 632 branch (0.0.601 = content head b3e1c044; hashes in the PR proof comment). Nothing from main consumes them yet except PR 1164's branch. No production service was deployed by this effort (verified: no CN route or Cloud Run revision was touched; all deploys remain operator-gated).

What was found and done (the chain)

• Registry inflation root cause: short-lived processes mint fresh identities and UrlResolver's prune retained departed peers because a relay refusal was classified INCOMPLETE (text match on a shared message) and relay-candidate probing re-registered the candidate and refreshed lastSeen. Fixes: PR 1165 (classification + registration), PR 1164 (prompt expiry of never-advertised peers) with upstream PR 632 (conditional removal at the registry).
• PR 632 needed three rounds: review 1 found a lock inversion and an identity-vs-equality snapshot defect; review 2 found two more cycles (L/A legacy re-entry, E/L evidence writer vs active action); round 3 was briefed with the complete invariant (one total lock order, every entry point, failing-first test per row) and went red-to-green in one pass. Lesson recorded in the status reports: brief lock/lifecycle fixes with the invariant, never the finding.
• PR 574 needed four rounds for the same reason; round 4 (invariant-first) went red-to-green in one pass but introduced the suite incompatibility in item 1.
• PR 604's remote check failed three times on infrastructure only; PR 1165's twice. The kotlin.build coordinator restarted around 18:45 UTC and lost the check dispatches for the 19:08/19:19 pushes on 574 and 632 (fixed by empty re-trigger commits at 21:27).
• Hypotheses refuted: an "admission retirement with in-flight RPC" mechanism for dashboard response loss (earlier, see memory); on PR 632 the callback/replaced-lifetime/restart rows were already correct on baseline; on PR 1164 the two policy failures do not reproduce in isolation.

Reusable / operational knowledge

• Local builds on the /code box must be serialized across lanes: concurrent cold kompile compiles blow the 20-minute build-script deadline. Use flock <scratchpad>/kompile-local.lock scripts/test.bash --local --test X per command.
• kompile replays cached verdicts for identical re-runs; N/N gates need a fresh marker comment per run.
• A kotlin.build (remote) red whose summary shows "SSH session is not connected" on every shard, "Provisioning deadline exceeded", or "expired active chunked upload session" executed no tests: re-request the check-run; never treat as a verdict. A red with "TESTS FAILED (n/m)" is a real failure to reproduce locally.
• After a coordinator restart, verify each push registered its external check-run; if only the Actions checks appear, re-trigger with an empty commit (GitHub API: create commit with same tree, PATCH the ref).
• UrlProtocol enqueue: GraphQL enqueuePullRequest; UrlResolver: gh pr merge N --merge --auto.
• Delegate lanes die silently on the provider's content filter; check the lane log tail for "flagged" and relaunch with plain wording. A lane's verdict draft can be posted by the manager if the lane dies after writing it.
• The TCP lifecycle invariant (for PR 574 reviews): one bound listener and one accept thread; every close path retains the listener until physical closure is confirmed; a close error always reaches the caller or is tossed as a notification effect even after physical closure; an accept-thread callback never waits for a cleanup owner joining that thread (publish intent, return; outer owner completes; cleanup-complete published exactly once); accept failure alone never cancels admitted work, ordinary stop cancels it, graceful drain lets it finish; every pairwise combination of stop/force/accept failure/callback/failed close/interrupt/retry completes once or stays retryable.
• The PeerRegistry lock order (PR 632): lifetime lock L ? caller evidence monitor E ? action lock A ? membership stripe; conditional removal must not be entered while holding E (rejected with a descriptive error at the public boundary).
• Session scratchpad (may be wiped): /tmp/claude-501/-code/e30ef9fd-6f49-4378-a8d2-f675f2e5ac27/scratchpad ? briefs p-.md, lane findings out/-findings.md, status reports out/report-NN.md; everything load-bearing is on the branches above.