Verify UrlResolver queue and retry changes and settle the deadline guarantee
Written 2026-10-21 08:54 UTC. RE-VERIFY: This is a point-in-time s11 lane snapshot. Check https://github.com/CodexCoder21Organization/UrlResolver/pull/1163 and its required bld-build. Source is frozen at the head below. This lane is stopping because strict P5 cannot be met together with P7; the concrete deadline choice remains unanswered. All code and tests are pushed; required full-suite CI is still running.
Design decisions in force — 2026-10-21 (verbatim brief)
- Run to completion ("Option A"): when a client goes away mid-request (disconnect, or the losing request of a direct-versus-relay race), the server frees the request slot, lets the handler code finish, and discards its result. The handler is never interrupted because the client left. A caller interrupting its own thread behaves as on main. Abandoned work stays bounded by the 30 s handler deadline (server-side elapsed time on the injected Clock), as on main.
- Over-cap behaviour, chosen 2026-10-21 — "Queue, then retry": ordinary requests beyond the worker cap WAIT IN A QUEUE as on main (so no client sees errors main never produced); only long-polls are capped (192). A long-poll rejection is marked retry-safe (a dedicated, recognisable rejection carried to the client, e.g. via the RPC exception type) and EVERY client of a parking path retries with bounded backoff: observable polls, projection polls (including child-projection paths like
<childKey>/__projection_poll), __projection_command_status, and __stream_pull (RemoteInputStream.refill).
Properties the final code must hold — each with a public-API test that fails first on the old head (assertion failure, not a compile error; stub new API first if needed)
P1. Worker threads stay bounded. Ordinary requests over the worker cap are queued (FIFO) and all eventually run; none are rejected. Test: a burst well above the cap of ordinary requests all succeed, and the concurrently running handler count never exceeds the cap.
P2. Parked long-polls never starve ordinary requests or other services (1254's test passes; ordinary requests still run while the long-poll cap is full).
P3. Long-poll classification covers every parking path, including scoped child paths (match the last path segment). Test with a membership child projection in poll mode.
P4. A long-poll rejected at the cap is retry-safe and retried by its client with bounded backoff, so the caller sees no failure: an observable poll, a projection poll, a command-status poll (the command's real outcome is reported — never a false FAILED/TRANSIENT that invites a re-run), and a stream pull (a large download completes while the long-poll cap is saturated by other parked polls). One test per client path.
P5. Every slot is released exactly once on every path (success, exception, abandonment then finish, deadline interrupt, executor throwing anything, shutdown), and before the outcome is reported to the caller. Sessions reaped on connection close (poisonIfSuspendedOnConnection) or expired (tryExpire) get the same abandoned-work flag and a fresh 30 s deadline (review finding 5).
P6. Shutdown is terminal and releases everything; requests after shutdown get a descriptive error.
P7. Docs (README, KDoc, PR description): why-first; run-to-completion stated as a deliberate change from main's TcpServer path; the queue-then-retry decision; caps are per service and judgment calls; interrupted work only frees its worker if the delegate honours the interrupt (finding 6); testEffectAwareHandlerReleasesWorkerBeforeAnsweringCaller KDoc notes it is probabilistic against a regression.
Also fix the review's nits 7 and 8 as described there.
Mission and remaining work
Finish the queue-then-retry work for the original request: add end-to-end tests exposing UrlResolver bugs under extreme conditions, show the failures, and fix them. Implementation, two independent review passes, all 28 targeted runner checks, README/KDoc, and PR description rewrite/read-back are complete. Await the real PR's full-suite bld-build, then report its result and the P5/P7 specification conflict. Main is still 4ebc725095b294944fb61068326a17dfae9a0640; fetch/rebase at 08:44 reported up-to-date, no conflicts.
Only bld-build is required (freshly read effective main rules). The runner compiled successfully and is running all tests with scripts/test.bash --local-and-remote --test . --log test_log_file.xml: https://github.com/CodexCoder21Organization/UrlResolver/actions/runs/39569633199 . The informational App's missing dispatch is not a gate and must not be rerequested. The owned local watcher was stopped at 08:55 UTC; the GitHub runner continues independently. Relaunch a plain tracked watcher when resuming. Do not merge/enqueue, deploy/publish, raise timeouts, lower iterations, disable/delete tests, use reflection/mocks, or spawn subagents. Runner verification explicitly supersedes local-before-push guidance for this run.
Specification decision still needed
P5 says every slot is released before every outcome, including deadline errors; P7 says interrupted work retains its worker until the delegate honors the interrupt. testEffectAwareDeadlineRetainsWorkerUntilDelegateStops deterministically proves main reports the 30-second error while both delegates continue despite interruption and capacity remains held. Unconditional release-before-timeout-error cannot be satisfied together with that behavior. The pushed implementation preserves main's immediate deadline error and retains real capacity until work stops. An unanswered asynchronous question asks whether to retain main's behavior (recommended) or delay the error until the delegate stops. It was asked again at 08:45 with the concrete test and options. Do not silently reinterpret the property or claim unconditional release; the findings refute that premise with evidence.
What changed and why
Ordinary requests beyond the per-service cap wait FIFO without consuming workers or execution deadlines until admission. Only parking reads reject before dispatch, through LongPollAdmissionRejectedException in the existing RPC envelope. Child projection classification uses the last path segment. Projection/status/stream clients preserve their read identity and retry with injected-Clock backoff from 50ms to 1s; observable already retries at 1s and is unchanged. STATUS retries never repeat INVOKE or report a false command failure. Reap/expiry/shutdown mark continuing work abandoned and arm a fresh 30-second deadline before waking it. Client departure releases the response wait while delegates finish, deliberately changing main's TcpServer disconnect behavior; a caller interrupting its own thread still interrupts its work.
Three late bugs were isolated and fixed after the initial 25/25 milestone. An accepted FutureTask cancelled before run could be dropped by a legitimate scheduler without releasing capacity; execution now has one claim owned either by pre-start cancellation or by the actual run, and cancellation releases before Future.get reports cancellation. The response owner dropped the caller's InterruptedException cause; both caller cancellation sites now forward their existing cancellation object, while owner close keeps its default. An executor can throw after starting work; submission cleanup now retains that work's slot and marks it abandoned under a fresh watchdog, rather than giving still-running work's capacity away. Repeated run/cancel cannot acquire release ownership twice. Separate test and implementation review passes at 08:47 found no additional issue.
Relevant PRs / refs
| Repo |
Branch |
Remote head SHA |
PR |
What is on it |
State |
| CodexCoder21Organization/UrlResolver |
https://github.com/CodexCoder21Organization/UrlResolver/tree/wip/urlfix3-handoff-2026-10-02b |
41a3b0366b60003f447dc5bd4331dd6e93712012 |
https://github.com/CodexCoder21Organization/UrlResolver/pull/1163 |
Final implementation and all tests |
OPEN; required full suite running |
| CodexCoder21Organization/UrlResolver |
https://github.com/CodexCoder21Organization/UrlResolver/tree/wip/s11-1163-queue-retry |
41a3b0366b60003f447dc5bd4331dd6e93712012 |
no PR |
Exact final implementation checkpoint |
Clean; same head as real PR |
| CodexCoder21Organization/UrlResolver |
https://github.com/CodexCoder21Organization/UrlResolver/tree/wip/s11-final-verify |
deb8b3010acad04538f93b692b79d1042231e3c8 |
no PR association asserted |
Final functional code with 28-test-only workflow |
28/28 PASS; verification only, do not merge |
| CodexCoder21Organization/UrlResolver |
https://github.com/CodexCoder21Organization/UrlResolver/tree/wip/s11-verify |
71d5fa657bc724e272f1095a199d9f88509ebcbe |
no PR |
Earlier 25-test-only workflow |
25/25 PASS; superseded |
| CodexCoder21Organization/UrlResolver |
https://github.com/CodexCoder21Organization/UrlResolver/tree/wip/s11-failfirst |
7d60eb0c7c39d2d17f67cba3d39307a1dac7dc7c |
https://github.com/CodexCoder21Organization/UrlResolver/pull/1352 |
Old implementation plus property tests/API stubs |
CLOSED; fail-first and observable compatibility evidence |
| CodexCoder21Organization/UrlResolver |
https://github.com/CodexCoder21Organization/UrlResolver/tree/wip/s11-slot-failfirst |
2d664ef8b3810c0c63a0d90c7a9ce2cdb2b4daaf |
https://github.com/CodexCoder21Organization/UrlResolver/pull/1358 |
Old ownership code plus three controlled-order tests |
CLOSED; 3/3 intended assertion failures |
No source changes are only local: s11-work is clean, no stashes, one worktree, and remote heads were verified with git ls-remote. The other interrupted checkout contains only incomplete old-main checkout/index state and no task edits. No handoff-artifacts directory is in the real PR. No service was deployed, no artifact published, and no merge/queue operation made in this lane; this is confirmed by the recorded command history/findings and source-only scope.
Evidence
- Initial compiled assertion baseline: https://github.com/CodexCoder21Organization/UrlResolver/actions/runs/39558666000 — 1 passed, 16 failed. Proves queue/FIFO, child classification, typed rejection, stream retry/backoff, and fresh reap/expiry/shutdown deadlines. The three early client fixtures had wrong resource paths; do not count them.
- Corrected client baseline: https://github.com/CodexCoder21Organization/UrlResolver/actions/runs/39561549111 — command expected SUCCEEDED but got FAILED; projection made 3 rejections and 4 subscriptions, failing continuity; queued executor Error fails queue assertion. Deadline-retention contract passes baseline, refuting unconditional P5.
- Observable compatibility: https://github.com/CodexCoder21Organization/UrlResolver/actions/runs/39564185480 — 1/1 PASS on unchanged baseline, confirmed from artifact. Earlier listener expectation was wrong: use onInvalidated, then calculateCurrentValue()==9.
- Three late ownership baselines: https://github.com/CodexCoder21Organization/UrlResolver/actions/runs/39568734062 — 3/3 intended assertion failures, not compile errors or host timing: testEffectAwareCancelledUndispatchedTaskReleasesCapacity, testEffectAwareOwnThreadInterruptReachesDelegate, testEffectAwareExecutorFailureAfterStartRetainsCapacity.
- Final targeted result: https://github.com/CodexCoder21Organization/UrlResolver/actions/runs/39569846087 — 28/28 PASS, directly counted from test-attempt-records XML at 08:46. Same final functional code as real PR; workflow only selects tests. Includes unchanged starvation companion from https://github.com/CodexCoder21Organization/UrlResolver/pull/1254.
- Earlier 22/22: https://github.com/CodexCoder21Organization/UrlResolver/actions/runs/39561728819 ; 25/25 after rebase: https://github.com/CodexCoder21Organization/UrlResolver/actions/runs/39563093554 .
- Real PR full suite: https://github.com/CodexCoder21Organization/UrlResolver/actions/runs/39569633199 — still running at 08:54 UTC, do not claim green.
Next steps
- Wait with plain build-watchman for the real required bld-build; no --to-merged. If it fails, inspect runner logs/artifact and show any claimed unrelated failure on main. One failed-job rerun is permitted only for a demonstrated unrelated flake, not repeated requeueing.
- Record and act on the P5/P7 answer if one arrives. If none arrives, report the exact conflict and unchanged deadline behavior honestly. Never mark strict P5 proven.
- Update real PR description with the full-suite outcome; its latest description already includes the late ownership fixes and 28/28 evidence and was REST-read-back checked.
- Report head SHA, per-property test/evidence/result, every existing-test edit, required-check state, and anything open. This lane stopped at the specification conflict rather than weakening P5. Resume only the outstanding decision/check work. No extra verification batches once the gate is green.
- Kill owned watcher before stop, sweep ps, preserve this handoff and remote branches, remove s11 checkouts and their exact ~/.aibuildcaches entries. The archived aggregate handoff must not be reopened or completed by this lane.
Operational notes
The aggregate was archived while this lane was active, so its report upload failed. This lane-specific handoff is the active record. report-challenge auto-enqueues/merges and is therefore prohibited by this brief; do not invoke it for the transient push rejection, loaded local box, or archived aggregate failure. A narrow fetchspec means shorthand force-with-lease can reject; use an explicit freshly observed remote SHA if a force push is needed. Initial continuation head 9f930b4 was uncompiled test drafts, not implementation. Remote/runner verification is intentional, not a local-pass claim. No subagents ran.
Cleanup verified at 08:55 UTC: both lane checkouts and their exact caches were deleted after clean-tree/stash/remote-head checks. No owned lane watcher or build/test process remains. All code/tests are on the linked remote branches.
Findings appendix (historical; snapshot above takes precedence)
S11 findings
Plan
- COMPLETED: Read brief/shared rules/reviews/continuation and linked guidance; preserved prior local work; rebased on main (latest base 4ebc725, no conflicts).
- COMPLETED: Record invariants; implement public-API property tests and collect assertion-based fail-first evidence on the evidence draft.
- COMPLETED: Implement FIFO admission, typed parking rejection, child classification, projection/status/stream retry, and fresh abandoned-session deadlines. Retain existing observable retry.
- COMPLETED: Independent test and adversarial reviews, including late ownership fixes; 28/28 targeted runner checks; README/KDoc/PR description updated and read back; no handoff-artifacts directory in head.
- COMPLETED: Baseline observable compatibility check (1/1 PASS; behavior already correct); both evidence drafts closed.
- BLOCKED: P5/P7 deadline-contract choice remains unanswered. Required full-suite bld-build on real PR head 41a3b0366b60003f447dc5bd4331dd6e93712012 (run 39569633199) is still running; no green claim. Code/evidence checkpoint complete; final handoff/report and local cleanup follow.
- OPEN SPECIFICATION QUESTION: Strict P5 release-before-timeout-error contradicts P7 and baseline behavior. Test proves this; main's immediate 30-second error is retained and worker capacity remains held until the delegate stops. No unconditional guarantee is claimed.
OBSERVED: Read p11-1163b.md, common.md, R2-1163-findings.md, L5-findings.md, and R-1163-findings.md fully. Shared rules prohibit subagents, merging, deployment, publishing, timeout increases, assertion weakening, and deletion/skipping of tests. Runner-based verification explicitly supersedes local-before-push guidance for this run.
OBSERVED 06:35 UTC: Target PR is OPEN, head 2800040b2727278553acc5568ece0332f776dd1d; branch wip/urlfix3-handoff-2026-10-02b. Effective main rules require only bld-build. Read continuation handoff at 9f930b4 fully. It contains eight uncompiled property-test drafts, not queue/retry implementation. Main already has response completion/delivery ownership from the later suspended-wait work; preserve it. Read Testing sections on hermetic tests, TDD, public API, extreme conditions, deterministic triggers, clocks, side effects, and script isolation; read Engineering Philosophy and current racing-CI procedure. The procedure requires runner build/test race commands and treats remote App checks as informational.
OBSERVED: No scheduling/cron tool is available; status cadence will be maintained in this active turn rather than creating an unrelated external automation. No subagents spawned. Full clone transferred about 595 MiB and remained indexing; stopped it and started a fresh filtered clone of the continuation branch to avoid repeating S3's full-clone cost.
Resource invariants (before implementation)
| Resource |
Created/owned/released by |
Terminal and public behavior |
Re-entry / close / restart |
| Queued ordinary request |
startHandler / handler FIFO / admission or caller cancellation or shutdown |
No worker held and no deadline before dispatch; FIFO; never capacity-rejected |
Cancel before dispatch never invokes delegate; shutdown fails queue outside locks; no persistence |
| Held worker slot |
admission / task / exactly-once release before ordinary completion signal |
Remains held through RequestEffect and abandoned running work; at most total cap and long-poll cap |
Executor failure gives slot back; callback/re-entry outside admission locks; interrupt only frees capacity when task stops |
| Session/effect exchange |
start / session / result, abandonment, expiry, shutdown |
Terminal sessions cannot resume or suspend again; late resume retains bounded reason |
Connection reap/expiry mark abandoned work and arm fresh 30 s watchdog before waking delegate |
| Response wait |
request / EffectResponseDelivery / completion or cancellation |
Preserve main's off-dispatcher completion and caller delivery ownership |
No signal completes under lifecycle/admission/session monitor; close drains accepted delivery |
| Retry rejection |
pre-dispatch long-poll admission / RPC exception envelope / client |
Dedicated type; delegate never ran; bounded backoff; cursor/offset/token/command identity unchanged |
Only STATUS retries for commands, never INVOKE; cancellation interrupts backoff |
| Shutdown |
owner / handler / terminal lifecycle flag |
Queue fails descriptively; active accepted delegates drain with execution deadline |
Repeated calls cannot admit work or restart resources; no persistent state |
Mechanism to fix: startHandler currently throws on total worker saturation instead of suspending admission, and throws an undistinguished ISE for parked requests; stream and command clients treat that rejection as an operation failure. Child projection classification compares the complete path. Session reap/expiry wakes delegates without marking and timing their remaining work. Evidence to obtain: draft public-API tests exercise each exact condition with small configured caps, real wrapped services, persistent pipe-backed RPC, and injected ManualClocks.
OBSERVED 06:49 UTC continuation: Remote s11 tip was 9f930b4. Inspected both local checkouts: s11-work has staged handoff-artifact removal and one new queue cancellation/shutdown draft; s11-1163 is an interrupted clone with no task edits. Preserved s11-work, committed 8c3f8e2 and pushed/verified s11 WIP and failfirst branches. Fetched main 1ddb52ac28 and rebase reports up-to-date (no conflicts). Target is still OPEN at 2800040b2. Evidence draft: https://github.com/CodexCoder21Organization/UrlResolver/pull/1352 . No queue/retry implementation yet. Runner verification supersedes compile-before-push per common.md. Read full S3 handoff and reviews; inherited invariant table remains applicable.
OBSERVED 06:44 UTC: Test checkpoint 3dd364d pushed to WIP and evidence branches. Added only the unused LongPollAdmissionRejectedException API stub (no admission implementation), plus testLongPollAdmissionRejectionIsRetrySafe. Updated four existing tests to the explicitly chosen queue contract, without changing their iteration/time limits: testEffectAwareHandlerRejectsRequestsBeyondWorkerLimit (4th ordinary call now awaits capacity, still verifies abandonment retains capacity and released work is not interrupted); testEffectAwareHandlerDefaultWorkerLimitRejectsExcessRequests (257th ordinary call queues, then completes after one effect resumes; still creates 256 sessions); testEffectAwareHandlerReleasesWorkerSlotsOnFailureAndShutdown (excess ordinary call queues then gets the full terminal shutdown error; error/failure/termination checks retained); testEffectAwareAbandonedHungWorkReleasesWorkerAtHandlerDeadline (ordinary probe queues while abandoned workers run, then completes at deadline; same clock/timeout/interrupt/result checks; observes all three completed tasks rather than assuming the ordinary probe finishes last). These assertions must fail on the prior rejection implementation. Further test edits will be recorded separately.
OBSERVED 06:50 UTC: First evidence runner compiled 8c3f8e2 (Build with kompile succeeded at 06:44:10); suite running, no assertion results yet. Latest test checkpoint includes typed-rejection stub and a Clock constructor stub on RemoteInputStream (unused until implementation), deterministic backoff assertions, and real executor afterExecute barriers in the new reap/expiry tests (prevents mistaking a delegate's finally latch for worker-slot reclamation). Added shutdown variant: a delegate catches the released effect exchange and keeps working; fresh deadline must interrupt it and allow owned executor termination. Added companion 1254 test verbatim so the runner suite exercises it; no companion assertions modified.
INFER / contract question raised: P5's unconditional release-before-outcome includes the handler deadline, while P7 explicitly retains capacity when the delegate ignores interruption. Main currently reports the 30-second error immediately before cancellation/worker reclamation. Those strict readings conflict for an interrupt-ignoring delegate. Asked whether to preserve main's immediate deadline error with the slot retained until delegate stops, or delay the error until that stop; no dependent behavior change will be made without an answer. Other implementation work can proceed after its fail-first evidence.
Concrete implementation plan (before edits)
- Preserve main's EffectResponseDelivery ownership. An ordinary queued request owns a response wait but no held worker, task, execution deadline, or session-table entry. FIFO entries carry a submission signal. Initial admissions retain existing immediate executor submission; releases select/reserve FIFO work atomically and hand submission to one owned lazy daemon executor, keeping custom executor calls off the shared completion worker. Submission completes its signal outside lifecycle/worker monitors and releases reservation before reporting any Throwable. Queue cancellation removes the entry; shutdown fails every queued entry descriptively and makes selected-but-not-submitted entries fail before dispatch. Accepted delegates drain.
- Only long-polls fail capacity admission. Return the existing RPC exception envelope with the dedicated rejection type; reconstruct it explicitly without adding reflection. Classify projection parking by last path segment. Keep exact original failure behavior for delegate failures and ordinary traffic.
- A shared package-level internal retry helper catches only that dedicated type, preserves each request's params/cursor/offset/command identity, schedules exponential 50/100/200/400/800/1000 ms backoff on injected Clock, and cancels its scheduled wake on interruption. Apply only to observable POLL, projection POLL, command STATUS, and stream PULL; never replay command INVOKE.
- Reap, expiry, and shutdown of suspended effects mark continuing work abandoned and arm a fresh handler-result Clock deadline before waking the delegate. Keep terminal session checks and exactly-once release. Address the signal-ready inner cancellation edge without changing the deadline contract pending the explicit P5/P7 answer.
- After assertion-based baseline evidence: implement, adapt direct wrapper tests to decode the dedicated envelope (no weaker messages/counts), review all implementations/call sites and tests independently, update README/KDoc/PR description, run full bld-build and report exact results. Do not merge/enqueue.
OBSERVED 06:57 UTC: Evidence-only commit ae0d02a on wip/s11-failfirst changes the draft bld-build test selector to the 17 property/new/edited tests plus starvation companion (all selected names are explicit in its workflow). Main/real-PR workflow is unchanged and still runs --test .. This is targeted red/green iteration, not an exclusion from the final suite. Cancelled superseded full evidence attempts 39556863307, 39557256015, 39557697097, 39558026065 so only the targeted evidence job consumes runner/buildtest resources. The first compiled successfully; none had returned a suite verdict at cancellation. No failing required real-PR check has been retried or hidden.
OBSERVED 06:58 UTC: Handoff claim succeeded for codex-s11, but uploading the exact 06:57 report failed: Handoff 'hf-2026-10-14-land-the-urlresolver-extreme-condition-bug-fixes-fix-the-two-merge-queue-flakes-then-merge-the-reviewed-prs-and-finish-the-rest' is already completed and archived; reopen it before attempting to complete it again. This aggregate is shared with other lanes, so I will not reopen or complete it on this lane's authority. Report is preserved locally at /tmp/s11-progress-report.md and its state is here. Read report-challenge skill; its CLI auto-enqueues/merges, which the brief's absolute no-merge/queue-mutation prohibition overrides, so no such invocation was made. A lane-specific continuation handoff will be used if needed at stop.
OBSERVED 07:04 UTC fail-first milestone: https://github.com/CodexCoder21Organization/UrlResolver/actions/runs/39558666000 (draft https://github.com/CodexCoder21Organization/UrlResolver/pull/1352 at ae0d02a) compiled all selected code/tests and ran 17 tests: 1 passed, 16 failed. Companion starvation test passed. Assertion failures prove ordinary queue/FIFO (100 excess requests), queued cancellation/shutdown, default 257th request, abandoned hung capacity, child classification, RPC rejection type, missing stream retry, bounded backoff, and missing fresh deadlines on reap/expiry/shutdown. Exact failure lines/full traces saved in /tmp/s11-failfirst.log and runner artifact test-attempt-records/test_log_file.xml. Command-status failed with a 5s TimeoutCancellationException rather than an assertion; observable and projection failed liveness assertions but contrary to expected old behavior (observable should already retry). Do not count those three as validated fail-first mechanisms yet. Added request-progress diagnostics and assertion wrapping for command terminal wait on baseline-only head 6d5451c (run 39559839121), also testing queued executor Error. No timeout/iteration/assertion weakened.
OBSERVED 07:12 UTC: Implemented queue admission/type envelope/child classification/stream retry and fresh reap/expiry/shutdown abandonment deadline locally after the relevant assertion evidence. Queue execution handoff avoids the shared completion worker and signals outside monitors; cancellations remove ordinary queue entries in outer cleanup before caller completion; selected-but-cancelled tasks remove any just-registered session. A request cancelled while awaiting submission gets a fresh watchdog if dispatch already started, because no normal execution wait had armed one yet. Deadline-error response ordering is unchanged pending the P5/P7 answer. P4 command/observable/projection client changes await fixture diagnosis.
OBSERVED 07:14 UTC: The P4 fixture problem is established end-to-end: baseline run 39559839121 recorded seen=[admission-retry/__projection_command_invoke]; rejected=0; work=0 and an Unknown request path exception. PersistentRpcConnection's second argument is a resourcePath; the draft passed a display label. Corrected all three fixtures to resourcePath="", retained progress diagnostics, and wrapped command await failure in an assertion. Baseline-only head 0465ef8 reruns them plus queued executor Error; the prior three failures are invalid as client-behavior evidence. No timeout was increased.
OBSERVED: Direct wrapper cap tests now decode the dedicated RPC exception envelope before their existing assertFailsWith/full-message assertions. Edits: testEffectAwareHandlerCountsDeclaredLongPollPaths, testEffectAwareHandlerCountsProjectionPollsAsLongPolls, testEffectAwareHandlerLongPollsLeaveWorkersForOtherRequests; same treatment in new membership/reap/expiry tests. Rejection must still occur, preserve full descriptive text, and not run the delegate. The separate typed-envelope test asserts the new representation/type explicitly.
OBSERVED: Documentation skill sweep found the UrlResolver project overview and plan workstream, and no HostedServiceDocumentation matches. Their existing scope does not claim this admission/deadline contract; no reflecting edit is needed there. README/KDoc are the contract home for this component. Added README back-link under the heading per the skill convention.
OBSERVED 07:29 UTC green milestone: https://github.com/CodexCoder21Organization/UrlResolver/actions/runs/39561728819 on verification-only 50e3f6a: 22/22 passed. Includes P1 FIFO/concurrency and default cap, P2 unchanged 1254 companion, P3 membership child paths, stream retry + deterministic backoff, typed rejection, P5 fresh connection-reap/session-expiry/shutdown deadlines, executor Error while admitting queued work, release-before-result, and terminal queued shutdown with accepted Unconfined delivery. A quick local racing build on e689f50 also succeeded (212939 ms; no tests), before further client edits. XML /tmp/s11-green-admission-records/test_log_file.xml.
OBSERVED 07:28 UTC valid P4 baseline: https://github.com/CodexCoder21Organization/UrlResolver/actions/runs/39561549111 on 124fb95: command-status assertion expected SUCCEEDED, actual FAILED; projection rejections=3 and subscriptions=4 (trace SUBSCRIBE/POLL repeated), then liveness assertion. Queued executor Error test also fails its initial queue assertion. The uncooperative-delegate deadline contract test PASSES unchanged baseline: both deadline errors are reported while both delegates remain running despite interrupts and capacity stays held. This is evidence that strict unconditional P5 release-before-deadline-outcome conflicts with main's behavior and P7; no change to that ordering was made. Observable receives three rejections and successful subsequent polls, but its onNewValue listener never fires without a recompute. Changed the new test to observe onInvalidated and then assert calculateCurrentValue()==9; this is the API's existing notification contract, not evidence of an observable retry bug. Existing one-second observable backoff is retained.
OBSERVED 07:36 UTC: Main moved from 1ddb52a to 4ebc725 (relay aggregation fixture change). Rebased all lane commits cleanly with no conflicts. Pushed exact WIP be10a5b using explicit force-with-lease after the narrow remote-fetch configuration caused shorthand lease checks to reject safely. Verification branch 71d5fa6 has an identical source/test tree, differing only in selecting 25 property tests in its workflow. Runner 39563093554 now verifies that rebased code; superseded pre-rebase verification 39562945533 was cancelled. No check failure was rerun unchanged.
Independent review passes (07:40 UTC)
OBSERVED — test-comprehensiveness pass: Reviewed the invariant table against the new tests and Testing.md's public-API, deterministic trigger, isolation, clock, failure injection, and scenario-per-file rules. FIFO ordering is observed at delegate entry with only one ordinary worker available, not inferred from awaiting futures in list order. Peak delegate concurrency is counted at the public handler. Queue cancellation/shutdown assert non-execution and exact terminal errors. Executor Error preserves object identity and verifies all later queued requests. Reap/expiry/shutdown use separate ManualClocks and executor afterExecute barriers, so delegate finally is not mistaken for reclamation. The backoff test verifies seven exact delays, no early request, no replay of a normal IOException, and now timer cleanup plus interruption preventing a second pull. Client fixtures use real handlers and real persistent RPC over NIO pipes; only the transport's root slash is normalized, metadata is forwarded, and diagnostics are bounded. Command asserts SUCCEEDED, exact work count 1, and final title; projection asserts same subscription and actual update; observable test corrects listener semantics while retaining three rejections and exact final value. No timeouts/iterations were raised/lowered and no tests were disabled/deleted.
OBSERVED — adversarial implementation pass: Reviewed admission selection vs cancellation/close, both initial and queued executor failures, accepted task drain, reentrant executor shutdown, ready signals vs worker reclamation, request-effect ownership, ready-signal cancellation, deadline arming before wake, all wrapper declarations, all four parking client paths, and persisted response-processing behavior. Signals are completed outside admission/session monitors. Queued entries own no worker/deadline until selection; exactly-once HeldWorkerSlot and FutureTask guards cover submission errors and late finishes. Selected cancellation removes any registered cancelled session and bounds work if submission had begun before the normal response deadline was armed. Projection/status retries preserve cursor/token/command identity, check lifecycle between waits, and never replay INVOKE. Observable already retries at one second and retains its implementation. Stream changes mutate offsets/buffers only after a successful pull. New timing uses injected Clock; existing Kotlin/JVM constructor entry points are retained by overloads. No reflection, mocks, artifact publishing, deployment, merge or queue actions added. Outstanding specification issue remains the unconditional P5 deadline-outcome wording versus P7 and observed main behavior; the 30-second error ordering is deliberately unchanged.
OBSERVED 07:46 UTC: Final real PR head is 8a1db7287a515169ef8e1f6ab3897dacd92be2e1, pushed to both WIP and target branch and verified via ls-remote. Required full suite is running; 39563844713 was superseded/cancelled after a final formatting/KDoc-only commit. PR description rewritten via REST and re-read (exact match apart from gh's trailing newline). Old peer reconnect test still equals main; handoff-artifacts directory is absent; worktree clean. Current baseline-only observable check is on 7d60eb0; evidence PR remains DRAFT and will close after its result is recorded. Target PR is OPEN; no merge/enqueue.
INFER 07:56 UTC — course-changing review finding, P5: ExecutorVisibleFutureTask releases only in run().finally or explicit rejected submission cleanup. FutureTask.cancel() on an accepted but never-started task completes its Future without invoking run(); a legitimate scheduler that discards that cancelled task leaves its held-worker slot permanently reserved. Mechanism to fix is missing pre-start cancellation ownership; reproducer is a controlled asynchronous executor that defers its second accepted Runnable, interrupts that request's own runBlocking caller, observes cancellation via public Future.get(), then discards it and sends a long-poll probe through the handler's public API. With one genuinely running delegate and cap=2, the probe must run; current 8a1db72 should return a capacity envelope instead. This also requires preventing duplicate run() calls from releasing the slot while the first run is active. No fix written before assertion evidence. Added testEffectAwareCancelledUndispatchedTaskReleasesCapacity; runner fail-first required before changing the FutureTask. The full suite remains in flight until the new head is ready; it will be superseded after the proven fix, not blindly rerun.
Additional P5 invariant before the cancellation fix: an accepted FutureTask has one execution claim, owned either by its first run() or by cancellation before any run(). The submission creates it and owns one reservation. A pre-start cancel atomically prevents all later run() calls, releases the reservation, then publishes Future cancellation. An actual run owns release until its delegate stops, even if its Future becomes cancelled earlier. A repeated run/cancel cannot claim ownership twice. Release callbacks remain private/non-blocking and schedule queued work without signalling caller code under the execution claim monitor. The public-API test drives both the cancellation barrier (Future.get()) and a later run() during cleanup without inspecting private state.
OBSERVED 08:08 UTC — the first new baseline (https://github.com/CodexCoder21Organization/UrlResolver/actions/runs/39566291003) failed earlier than the inferred slot leak: callerFailure was InterruptedException, but the accepted task's public Future.get() timed out rather than observing cancellation. The response owner replaces every caller cancellation with a new CancellationException("Resolver closed while waiting for an effect handler response.") in EffectResponseWait.cancel(), dropping the InterruptedException cause. EffectAware then misclassifies the explicit caller interrupt as client departure. This is an integration bug introduced when rebasing the Option A discriminator onto main's owned response waits. Do not count this first failure as pre-start-slot evidence. New testEffectAwareOwnThreadInterruptReachesDelegate isolates cause preservation with a started delegate and a ManualClock that never advances. The pre-start test now parks its first worker on a RequestEffect, waits for both execution deadline registrations, and advances only the result clock after cancelling the second caller; that forces Future cancellation independently and cannot free the first suspended worker.
INFER 08:20 UTC — adjacent P5 ownership path: execute() may throw after starting the submitted task. Current catch calls releaseWithoutRunning plus an unconditional outer workerSlot.release, even though the delegate still runs; this frees capacity early. New testEffectAwareExecutorFailureAfterStartRetainsCapacity uses a real ThreadPoolExecutor, forces entry before its execute error, and requires a second declared long-poll to be refused while first work remains uninterrupted. Baseline run 39568734062 now exercises all three cases on the previous implementation without either fix.
OBSERVED: The narrow evidence-branch push was rejected once as remote rejected ... (failed), with no further server detail. Remote stayed at its prior SHA; the next fast-forward push succeeded. This is recorded here; report-challenge's automatic enqueue/merge remains prohibited by the brief.
OBSERVED: Cause forwarding is now implemented locally based on the first public-API assertion and the named cancel() mechanism. It reuses each existing getCancellationException result (no extra Job-interface calls), passes it to the owned Job, and retains the old default cause for explicit resolver close. The two caller-cancellation sites and owner-close site are all uses; no fake implementation is involved. No FutureTask/admission ownership fix is written until its independent baseline assertion is obtained.
OBSERVED 08:27 UTC — valid late ownership baseline: https://github.com/CodexCoder21Organization/UrlResolver/actions/runs/39568734062 ran all three tests on the previous implementation. All three failed on the intended assertions: testEffectAwareCancelledUndispatchedTaskReleasesCapacity got the all-2-workers-held rejection instead of ready after public Future cancellation; testEffectAwareOwnThreadInterruptReachesDelegate did not interrupt its already-running delegate; testEffectAwareExecutorFailureAfterStartRetainsCapacity ran a second long-poll instead of retaining the first one's capacity after execute threw. This is deterministic controlled ordering, not host timing calibration. Full XML/stacktraces are in that run's test-attempt-records artifact and /tmp/s11-slot-baseline3/test_log_file.xml.
OBSERVED 08:29 UTC: Pushed 41a3b0366b60003f447dc5bd4331dd6e93712012 to both real PR and WIP. FutureTask has a private single execution claim: pre-start cancellation prevents any later run and releases before Future cancellation is visible; an actual run owns capacity until it stops. Duplicate run calls cannot take release ownership. Submission cleanup cancels only never-started tasks; if execute already started work, it retains the slot and marks that work abandoned under a fresh watchdog. Cleanup failures are attached to the original executor failure. Both response-owner caller-cancellation paths preserve the original cancellation cause; explicit owner close retains its existing default cause. Prior 8a1db72 full run 39564339310 was superseded/cancelled for these proven code fixes, not retried unchanged. Evidence draft https://github.com/CodexCoder21Organization/UrlResolver/pull/1358 is now CLOSED.
OBSERVED 08:38 UTC green milestone: https://github.com/CodexCoder21Organization/UrlResolver/actions/runs/39569846087 ran the prior 25 properties plus the three late ownership tests on the final functional code: 28/28 PASS. The real PR's required full suite is running at https://github.com/CodexCoder21Organization/UrlResolver/actions/runs/39569633199. Source is frozen at 41a3b03 and worktree is clean. No further selected-test batch is needed.
OBSERVED 08:45 UTC continuation: Resumed from the pushed final functional head 41a3b0366b60003f447dc5bd4331dd6e93712012; s11-work is clean. The other interrupted clone contains only incomplete index/checkout state matching its old main and no task edits. Fetched main and rebased: still 4ebc725095b294944fb61068326a17dfae9a0640, no conflicts and no changes. Target remains OPEN. bld-build run 39569633199 compiled successfully and is in full-suite tests; own tracked plain watchman session 99837 is active. Its lost-dispatch warning concerns informational kotlin-build-ci-test, so no rerequest is authorized or necessary. The observable baseline check 39564185480 PASSED; evidence draft 1352 is already CLOSED, as is late evidence draft 1358. Re-asked the P5/P7 deadline choice with a concrete description; retained main behavior pending any answer. No new source edits or verification batches.
OBSERVED 08:47 UTC — final late-change independent review: Read the three controlled-order public-API tests independently of the implementation. The deferred accepted Runnable is observed only through Future.get and a new handler request; a separate non-advancing clock test distinguishes caller interruption from deadline interruption; the after-start executor Error test asserts exact original throwable identity, retained long-poll capacity, no interrupt, and eventual reuse after executor afterExecute. Reviewed the execution-claim state transitions and their lock ordering separately: only the first run owns actual execution, pre-start cancel claims and releases before publishing Future cancellation, active cancellation cannot transfer release ownership, duplicate run cannot release it, and submission failure after start retains capacity and arms abandonment. Rechecked response cancellation forwarding at both caller sites and explicit owner-close default; no extra Job calls or fixtures need updates. No additional issue found. XML downloaded directly from runner confirms 28/28 PASS and the baseline observable test 1/1 PASS. PR description now includes late-fix failure evidence and 28/28 result and distinguishes never-started submission errors from errors after start; REST read-back matches. No source changes.
Final property and evidence audit (08:50 UTC; full-suite gate still running)
- P1 DONE: testEffectAwareQueuesOrdinaryRequestsInOrder and testEffectAwareHandlerDefaultWorkerLimitRejectsExcessRequests fail assertion-based baseline 39558666000 and pass final 28/28 run 39569846087; FIFO delegate-entry order and bounded peak, 100 excess requests, default 257th queue verified.
- P2 DONE regression coverage: testParkedObservablePollsDoNotStallOtherRequests (unchanged companion), testEffectAwareHandlerLongPollsLeaveWorkersForOtherRequests. Companion already passes the previous PR head, because its original dispatcher starvation was already fixed there; no invented new fail-first claim. Both pass final 28/28. Earlier original failure/run-to-completion evidence is recorded in L5 findings.
- P3 DONE: testEffectAwareCountsMembershipChildPoll fails baseline 39558666000 and passes final 28/28; real membership child poll mode matches last path segment. Declared/projection parking tests also pass.
- P4 DONE with an evidence-backed refutation: testCommandStatusRetriesLongPollAdmission and testProjectionPollRetriesLongPollAdmission fail corrected baseline 39561549111 and pass final 28/28; testRemoteInputStreamRetriesLongPollAdmission and testRemoteInputStreamAdmissionBackoffIsBounded fail 39558666000 and pass final 28/28. testObservablePollRetriesLongPollAdmission passes unchanged baseline 39564185480 and final 28/28: observable already retries correctly, so a new observable fix/fail-first is unnecessary. Early fixture-prefix/listener failures are excluded.
- P5 implementation DONE for real capacity ownership: testEffectAware{ConnectionReap,SessionExpiry,Shutdown}BoundsAbandonedDelegate fail baseline 39558666000 and pass final 28/28. testEffectAwareQueuedExecutorFailureDoesNotLoseCapacity fails corrected baseline 39561549111, passes final. testEffectAware{CancelledUndispatchedTaskReleasesCapacity,OwnThreadInterruptReachesDelegate,ExecutorFailureAfterStartRetainsCapacity} fail controlled-order baseline 39568734062 and pass final. Existing success/failure/release-order tests pass; original release-order failure observed locally by L5, probabilistic against old code. Strict release-before-deadline-error REFUTED as incompatible with P7 and main: testEffectAwareDeadlineRetainsWorkerUntilDelegateStops passes unchanged baseline and final, proving deadline error while real workers remain occupied. User decision still unanswered.
- P6 DONE regression coverage: testEffectAwareQueuedCancellationAndShutdown, testEffectAwareHandlerReleasesWorkerSlotsOnFailureAndShutdown, testEffectAwareShutdownDuringSubmissionPreservesUnconfinedResult, testEffectAwareShutdownBoundsAbandonedDelegate pass final 28/28. Queued shutdown/fresh deadline changes have assertion baseline 39558666000; terminal behavior already correct on some old-head paths is preserved rather than artificially made to fail.
- P7 DONE: README and handler KDoc explain Option A change from main TcpServer, per-service 256/192 judgment calls, FIFO/retry, and honest interrupt limitation. Release-order test KDoc labels regression detection probabilistic. PR description is why-first, includes all current evidence and late ownership fixes, REST read-back verified. Nits: ready-signal cancellation gets fresh abandonment watchdog; unrelated peer reconnect test equals main.
No tests were weakened/deleted/disabled; no timeout increases/iteration reductions. All existing-test edits are recorded above with before/after and reason and repeated in the PR description. The full runner suite includes every EffectAware/StreamAware/ObservableAware/live-projection/stream scenario through --test .; selected 28 names are in the verification branch workflow and XML.
OBSERVED 08:54 UTC terminal checkpoint: Required full-suite bld-build remains IN_PROGRESS on OPEN real PR at 41a3b0366b60003f447dc5bd4331dd6e93712012; build succeeded. Watchman reports 22 minutes versus about 67 minutes typical, not a stall. Stopping on the brief's explicit cannot-meet-a-property condition: strict P5 conflicts with P7 and verified main behavior, and the concrete decision question remains unanswered. No claim of complete required checks or unconditional P5 is made. All code/test work is pushed; no merges/enqueues/deployments/publications. Final handoff will retain exact state and the unanswered choice. Interrupted clone checked file-by-file against git blobs: only difference is the zero-byte file at checkout interruption; all other present files match HEAD exactly, with 1,976 not yet checked out. No task work is lost by removing it.
OBSERVED 08:55 UTC cleanup complete: verified real PR and WIP remote heads both equal 41a3b0366b60003f447dc5bd4331dd6e93712012 immediately before deletion; clean source tree and no stashes. Stopped owned watchman PID 625987; tracked process exited 130 with RESULT INTERRUPTED, and ps confirms no lane build/test/watch process remains. Removed both lane checkouts and exact lane cache. All source/tests remain on linked remote branches; full-suite GitHub runner job continues independently. Lane-specific handoff updated with the full current findings and blocked reason; no aggregate handoff mutation.