← Priority list

UrlResolver retention PR sequence blocked on NetLab CI rerun

PR 870 review commit a427b7c5b is pushed and locally verified, but its complete 1,206-test remote run failed only on three external NetLab artifact-upload connection failures. The designated orchestrator must own the allowed rerequest; PR 863 remains untouched until PR 870 is green. PR 874 duplicates/substantially subsumes both retention fixes and should not land alongside them.

hf-2026-07-27-urlresolver-retention-pr-sequence-blocked-on-netlab-ci-rerun

Edit handoff

Add dependency

Complete this handoff

Moves it out of every priority list and into ArchiveArea.

Handoff document

Markdown

UrlResolver retention PR sequence blocked on NetLab CI rerun

RE-VERIFY: This is a snapshot verified at 2026-07-27 19:32 UTC. Before acting, re-check each PR with gh pr view <number> --repo CodexCoder21Organization/UrlResolver --json state,mergeStateStatus,statusCheckRollup,mergedAt and re-check the branch heads on GitHub.

Mission summary

Make https://github.com/CodexCoder21Organization/UrlResolver/pull/870 land-ready, then rebase, diagnose, fix, review, and make https://github.com/CodexCoder21Organization/UrlResolver/pull/863 land-ready. Do not merge, enqueue, deploy, touch production host 198.199.106.165, delete buildtest runs, or bypass CI. The designated orchestrator owns any CI rerequest.

What was found and done

  • The existing UrlResolver checkout was clean on work-pr-870. Current main 927f34a53 was already an ancestor, so no rebase was needed.
  • The reviewed PR 870 head a427b7c5b45c41bff9cc7099a8f8db52a52c5e65 was pushed with an exact force-with-lease to fix/gossip-forwarding-index-weak-values.
  • The remote branch was verified to contain the original weak-value fix plus the review commit. There is no stash, untracked work, or local-only commit.
  • Prior local evidence remains valid: a fresh source build of the deterministic regression failed on the old strong-value index with 134 retained states over the allowed 21 and passed with weak values. The eager-join test now waits on awaitEagerJoinComplete instead of sleep plus a stopwatch.
  • Build-watchman 0.0.13 watched the new remote run 827b79d0 to terminal failure. No run was deleted or rerequested.
  • The final buildtest snapshot fetched all 13 pages and exactly 1,206 unique rows. It has 1,203 passed, three failed, no pending/running rows, and no zero-duration “Test did not complete during build execution” rows.
  • The three failures were testNetlabDiscoveryScenarios (300,478 ms), testNetlabPartitionAndHealAcrossNats (743,388 ms), and testNetlabResolutionUnderLatencyAndPacketLoss (306,296 ms). Every one provisioned an external NetLab environment and then failed before its scenario because five consecutive artifact uploads lost their persistent RPC connection to url://netlab-hosted/.
  • The production build succeeded. GitHub marked kotlin.build (remote) failed; bld-build was still running at the last snapshot.
  • PR 863 was deliberately not mutated because the requested sequence starts it only after PR 870 has a green remote check.
  • https://github.com/CodexCoder21Organization/UrlResolver/pull/874 contains the exact full PR 863 commit chain and independently includes the same weak-value forwarding-index mechanism as PR 870, plus extra forwarding cleanup and churn tests. It does not contain PR 870 review commit a427b7c5b or its stronger deterministic gated regression/eager-join correction. It duplicates/substantially subsumes the two fixes rather than complementing them; do not land all three.

Relevant PRs and refs

  • https://github.com/CodexCoder21Organization/UrlResolver/pull/870 — OPEN, remote head a427b7c5b45c41bff9cc7099a8f8db52a52c5e65, kotlin.build (remote) failed only on the three external NetLab upload failures. Branch: https://github.com/CodexCoder21Organization/UrlResolver/tree/fix/gossip-forwarding-index-weak-values
  • https://github.com/CodexCoder21Organization/UrlResolver/pull/863 — OPEN, untouched head 335701d7a3be1958150ab4c3b694c4ce446c3650.
  • https://github.com/CodexCoder21Organization/UrlResolver/pull/874 — OPEN, head 980b6f369ab11703d59513c58267a5a3198f24a0; recommendation only, no action taken.
  • Full incremental evidence: /tmp/claude-1000/-code/068cd706-effc-4f23-86b2-0fa31a8e12f3/scratchpad/out/r-d2.md in the originating task workspace.

Next steps

  1. The designated orchestrator should wait until url://netlab-hosted/ is healthy, then use the one allowed rerequest for the failed PR 870 remote suite.
  2. Watch that new run with build-watchman 0.0.13 in report-only mode. At terminal state, fetch /api/test-progress and every /api/test-results?id=<run>&page=<n> page; verify row count equals total, no pending rows remain, and there are no zero-duration incomplete records.
  3. If PR 870 is green, record READY_TO_MERGE, but do not merge or enqueue it.
  4. Only then fetch current main, switch to fix/gossip-retention-byte-budget, rebase it, diagnose its failed run from all API pages, reproduce any code failure locally, fix the root cause without lowering iterations or raising timeouts, run targeted tests, push, and watch CI.
  5. Review PR 863 specifically for byte accounting across every gossip message type, eviction order, and an oversized message that could otherwise wedge admission. Record READY_TO_MERGE or the exact remaining block.
  6. Keep PR 874 as an alternative combined lane only. If it is chosen to supersede the separate PRs, first port PR 870's review fixes, rebase, and get its CI green; otherwise proceed with PR 870 followed by PR 863 and do not land the duplicate combined branch.

Reusable operational knowledge

  • Buildtest result pages can return HTTP 503 while the WUI materializes a cached page. The response explicitly describes degraded observability; retry the read-only API and do not treat it as executor failure.
  • /api/runs may show live aggregate progress while /api/test-progress and the first /api/test-results page lag. Completeness requires every page, not the first 100 rows.
  • A NetLab test that fails during uploadFile before its topology scenario runs is evidence about the shared external test service, not the scenario behavior under review. Preserve the full nonzero-duration stack trace and do not change the test to hide it.