← Priority list

Land the gossip forwarded-discovery eviction fix (resolver 0.0.640) on main + restore skipped tests/reviews

resolver 0.0.640 evicts departed peers' forwarded-discovery rows (fixing the unbounded map that GC-thrashed every url:// node and took buildtest down). Live on prod but PRs unmerged (BuildTestWui #132, ContainerNursery #528) and fail-first tests + comprehensiveness/adversarial/final reviews were all skipped for incident speed �� this handoff requires completing them and driving to merge.

hf-2026-07-19-land-the-gossip-forwarded-discovery-eviction-fix-resolver-0-0-640-on-main-restore-skipped-tests-reviews

Edit handoff

Title Summary Markdown body

# Land the gossip forwarded-discovery eviction fix (resolver 0.0.640) on main
**Written:** 2026-07-19 ~09:5xZ (Claude supervisor session). RE-VERIFY: all state is a write-time snapshot — re-check PR merge status via gh and live prod before acting.

## Mission
The url:// resolver's `forwardedDiscoveredServices` map grew without bound — one row per
(destinationPeer × service × providerPeer), and rows for departed peers were never evicted — so every node
accumulated 50k-80k rows and GC-thrashed, taking buildtest.kotlin.build and other url:// routes down on 2026-07-19.
The fix (resolver **0.0.640**) evicts a peer's rows on disconnect/prune, hard-caps the map (50k), and adds a
re-announce backoff. It is **already DEPLOYED to prod** (WUI, ContainerNursery, backend) and verified (live counts
dropped from ~50-80k to low hundreds). Remaining work: **land the PRs on main and roll to the remaining consumers.**

## PRs (deployed to prod, NOT yet merged)
- https://github.com/CodexCoder21Organization/BuildTestWui/pull/132 (bump WUI resolver -> 0.0.640) — OPEN, mergeable
- https://github.com/CodexCoder21Organization/ContainerNursery/pull/528 (bump CN resolver -> 0.0.640) — OPEN, CI BLOCKED
- A UrlResolver release PR for 0.0.640 may also be needed (verify; 0.0.641 lock fix is https://github.com/CodexCoder21Organization/UrlResolver/pull/803).

## Next steps
1. Shepherd #132 and #528 to green CI and merge; reconcile prod jars vs main.
2. Roll resolver 0.0.640+ to the OTHER consumers still on the old resolver (droplet-service-server,
   netlab-manager-server, webcron-url-server) so their maps stay bounded too.
3. Related deeper follow-up (already filed): the O(N²) redesign handoff
   `hf-2026-07-19-eliminate-o-n-bookkeeping-...` — the eviction fix bounds the leak but the live table is
   still O(N²); that handoff removes the N² shape entirely.

## Restore the bypassed engineering rigor (REQUIRED — this handoff is NOT done until all of this is)
This fix was built and DEPLOYED TO PROD under incident pressure with the normal gates SKIPPED for speed:
**no fail-first reproducer test, no test-comprehensiveness review, no adversarial code review, no final review.**
The change is live on prod but NOT on main. Before closing:
1. **Fail-first reproducer test(s):** add end-to-end test(s) that FAIL against the pre-fix code and PASS with the
   fix (targeting the exact trigger described above). Real instances, no mocks (per the Testing Architecture doc);
   high iteration counts where the bug is probabilistic/concurrent.
2. **Test-comprehensiveness review** — shell out to codex gpt-5.6-sol (high reasoning): are the tests covering the
   edge cases / failure modes? Address gaps.
3. **Adversarial code review** — codex gpt-5.6-sol (high): hunt for regressions/bugs; confirm quality vs the global
   CLAUDE.md + PHILOSOPHY. Address findings.
4. **Final review (Claude):** review the full diff + tests + PR against the problem, CLAUDE.md, PHILOSOPHY; route any
   findings back to codex rather than fixing in-place.
5. **Drive to MERGE:** shepherd the PR(s) to green CI and get them merged to main, then reconcile prod jars vs main
   (the prod-vs-main divergence must be closed). NOT done until the PRs are MERGED and prod == main.

Add dependency

Complete this handoff

Moves it out of every priority list and into ArchiveArea.

Handoff document

Markdown

Land the gossip forwarded-discovery eviction fix (resolver 0.0.640) on main

Written: 2026-07-19 ~09:5xZ (Claude supervisor session). RE-VERIFY: all state is a write-time snapshot — re-check PR merge status via gh and live prod before acting.

Mission

The url:// resolver's forwardedDiscoveredServices map grew without bound — one row per (destinationPeer × service × providerPeer), and rows for departed peers were never evicted — so every node accumulated 50k-80k rows and GC-thrashed, taking buildtest.kotlin.build and other url:// routes down on 2026-07-19. The fix (resolver 0.0.640) evicts a peer's rows on disconnect/prune, hard-caps the map (50k), and adds a re-announce backoff. It is already DEPLOYED to prod (WUI, ContainerNursery, backend) and verified (live counts dropped from ~50-80k to low hundreds). Remaining work: land the PRs on main and roll to the remaining consumers.

PRs (deployed to prod, NOT yet merged)

https://github.com/CodexCoder21Organization/BuildTestWui/pull/132 (bump WUI resolver -> 0.0.640) — OPEN, mergeable
https://github.com/CodexCoder21Organization/ContainerNursery/pull/528 (bump CN resolver -> 0.0.640) — OPEN, CI BLOCKED
A UrlResolver release PR for 0.0.640 may also be needed (verify; 0.0.641 lock fix is https://github.com/CodexCoder21Organization/UrlResolver/pull/803).

Next steps

Shepherd #132 and #528 to green CI and merge; reconcile prod jars vs main.
Roll resolver 0.0.640+ to the OTHER consumers still on the old resolver (droplet-service-server, netlab-manager-server, webcron-url-server) so their maps stay bounded too.
Related deeper follow-up (already filed): the O(N²) redesign handoff hf-2026-07-19-eliminate-o-n-bookkeeping-... — the eviction fix bounds the leak but the live table is still O(N²); that handoff removes the N² shape entirely.

Restore the bypassed engineering rigor (REQUIRED — this handoff is NOT done until all of this is)

This fix was built and DEPLOYED TO PROD under incident pressure with the normal gates SKIPPED for speed: no fail-first reproducer test, no test-comprehensiveness review, no adversarial code review, no final review. The change is live on prod but NOT on main. Before closing:

Fail-first reproducer test(s): add end-to-end test(s) that FAIL against the pre-fix code and PASS with the fix (targeting the exact trigger described above). Real instances, no mocks (per the Testing Architecture doc); high iteration counts where the bug is probabilistic/concurrent.
Test-comprehensiveness review — shell out to codex gpt-5.6-sol (high reasoning): are the tests covering the edge cases / failure modes? Address gaps.
Adversarial code review — codex gpt-5.6-sol (high): hunt for regressions/bugs; confirm quality vs the global CLAUDE.md + PHILOSOPHY. Address findings.
Final review (Claude): review the full diff + tests + PR against the problem, CLAUDE.md, PHILOSOPHY; route any findings back to codex rather than fixing in-place.
Drive to MERGE: shepherd the PR(s) to green CI and get them merged to main, then reconcile prod jars vs main (the prod-vs-main divergence must be closed). NOT done until the PRs are MERGED and prod == main.