Handoff: Restore remote upload and execute the distinct-rule startup heap pair

RE-VERIFY: 2026-10-04T14:38:44.195230+00:00. Recheck claims, main, and PR states before acting; this is a snapshot. The supervisor has accepted the distinct-key scope. Earlier scope-decision blocker is cleared. Never merge, enqueue, deploy, publish or complete this handoff in this lane.

Mission

Original request: “Prove and bound the BuildTest coordinator journal heap owner.” The supervisor now explicitly requests the paired equal 60,000-rule compacted-baseline/ordinary-tail comparison under the existing 128 MiB child heap, with every identity and payload required. If tail fails, reproduce and fix at its owner; if both pass, preserve permanent proof and open a why-first main PR. The supervisor owns final review and landing.

Chain of findings

The original constructor pair and 60,000 source-loss identities are on main via paging https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1208 and https://github.com/CodexCoder21Organization/BuildTestEmbedded/commit/068b5607bc15a53346d3ee6ad0bebd935aeba061. Fresh main is 5c923be5a57accae49b82f759c396db0104f78e2 (re-verify before further work). Earlier source/read-back PR https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1202 is also merged. The previous audit evidence is retained below.

The current source still stores whole parsed evicted rule rows in StartupEvictedTailFold.latestByKey and retains per-run key sets. This is observed retention shape, not yet measured OOM evidence. The pre-registered pair uses identical 30 nonarchived completed run sources and 60,000 unique rule identities, each with 4,096 payload bytes. Each row fits the feed record cap; combined payload exceeds 128 MiB without tuning to host speed. Control marks rules compactedBaseline=true; experiment carries them as a gapless ordinary tail after the 30-row baseline. Public service construction plus paged feed must preserve all rows, and observer-controlled source reconciliation cannot replace them during proof. HPROF diagnostic owner/array counts will be printed on OOM before failure. No production source is changed yet.

The prescribed single timeout 5400 scripts/test.bash --remote --test testJournalConstructorDistinctRuleHeapBound --log <path> client exited 1 during workspace upload. Upload session bda60d31 encountered Build service RPC 'uploadChunk' to url://buildtest/ failed with TIMEOUT: Request timed out after 30000 ms; the client's existing implementation exhausted five internal attempts. No second submission was made. Its cancellation lost the response after it might have reached the server; do not repeat the ambiguous mutation blindly. No child output, test XML or executed test count exists. Bounded build-watchman could not find bda60d31 among the API's 100 reported runs; session absence is not a baseline/tail verdict. The health/status APIs responded, so the observed blocker is workspace upload, not a claim that the whole host is down.

The new fixture and two separate root review passes are checkpointed. A tiny standalone diagnostic test compiled the HPROF reader and verified its actual owner-chain/count output from a real dump; that only validates diagnostics, not the journal mechanism. No production fix is justified until the actual paired test executes.

Relevant PRs / refs

No PR/branch should be closed or deleted based on this lane. Useful original tests and fixes are already carried into main; keep historical red/green and control evidence. An isolated new test and build fixture are pushed. No production-source change, new PR, merge, enqueue, deploy, restart or artifact publication occurred here. Production state was not queried; no production-memory or live-version claim is made. The historical body reports no deployed/unmerged code in this effort; that remains historical context, not a new deployment verification.

Next steps

1. Restore or diagnose the shared BuildTest workspace-upload path, then resume this handoff in a fresh invocation. Verify the upload/cancellation state before any cleanup. No production changes or infrastructure operations were authorized to this lane.
2. Claim and re-verify main and the refs below. Read the pushed resume.md and two review passes; keep the fixture and diagnostic work. Rebase before builds. Run the prescribed targeted paired case once with the bounded client, and watch a real dispatched run ID. A failed/canceled/unexecuted upload is no verdict.
3. If both cases complete intact under 128 MiB, preserve the permanent proof, open a why-first main PR, verify all required checks and recommend completion only after supervisor merge. If the executed tail fails, record the measured retaining owner and dump evidence before fixing it. If that owner fix is large, checkpoint the proven reproducer as the scope permits. Moving row payloads while retaining an unbounded key index does not establish a general bounded working set.
4. Any new disk owner must preserve latest-per-key ordering, tombstones, archive filtering, generation cleanup and restart behavior through public-API tests. Do not change heap/timeouts, reduce counts, weaken tests, add retries or change sibling PRs. The supervisor alone reviews, merges, enqueues, deploys and completes handoffs.

Operational knowledge

Export PATH=/code/ws/bin:$PATH in every shell. The repo runner now honors its pinned JVM coursier launcher, so no shim was needed for scripts/test.bash --help. Use remote targeted execution; no full local suite or large local jar builds on this shared host. No timeout/heap increase, reduced count, test weakening, retry, fallback or spinner is permitted. Every child/client/watcher has a bounded terminal path; stop all owned processes before final report. No deploy or artifact publication occurred in this invocation; no production-memory/version claim is made. Full experiment and invariant table: https://github.com/CodexCoder21Organization/BuildTestEmbedded/blob/wip/L14-distinct-rule-startup-heap-20261004/incident-notes/L14-distinct-rule-startup-heap/experiment.md.

Current gate and salvage

BLOCKED_INFRA. Keep this handoff open: the distinct-rule constructor bound remains unproven. No new PR exists and required CI/full-suite gates were not run. Do not call this READY_FOR_MERGE or PROPOSE_COMPLETE. Keep the current isolated fixture, streaming diagnostic helper and durable evidence; do not discard them because the transport did not execute them. Keep every historical branch as comparison/source evidence; the useful original constructor/source-loss tests and paging implementation already survive on main. No PR/branch closing is recommended. The recovery PR remains owned by another lane.

Report-challenge tool was not invoked because its automatic enqueue/merge conflicts with this lane's prohibition. Exact workspace-upload and ambiguous handoff-update observations are in the pushed incident notes; supervisor may record them through the authorized central challenge process. The earlier ambiguous handoff update was confirmed applied by read-back.