Handoff: Diagnose the actual budget overruns in four kompile-core tests

Written 2026-10-09 08:18:39 UTC; branch heads refreshed after final evidence checkpoint.

RE-VERIFY: This is a write-time snapshot. Check git ls-remote https://github.com/CodexCoder21Organization/kompile-core.git refs/heads/main refs/heads/wip/sol-kcflakes refs/heads/wip/sol-kcflakes-evidence, download the original run artifacts if needed, and run the saved audit. No product fix is established.

Mission summary

The user requested: “Read the file .../briefs/kcflakes.md and carry out the task it describes.” The brief asks what single mechanism makes four unchanged-main tests fail at about 32 seconds and pass on retry, and requires red-first reproduction before any fix. It allows an evidence-backed refutation and instructs stopping rather than guessing. The four tests are testExplicitBuildFailureKeepsFailedTestInResultsAndXml, testExplicitDirectoryKilledAttemptRecovery, testResultIndexPreservesObservableWorkspacePath, and testExplicitBuildPrimesTestDependencyResultCache.

The timing-based shared-stall premise is refuted. The actual overrun causes remain unresolved; this handoff preserves that distinction for the orchestrator. No timeouts, iterations, original tests, product sources or pins were changed.

What was found and done

1. Fresh main checkout: https://github.com/CodexCoder21Organization/kompile-core/commit/3e5dd82b73b3c13cc9fe800cbc2fb52354d4fd0b. Downloaded all four raw XML artifacts and the entire console archive from https://github.com/CodexCoder21Organization/kompile-core/actions/runs/37331069611 at https://github.com/CodexCoder21Organization/kompile-core/commit/7de0729dedab4871143e138de85db581667a71f8. The four original test files are unchanged between these heads.
2. Actual attempt sequences (seconds): failure-report FAIL 32.237, FAIL 32.513, PASS 11.767; recovery FAIL 32.243, PASS 26.030; observable-path FAIL 32.510, PASS 27.541; priming FAIL 32.502, PASS 14.295. The claim that all four retry successes take 10–15 seconds is false, and one fails twice before passing.
3. Each first failure explicitly says “timed out after 30000ms.” All distinct first-attempt main snapshots are RUNNABLE. Failure-report indexes Kotlin compiler ZIPs; priming generates consumer Kotlin IR; observable-path serializes compiler metadata; recovery deletes runtime files. Recovery advances from its callback finally to its owner's finally across the captures. Coursier worker snapshots show idle queue waits. No captured main stack waits on a lease, repository read or bootstrap connection.
4. Reference source defaults the body limit to 30,000 ms, passes it to the launcher and measures the whole executeSync call. The 0.0.29 launcher source arms that limit after readiness, then collects diagnostics and terminates/drains the child. SIGQUIT harvesting alone retains a fixed one-second window. The measured failed durations therefore include work after the body cutoff and startup. This explains why different overruns produce a tight duration cluster; it does not explain why each workload exceeded its body budget.
5. The audit independently checks original exact attempt numbers/status/durations, unique histories, RUNNABLE main snapshots and the corresponding compiler/deletion frames. It passes. This is an evidence audit, not a product reproducer. Root-cause gates and mutation control have zero credit. No product PR was created.
6. Full source snapshots, original XML, complete target histories/consoles, raw reference console archive, the linked comparison parser, reading snapshots and findings are pushed. Cleanup removes only this lane's checkout/cache. No background test/watcher was started. CLI helper processes have bounded invocations and are collected before stopping.

Limits: Snapshot dumps cannot exclude an earlier transient wait or measure the full preceding timeline; they do not prove independent causes. Raw outer manager stack line numbers differ from the checkout's source because the suite is launched by a pinned published CLI. Do not silently equate those line numbers. The raw reference logs identify launcher 0.0.29 and the manager built for these tests, but full loaded-classpath attribution remains a useful check. Do not replace the disproven shared-stall inference with a claim about host capacity.

Relevant PRs / refs

Evidence directory: https://github.com/CodexCoder21Organization/kompile-core/tree/wip/sol-kcflakes-evidence/handoff-artifacts/kcflakes . The related broader PR https://github.com/CodexCoder21Organization/kompile-core/pull/385 supplied comparison provenance; it was read, not modified or shepherded by this lane. Its live status is not asserted here.

Nothing deployed, restarted, published, merged or queued. Verification: this invocation performed read-only diagnosis and evidence-branch pushes only; its diff is confined to handoff-artifacts/kcflakes. No build outputs or dependency caches are committed.

Next steps

1. Re-verify current state first. Clone fresh under workspace/ and read root instructions/README. Run python3 handoff-artifacts/kcflakes/audit_reference.py on an evidence branch to recover the exact histories without counting retry success as a first-attempt pass.
2. Have the orchestrator accept the refutation or scope an investigation of the actual workload overruns. Treat the four test paths separately until measured timelines show a common owning mechanism. A useful next experiment records public compilation/preparation events and phase boundaries on unchanged main, with samples before expiry, and checks exact loaded manager/launcher versions.
3. Identify and force the exact contract condition before editing product code. No CPU spinners, mocks, reflection, timeout increases, reduced iterations, weakened assertions, skipped tests or blind reruns. Fix at the owning layer; if upstream, preserve consumer evidence and stop for upstream scope as the brief requires.
4. For a proven fix, meet the task-specific new-test 10/10 first-attempt gate, 3/3 each original test and product-revert red control using Actions partitions. Review tests/code independently, rebase and create a draft why-first PR with the required footer/trailer, then observe its CI. Do not merge, enqueue, mark ready or deploy.

Operational knowledge

The common-flake instructions contain generic BuildTestEmbedded rules; this lane's kompile-core brief sets its actual selectors, 40-minute diagnosis stop and 85-minute hard limit. tools/gha-run.sh currently contains a BuildTestEmbedded build-rule step; inspect its applicability before using it for a future kompile-core partition rather than assuming it builds this repository. No partition was submitted in this investigation. Local rg and python commands are unavailable; use the next available search tool and python3. Coursier at ~/bin works; the Handoff CLI health returned OK. The service workflow supersedes the older git handoff-PR paragraph in the canonical README.

Read documentation snapshots and full findings are included in the evidence directory. The supplied brief itself is preserved as brief.md, including original prohibitions and exact PR provenance.

Final registration: url://handoff/handoffs/hf-2026-10-09-diagnose-the-actual-budget-overruns-in-four-kompile-core-tests . An initial API bytecode-load failure prevented the first create; after a fresh health returned OK, one further bounded create succeeded. Its raw logs and a challenge record are on the evidence branch. No claim is held.

kcflakes2 checkpoint — 2026-10-09 10:18 UTC

The cold-compilation cause is measured for three targets. First original profiles: failure/XML30.963s, observable-path27.987s, priming18.975s. Resolver park totals5.236/7.640/2.439s respectively; public consumer compile boundaries7.7–15.3s failure and6.540s priming. Compiler input hashing is visible in original full-stack JFR. Recovery is already compiler-warm: profile20.436s, two real halted children and four real consumers; five directory checks1.411s on a separate profile. Its remaining work is partly unattributed, so no speculative product edit was made. Recovery is unchanged.

Three new compiler-only prepared fixtures pass first attempts locally in8.500/6.703/7.682s. They preserve all original behavior assertions and limits, assert zero timed compilation, and route repositories to the packaged local server. Priming's random JAR basename prevented cache reuse (new warm guard failed3/3); changing only its empty fixture JAR to a stable filename fixed that, with an independent review of the contract. No successful setup results or producer outputs/logs enter the timed test.

Code: https://github.com/CodexCoder21Organization/kompile-core/tree/wip/sol-kcflakes2 at b22237dd1b42d80eddde28fb06a417b58092dd33. Evidence: https://github.com/CodexCoder21Organization/kompile-core/tree/wip/sol-kcflakes2-evidence/handoff-artifacts/kcflakes2 at6a721f9cd (latestcheckpoint). It includes original profiles, guarded red attempts, phase boundaries, recording identity map, independent review, gate watcher/audit scripts and completed raw Actions XML.

Actions raw XML audit: failure/XML10/10 first attempts4.502–5.801s; observable-path10/10 first attempts3.909–4.579s; all four selected neighbors3/3 first attempts10.518–12.439s. Runs https://github.com/CodexCoder21Organization/kompile-core/actions/runs/37914789640, https://github.com/CodexCoder21Organization/kompile-core/actions/runs/37915143941, https://github.com/CodexCoder21Organization/kompile-core/actions/runs/37914791832. Priming10 then compiler-seedomission controls for all three are still running: https://github.com/CodexCoder21Organization/kompile-core/actions/runs/37915331001. No reruns requested. Local dispatch monitors were broken by my editing their script while running; Actions continued unchanged. A new immutable, read-only, bounded watcher attaches to all existing run IDs.

Next: audit final priming/mutation XML; rebase and open a draft PR only after those gates pass; let its four-shard CI run once and audit first attempts; checkpoint all results and update this handoff before cleaning owned checkout/cache and monitors. Deadline10:53UTC. No merge, enqueue, ready-for-review or production deployment authorized. Handoff is claimed as agentkcflakes2; update and later RUNNING report succeeded, earlier reportfailure is preserved.

Draft PR checkpoint — 2026-10-09 10:29 UTC

All targeted gates are complete: 10/10 first attempts for each converted test, 3/3 for every selected neighbor, first-attempt work-guard failure for every omitted compiler seed. Priming range 5.392–5.729 s. Raw XML and audit are preserved at evidence checkpoint e814c26d3.

Draft PR https://github.com/CodexCoder21Organization/kompile-core/pull/387, code head https://github.com/CodexCoder21Organization/kompile-core/commit/b22237dd1b42d80eddde28fb06a417b58092dd33. Fetch/rebase immediately before opening found no main changes. The PR remains a draft; never mark ready, merge, enqueue or deploy on this lane.

Full-suite gate is now the single four-shard PR CI run https://github.com/CodexCoder21Organization/kompile-core/actions/runs/37917126794; related Fat-JAR build checks https://github.com/CodexCoder21Organization/kompile-core/actions/runs/37917126462. A bounded plain build-watchman with --all-checks and 240-second interval follows them; no --to-merged or rerun. Four-shard raw first-attempt audit remains pending.

A last local recovery diagnostic records whole operation windows to reduce the unattributed remainder; it is temporary and will be restored before cleanup. CI uses the committed unchanged recovery test. At deadline 10:53 UTC, retain the draft and report NOT READY if any CI/audit gate is unfinished. No final result should count retries as first-attempt passes. Owned checkout/cache and own background watchers must be removed/stopped before the lane ends.