Handoff: TransitiveTestDependencyPruner tool + fleet prune-and-merge of the kompile build-system repos (paused mid-campaign, blocked on droplet saturation)

Written: 2026-07-06 (workstream began 2026-06-29). Authoring date differs from the effort date.

⚠️ RE-VERIFY BEFORE ACTING

Every state claim below is a write-time snapshot and was true only when written. PRs merge, merge queues shuffle, CI re-runs, branches get force-pushed, and (critically here) 6 days elapsed between the campaign pausing and this handoff being written, and the shared buildtest droplet's saturation comes and goes hour to hour. Re-verify everything before acting.

Authoritative checks:

• PR state: gh pr view <n> --repo CodexCoder21Organization/<repo> --json state,mergeStateStatus,statusCheckRollup,mergedAt,headRefOid
• Whether a repo already has a prune PR: gh pr list --repo CodexCoder21Organization/<repo> --head prune-redundant-withartifact-test-deps --state all --json number,state
• Merge-queue membership: gh api graphql -f query='{repository(owner:"CodexCoder21Organization",name:"<repo>"){pullRequest(number:<n>){isInMergeQueue mergeQueueEntry{state position}}}}'
• A fresh query always wins over this document.

---

1. Mission

The user's requests, in order:

1. "create a new CLI tool/module/project which will automatically clean up the test files by removing any dependencies which are already transitively imported … Test it by running this script on a clean checkout of ContainerNursery … create a PR for ContainerNursery … ensure it goes green in CI."
2. Rebase the ContainerNursery PR and get it merged; rename the tool repo WithArtifactPruner → TransitiveTestDependencyPruner; cross-link it from DependabotCli.
3. "Let's run the pruner on UrlProtocol, UrlResolver, and all the kompile projects. For each one, ensure the repository goes green and then ensure the PR gets merged." — later clarified to "just the repositories associated with the kompile build system (eg. kompile-cli, and upstream dependencies)", at 5 parallel workers.
4. Create/merge a PR for the pruner correctness bug fix (see §4).
5. This handoff.

The tool and its first application (ContainerNursery) are done and merged. The build-system fleet campaign is paused mid-flight, blocked on infrastructure (see §4.6).

2. The tool (built, working, merged)

Repo: https://github.com/CodexCoder21Organization/TransitiveTestDependencyPruner (renamed from WithArtifactPruner; local checkout still at /code/workspace/WithArtifactPruner). Maven coord community.kotlin.withartifactpruner:withartifact-pruner, main class community.kotlin.withartifactpruner.MainKt. Initial PR #1 MERGED.

What it does: removes a @file:WithArtifact("g:a:v") line from a kompile tests/*.kts file only when removing it leaves the fully-resolved classpath unchanged, computed by greedy differential resolution via the embedded Coursier interface (io.get-coursier:interface:1.0.19, repos https://kotlin.directory/ + Maven Central). Only edits files under tests/. Dry-run by default; --apply writes.

Run: java -jar /code/workspace/WithArtifactPruner/jars/withartifact-pruner.jar [--apply] [--verbose] [REPO_DIR].

3. First application (done, merged)

| PR | State (write-time) | |---|---| | https://github.com/CodexCoder21Organization/ContainerNursery/pull/472 — pruned 15,435 decls / 250 files | MERGED | | https://github.com/CodexCoder21Organization/DependabotCli/pull/17 — reciprocal cross-link | MERGED | | https://github.com/CodexCoder21Organization/TransitiveTestDependencyPruner/pull/1 — the tool | MERGED |

Notes: ContainerNursery's version-locked libp2p regression guard stressTestServerSurvivesMidNegotiationChurnDropWithConstrainedHeap.kts was deliberately excluded from pruning (it self-checks the loaded libp2p jar version at runtime — its pins are load-bearing).

4. The build-system campaign — what was found and done (the chain)

4.1 Scope

"kompile build system + upstream dependencies" was resolved precisely as kompile-cli's first-party transitive closure (via coursier resolve kompile.cli:kompile-cli:0.0.77 — note coursier prints to stderr) plus the explicitly-named UrlProtocol/UrlResolver = 37 repos (full list in artifacts/.../workset_final.txt). A read-only dry-run filter found 35 have redundant declarations; community.kotlin.logging and buildkts-generator-api are already clean (no-op, skip).

4.2 THE PRUNER CORRECTNESS BUG (root-caused, fixed — fix PR still OPEN, see §5)

On kompile-cli and community.kotlin.psi.leakproof the pruner over-removed load-bearing declarations, breaking their test builds (ClassNotFoundException: gnu.trove.THashMap, unresolved reference: test).

Root cause (verified via the produced .pom + coursier, and confirmed across CI runs): kompile's buildSimpleKotlinMavenArtifact* publishes a POM whose dependencies each carry a wildcard <exclusion>*:*</exclusion>. So depending on a pkg.buildMaven() artifact provides each direct compileDependency JAR but not its transitive closure. The pruner had modelled build-rule deps as fully transitive, so it deemed a declaration redundant whenever its coordinate was merely transitively reachable from build.kts — and removed declarations that were the only source of a needed transitive (e.g. kotlin-compiler-embeddable's trove4j, or a transitively-pulled kotlin-test). ContainerNursery was unaffected only because its build.kts lists the full closure directly.

Fix (in PR #2): model a build rule's compileDependencies as non-transitive resolution roots (ruleDependencies.mapNotNull { it.toResolutionRoot()?.copy(transitive = false) } in src/Pruner.kt), plus a belt-and-suspenders never-prune kotlin-test family (NEVER_REMOVE_COORDINATES). Two regression tests added (testBuildRuleDependenciesAreNonTransitive, testKotlinTestIsNeverRemoved) and the existing testRedundantTransitiveDependencyIsRemoved updated (it had assumed transitive build-rule deps). Full pruner suite 21/21 green locally against a fresh build.

4.3 Fixed jar is LOCAL only

The fix lives in PR #2 branch + the locally-rebuilt jar at /code/workspace/WithArtifactPruner/jars/withartifact-pruner.jar (12.4 MB, built with ./scripts/build.bash community.kotlin.withartifactpruner.buildFatJar jars/withartifact-pruner.jar). The pruner repo's main does NOT yet contain the fix (PR #2 unmerged) — anyone rebuilding the jar from main gets the buggy version. Merge PR #2 first, or keep using the local jar.

4.4 Merged so far (5)

kompile-interfaces-internal, community.kotlin.psi.leakproof, kompile-effects, CacheKeyBuilder, SourceCodeManager (see table §5).

4.5 Orchestration that worked (and what didn't)

• Autonomous workers waiting on CI stall/flood — fully-autonomous subagents (clone→prune→PR→wait-for-CI→merge) either backgrounded their CI poll and came to rest un-resumable, or spun up self-re-arming Monitors that flooded. Do NOT have a subagent wait on long CI. Working split: subagents do PR creation only (fast, clean, no waiting); a central bash auto-driver enqueues greens / re-triggers flakes / reports merges.
• The campaign engine artifacts/.../campaign.sh automates the whole thing: pool of 5, clone→prune(fixed jar)→safety-check(only tests/)→PR→enqueue-when-green→re-trigger-on-fail(≤4)→merge→pull-next, driven by /tmp/campaign_queue.txt + /tmp/campaign_active.txt + /tmp/campaign_done.txt. It was stopped (see §4.6); re-launch instructions in §6.

4.6 THE BLOCKER: shared buildtest droplet saturation jams the merge queue

The required status check is kotlin.build (remote) (org-standard; runs the full suite on the shared buildtest DigitalOcean droplet pool). Under saturation (a concurrent fleet operation + this campaign's own load) that check fails/flaps — provisioning watchdog timeouts, SocketTimeoutException, remote-build e2e tests failing. The merge queue (SQUASH, grouping ALLGREEN) re-runs the check on the merged result, so green PRs cycle (enqueue → flake → dequeue → re-enqueue) without merging (observed: SourceCodeManager sat green-in-queue ~45 min, then eventually merged during a lull). The 5-wide engine was adding to the saturation, so it was stopped. This is an infra/capacity limit, not a code issue — do not bypass the check (--admin/--no-verify are forbidden).

5. Relevant PRs / refs (WRITE-TIME state — RE-VERIFY)

Pruner correctness fix — MERGE THIS FIRST: | PR | Head | State | Note | |---|---|---|---| | https://github.com/CodexCoder21Organization/TransitiveTestDependencyPruner/pull/2 | fix-nontransitive-buildrule-deps @ 975a4a159 | OPEN / DIRTY | Needs rebase on main; 21/21 pruner tests green locally. Merges the §4.2 fix. |

Merged (5):

• https://github.com/CodexCoder21Organization/kompile-interfaces-internal/pull/21
• https://github.com/CodexCoder21Organization/community.kotlin.psi.leakproof/pull/18
• https://github.com/CodexCoder21Organization/kompile-effects/pull/27
• https://github.com/CodexCoder21Organization/CacheKeyBuilder/pull/29
• https://github.com/CodexCoder21Organization/SourceCodeManager/pull/23

Open prune PRs (pruned with the FIXED jar; stale after 6 days — re-verify/rebase): | Repo | PR | Head | State | |---|---|---|---| | kompile-cli | https://github.com/CodexCoder21Organization/kompile-cli/pull/124 | e682704b3 | OPEN / UNKNOWN — was BLOCKED on flaky remote-build e2e tests (cliRemoteBuildRunsTestsEndToEnd), not on pruning | | build-kotlin-jvm | https://github.com/CodexCoder21Organization/build-kotlin-jvm/pull/56 | a7e6a27be | OPEN / BLOCKED | | kompile-core | https://github.com/CodexCoder21Organization/kompile-core/pull/203 | 64b61efc3 | OPEN / UNKNOWN | | kompile-buildscript | https://github.com/CodexCoder21Organization/kompile-buildscript/pull/62 | 40800a361 | OPEN / BLOCKED | | kompile-executionenvironment | https://github.com/CodexCoder21Organization/kompile-executionenvironment/pull/58 | d2f84e8e0 | OPEN / BLOCKED |

All prune branches are named prune-redundant-withartifact-test-deps.

Not yet PR'd (24): in artifacts/.../campaign_queue.txt — RpcProtocolApi, RpcProtocolImplementation, community.kotlin.kompile.testrunner.jvm, community.kotlin.kompile.testrunner.bootstrap, community.kotlin.coursier.bldbinary, kompile-runtime, tools.kotlin.build.compile.interfaces.executionenvironment, tools.kotlin.build.compile.interfaces.diagnostics, kotlinc-diagnostic-collector, kompile.executionenvironment.bridge.interface, file-utils, HashToFileCache, FileMetadataCache, community.kotlin.test.results.summary, Refcounted, ResourceArena, util.classloader, util.stacktrace, util.cache.cachekey.interfaces, community.kotlin.clocks.hierarchical, community.kotlin.clocks.simple, kompile-buildrule-enhancer, kompile-buildcache-interfaces, UrlProtocol, UrlResolver. (No-ops to skip: community.kotlin.logging, buildkts-generator-api.)

6. Next steps (re-verify current state first)

0. Re-verify all of §5 with fresh gh queries — 6 days passed; several open PRs may have merged or gone stale, and main has advanced under all of them.
1. Merge the pruner fix PR https://github.com/CodexCoder21Organization/TransitiveTestDependencyPruner/pull/2 first (it's DIRTY → cd /code/workspace/WithArtifactPruner && git fetch origin main && git rebase origin/main on branch fix-nontransitive-buildrule-deps, resolve, git push --force-with-lease, re-run CI green, enqueue via GraphQL). This lands the correctness fix so main/the published jar is correct. Then rebuild the jar from main and use that going forward (or keep the local jar).
2. For each of the 5 open prune PRs: rebase on the repo's default branch, re-run the pruner with the fixed jar on the fresh tree (git reset --hard origin/<DB> then java -jar <fixed-jar> --apply ., safety-check only tests/ changed, commit, force-push), then drive to green + enqueue. For kompile-cli #124 specifically, its blocker was flaky remote-build e2e tests under saturation, not pruning — retry when the droplet is quiet, or investigate cliRemoteBuildRunsTestsEndToEnd separately if it fails on a clean droplet.
3. Create PRs for the 24 remaining repos (skip the 2 no-ops) using the campaign engine (§4.5) or PR-only subagents (playbook artifacts/.../playbook_pronly.md). UrlProtocol/UrlResolver are huge (876/912 test files) and likely contain libp2p version-self-check tests like ContainerNursery's — expect to have to restore those specific files (git checkout origin/<DB> -- tests/<file>).
4. Pace against the droplet. Only proceed at meaningful concurrency when kotlin.build (remote) is actually passing/merging; if it's jammed (green PRs cycling in the queue with 0 merges over ~30–45 min), pause rather than add load — the merge queue can't pass anything while the pool is saturated, and grinding makes it worse. The user's standing preference (asked mid-session) was "grind all to merged (slow)" but they were about to reconsider given the stall — confirm with the user whether to (a) pause until the pool frees, (b) grind 1-at-a-time at minimal load, or (c) resume 5-wide.
5. Track & report the merged/blocked tally; anything a repo is genuinely BLOCKED on that isn't the droplet (e.g. a real over-removal the fixed jar still misses) should be root-caused — do not paper over.

7. Reusable / operational knowledge

• Fixed jar: /code/workspace/WithArtifactPruner/jars/withartifact-pruner.jar (has the §4.2 fix; main does not until PR #2 merges).
• kompile test.kts constraint: files may contain only top-level functions (no top-level class/object/val), and test funs must have no return type + block body. Put all helpers (fake resolvers, comparators, typed funs) local inside the test fun.
• Stale-module cache: the aibuildcache/bldbinary serves a stale 0.0.1 module jar for src/ edits at the same coordinate → local scripts/test.bash tests the OLD module. Purge ~/.aibuildcaches/_code_workspace_WithArtifactPruner before trusting a local pruner-suite run.
• Merge mechanism: these repos use a SQUASH merge queue with auto-merge OFF — gh pr merge --squash fails ("Auto merge is not allowed"). Enqueue via GraphQL: get id from gh pr view <n> --json id, then gh api graphql -f query='mutation{enqueuePullRequest(input:{pullRequestId:"<ID>"}){mergeQueueEntry{state}}}'. The queue drops PRs when the one ahead merges (re-evaluation) — re-enqueue dropped-but-green PRs.
• Re-trigger a flaky check: git commit --amend --no-edit --date=now && git push --force (new SHA re-runs the external kotlin.build (remote) check; GitHub can't re-run external statuses).
• coursier resolve prints to stderr — capture 2>&1.
• Campaign files (in /tmp, also archived as artifacts): campaign.sh (engine), wave_driver.sh (auto-enqueue driver), playbook_pronly.md (PR-only worker prompt), workset_final.txt (37 repos), campaign_queue.txt (24 not-PR'd), campaign_done.txt.
• Two stale local commits exist and should NOT be pushed: community.kotlin.psi.leakproof (already merged via #18) and kompile-cli (branch already carries its prune) each have an unpushed redundant re-prune commit in their /code/workspace/<repo> clones — ignore them.
• Don't bypass CI/merge-queue (--admin/--no-verify forbidden); an infra-jammed required check is reported to the user, not bypassed.

Related memory / handoffs

See the author's memory notes withartifact-pruner-tool, kompile-test-kts-only-top-level-functions. Recent buildtest/droplet-saturation handoffs in this directory (e.g. 2026-07-05-ci-stabilization-and-fleet-redeploy.md, 2026-07-05-buildtest-scheduling-cn-outage-and-getbuildrun-rpc-flake.md) are the relevant infra context for the §4.6 blocker.