Handoff: identify the first-attempt CI read-test cost and resolve the raw client-call target

Written 2026-10-19 09:46 UTC.

RE-VERIFY: This is a snapshot. Recheck https://github.com/CodexCoder21Organization/SimpleFileSystemServiceServer/pull/35 with gh pr view (state/headRefOid/body/statusCheckRollup), and BuildTest run b1301d84 at https://buildtest.kotlin.build/run?id=b1301d84 before acting.

Design decisions in force

The 2026-10-19 user brief asks to measure head702e4713 against a diagnostic bypassing connection checks on two CPUs, record a mechanism before changing code, add a failing work-count test, fix and push the existing open PR, then replace its description with a final description and current full-remote first-attempt counts. The diagnostic disabling checks is local only, not to push. Exact remaining user constraints:

• “Never raise timeouts, cut iterations, or weaken assertions.”
• “Push the fix to PR 35's branch (check it is still OPEN; fetch, rebase, --force-with-lease).” Existing PR: https://github.com/CodexCoder21Organization/SimpleFileSystemServiceServer/pull/35 .
• “Commits end with Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>. Do NOT merge, enqueue, publish or deploy. Any wait loop has a hard deadline and is killed before you finish. Checkpoint out/r78-findings.md every 15 minutes.”
• “Disk: df -h / before any clone/build; stop under 10 GB.”
• “Memory: the machine has ~15 GB shared with other jobs and local runs get killed for memory — prefer scripts/test.bash --remote --test <name>, and run at most one capped local JVM. Delete clones and ~/.aibuildcaches entries you create.”
• Rewrite the description with the original opening why-paragraph, then mechanism/change/current full-remote first-attempt counts; use gh api -X PATCH repos/CodexCoder21Organization/SimpleFileSystemServiceServer/pulls/35 -F body=@file, read back, and end exactly with “🤖 Generated with Claude Code”.

Mission and state

The original test testSmallSourcesCompleteInOneRpc failed its first attempt around33s in three full CI runs and passed automatic retries. It has112 serial tiny reads but also many later scenarios. The user narrowed the earlier r75 investigation to measuring extra connection-check work, proving a work-count failure, then fixing the existing PR. No product implementation was changed here, and the PR body was not rewritten with unverified claims. The33s cause is still unknown.

Findings and proof

1. Ten local two-CPU timing bodies completed all original assertions: eight direct-main pairs, then one pair through existing native kompile.TestRunner.executeTest using saved r75 prepared artifacts. Heap128MiB, SerialGC, ActiveProcessorCount2, TieredStopAtLevel1, taskset0,1. Exact saved runner classpath/native SHA agent was included for six of the ten. These are LOCAL samples, not CI-droplet/native-parent samples. CSVs preserve each call.
2. The first112 reads use112 service RPCs and224 empty checkReadConnection sandbox proxy calls, not336 service RPCs. Check totals ranged0.179–0.275 seconds; no check took seconds. In the latest native-entry pair the tiny loop took2508/2059ms (head/disabled), checks275.085/0.190ms, readStart2207/2040ms. Whole Java-source-launch processes13.048/11.547s include Java launcher compilation, so do not treat these as runner-reported test durations.
3. Specific excess-work mechanism: client SimpleFileSystemClient.kt:109 unconditionally invokes wire.checkReadConnection() from inline first-read and close (lines271/296). VNextSjvmClient.java:14 is empty, but resolver SandboxedProxyGenerator.kt:1502 calls invokeSandboxMethod at1264, submitting a worker/Future and entering SJVM. This proves extra work, not the33s cause. The full original test actually makes271 readStart calls and518 checks, plus27 calls during its measured Filedrop commit.
4. New work-count regression source is saved inertly as review-evidence/r78/testSmallSourcesClientCallBudget.kts.txt. It retains112 reads and now requires0 extra checks and112 total entries. It failed against unchanged head: expected0, actual224. An earlier copied224-check assertion contradicted the total112 target; independent review caught it and it was corrected before the rerun.
5. Simply deleting checks violates an existing observable behavior. The unchanged testInlineConnectionShutdownMatchesStreaming passed. A diagnostic bypassing ONLY its owning wire's checks, after closing the real raw proxy with resolver's public close API, let inline read return1 byte while streaming threw SandboxException; existing assertion failed. Both full failure stacks are saved. This is a deterministic public-close control, not a guessed failure.
6. Independent review found that raw proxy lifecycle handle exposes only close, no local status check in resolver0.0.1306 (server runtime) or0.0.866 (client compile dependency). Creation observer is not a proxy status API. Managed factory owns a volatile closed flag and can optimize open checks locally, but the original test uses raw RemoteSimpleFileSystemManager(wire) and factory-only optimization would leave it unchanged. Keeping raw close behavior while requiring only112 raw entries needs a new upstream local lifecycle API or an explicit change of construction-path scope. Service-RPC promise of112 is already met; total112 proxy entries is the stronger new target.
7. Additional anomaly: original raw-close control process took29.504s despite no112-read loop; the mutation process8.520s. There were no body phase timers on that control, so process source compilation/setup/body/teardown are unseparated. Do not call this a reproduced small-read stall or blame checks.
8. Native remote probe run b1301d84 failed BEFORE tests/machine allocation. Its last provisioning error was a30s renewDroplet(J) timeout invalidating the provisioning sandbox; later DropletCreationStatusImpl.getRequestId failed on that invalidated sandbox; provisioning was requeued, then the5-minute watchdog ended run at6 minutes.0 tests ran. Full logs/result and isolated run metadata are saved. Do not retry unchanged. Earlier local client373d15c6 was externally interrupted; selected baseline492daa0a and artifact preparation af883963 were canceled before allocation in favor of the native probes. None is timing/test evidence.

Open questions / blockers

The assistant has asked the user whether an unpublished upstream resolver API change may be prepared, versus keeping changes within the file-storage PR. No approval has been inferred. That API would address the stronger raw-call target; it is NOT a demonstrated33s fix. Native CI cost still needs measured evidence; BuildTest provisioning currently prevents it. No runtime fix should be guessed from these samples.

Relevant PRs / refs

No code was deployed or published; no merge/enqueue was performed. All r78 commits use the requested coauthor. Diagnostic code that disables checks is deliberately excluded from remote branches per the user's instruction; compiled jars/caches are also excluded. The original source can be recovered from head702e4713 and the diagnostic described above; traces and the corrected regression are durable. Prior r75 evidence remains on https://github.com/CodexCoder21Organization/SimpleFileSystemServiceServer/tree/wip/r75-stall-evidence .

Next steps

1. Reverify PR, provisioning and current source. Read the evidence findings and independent review from the evidence branch, plus live Testing Architecture and Engineering Philosophy. Start a fresh clone under workspace and check disk first.
2. Resolve the raw-vs-managed construction-path target with the user. Do not replace the existing raw-close assertion or claim factory-only changes fix the raw test. If upstream API work is chosen, keep it unpublished unless the user changes the prohibition.
3. Recover remote provisioning or use an authorized two-CPU environment that reproduces the CI test context; get per-call timings from a first-attempt native run. Keep the old test workload/assertions/budgets. Follow progress/cost rather than assigning one30s wait from a Future stack.
4. Record the actual33s mechanism before implementation changes; preserve the work-count failing test, fix the owning code, and prove both call count and connection errors. Run full remote suite and read raw first-attempt records (not final success alone), then update existing PR via requested PATCH/readback. No merge/enqueue/publication/deployment.

Operational notes

Full findings: https://github.com/CodexCoder21Organization/SimpleFileSystemServiceServer/blob/wip/r78-call-timing-evidence/review-evidence/r78/r78-findings.md . Independent review: https://github.com/CodexCoder21Organization/SimpleFileSystemServiceServer/blob/wip/r78-call-timing-evidence/review-evidence/r78/r78-independent-review.md . All native/provisioning waits were bounded. Cleanup verified 2026-10-19 09:49 UTC: r78 clone and four generated diagnostic jars deleted; no owned r78 .aibuildcaches entries remain. No subagent is running. report-challenge CLI was not used because it automatically merges/enqueues, prohibited by the user; the infrastructure challenge is preserved on the evidence branch instead.