Handoff: Approve and publish the corrected projection-journal candidate, then finish the consumer gate
Written 2026-10-17T08:08:54.478837+00:00.
RE-VERIFY: This is a write-time snapshot. Re-check both PRs with gh pr view <full-URL> --json state,mergeStateStatus,statusCheckRollup,mergedAt, verify branch heads with git ls-remote, and verify Maven version absence with HEAD requests before any publication. No merge, queue, production deployment or service restart is authorized. The original brief says not to watch CI; only one final status read was performed.
Mission and decision needed
The user assigned round-3 fixes on the two PRs below after projection maintenance consumed about 27% of a core continuously serializing a growing journal during the 2026-10-15 CPU-saturation incident. The brief granted exactly one publication of 0.0.14-cpuamortize2-rc3, 27 consumer pins, targeted consumer tests, final module suite, pushes and updated descriptions, then stop. That publication grant has been used. rc3 passed its module suite but failed a real consumer heap guard. A corrected unpublished rc4 is built and locally verified; it must not be published without a new grant.
The user was asked through asynchronous input: May I publish 0.0.14-cpuamortize2-rc4 and finish the consumer repin? Options: authorize one rc4 publication and finish (recommended), or stop after checkpointing. No answer has been received or assumed. This is the only approval gate; do not overwrite or delete rc3.
What was found and done
- F1 reproduced: every 11-minute collection expired reset liveness, then a reset during staging made the old null-floor check discard the whole baseline. The deterministic public-clock test collected 720 rows for 18 appends and published zero times in six attempts. Reusable reset slots now remain represented until replacement/ceiling. At every delta capture, actual reset targets and that committed cut are sampled under the same journal lock. A reset created after the cut needs only later residual rows. Every tested long-collection attempt now publishes, without a retry cap.
- F2 reproduced under the unchanged default 128 MiB heap: 650 valid 256 KiB tail rows retained only by count caused OOM and committed sequence 2050 became 0. The loading cache is capped by count and min(fixed byte budget, 8 MiB), leaving the permitted tail on disk.
openOrReplace and loadFingerprints catch Exception, allowing Error to propagate. Real large-row test now preserves sequence, cursor, fingerprints and committed file digest. A callback Error test failed first and now verifies unchanged Error identity and a later unchanged reopen.
- F3 fixed: consumer guard uses static class references, including
BuildTestEmbeddedService::class.java, instead of Class.forName. All 27 consumer pins currently name rc3. README contracts were updated.
- Initial module head 0725bc377843c4657e9868f1b528b6e3123e3439 passed 77/77 and was published as rc3. That version is immutable. Consumer checks found a regression: a speculative floor set to latestSequence retained 4,500 distinct 12 KiB delta rows for a reset that never arrived, and the 128 MiB child exited 3 with OOM. Served-rc3 consumer result is 22/23, not green; the single heap scenario failed in three independent local invocations.
- Added fail-first public module test
journalNoResetDuringLargeRefreshDoesNotHoldTail; rc3 fails its settle-in-one-publication assertion. Removed the speculative floor and replaced it with the actual reset/delta-cut design in item 1. The revised module passes 78/78 full local tests, 0 failed, at exact remote head 88d0d806c5a01d6ed75a3e547a9574f4bfa1db10. Targeted five-case reset/no-reset group passes 5/5.
- Captured the real consumer heap child's runtime dependencies from this checkout, removed its single rc3 journal JAR, and supplied only the local rc4 journal JAR instead. The unchanged child entry point, 4,500 x 12 KiB data and
-Xmx128m -XX:+UseSerialGC -XX:+ExitOnOutOfMemoryError now exit 0 and print OK largeDeltaRuns=4500 notesBytes=12288 handlerCalls=512 handlerCallsDuringCompaction=512 maxHandlerMs=7. Class-load output proves the journal came from the rc4 file. This is local cross-repository evidence, not a served-rc4 23-test result.
Relevant PRs and branches
| Repo |
Branch |
Remote head SHA |
PR |
Contents |
State |
| BuildTestProjectionJournal |
https://github.com/CodexCoder21Organization/BuildTestProjectionJournal/tree/fix/amortized-compaction-trigger |
88d0d806c5a01d6ed75a3e547a9574f4bfa1db10 |
https://github.com/CodexCoder21Organization/BuildTestProjectionJournal/pull/8 |
All F1/F2 fixes, seven new review/regression scenarios total, rc4 version line |
OPEN, unmerged; full local 78/78; bld-build requested and IN_PROGRESS at 08:07 UTC |
| BuildTestEmbedded |
https://github.com/CodexCoder21Organization/BuildTestEmbedded/tree/fix/amortized-projection-compaction |
a2285da0d4b28c596eb8c18bcd28f3c442880eb8 |
https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1490 |
27 rc3 pins, static classpath references, README; rebased on 8de16ac48cfa6837947c74811f2dff49b4710813 |
OPEN, unmerged; served-rc3 local 22/23, inherited large-delta OOM; 16 CI shards requested |
Both working trees are clean; no stashes, local-only branch commits, extra worktrees or delegated agents exist. Both PR descriptions were updated through REST PATCH, describe the known-bad rc3 and unpublished corrected rc4, and open with the original motivation.
Published but not merged
Published coordinate: buildtest.projection.journal:buildtest-projection-journal:0.0.14-cpuamortize2-rc3, built from https://github.com/CodexCoder21Organization/BuildTestProjectionJournal/commit/0725bc377843c4657e9868f1b528b6e3123e3439, not merged to main. Repository: https://kotlin.directory/buildtest/projection/journal/buildtest-projection-journal/0.0.14-cpuamortize2-rc3/ . Both files were HTTP 404 before the single documented publisher invocation, then byte-verified against the local build. JAR SHA-256 d264f46baa03aff021e67af4195085bc8b99a0b3fb4afc6ba3f6fe2a2f53f18c; POM c85425602a920410b79ac0802697468363cdc90d560f41a9dc8580d00fd76ad1. Known consumer heap regression: do not land this version. Do not overwrite it. No production deployment or service restart occurred.
Corrected local artifact: 0.0.14-cpuamortize2-rc4, built from final module remote head above. Both proposed served POM/JAR paths returned 404 at 08:05 UTC. Local JAR 3a561d8d44952ba24ed22c0398373b8101532328338f3c0e9eee80231892dcb5; POM 99a521b58eda8c19eabbdf2b13be1110632d347f7aeb9fd0d90600262adda810. It has NOT been uploaded. Build outputs/dependency caches are intentionally not committed; recreate them from the remote source head if this machine is cleared, then verify served bytes against that local build.
Exact remaining steps
- Re-verify live state and resolve the explicit publication approval above. An elapsed wait is not approval. If permission is denied, stop with both PRs open and the failure stated plainly.
- If approved, fetch/rebase module on latest main before building. If head changes, re-run the full module local suite after targeted checks as required. Build:
scripts/build.bash --local buildtest.projection.journal.buildMaven /tmp/btcpu-rc4-artifact from the module root.
- Confirm rc4 POM and JAR are both absent; refuse to overwrite an existing version. Publish via the skill's CLI:
jars/coursier launch community.kotlin.maven.artifact.publishing:community-kotlin-maven-artifact-publishing:0.0.6 -r https://kotlin.directory -- --artifact-dir /tmp/btcpu-rc4-artifact --api-url https://api.kotlin.directory/upload --force. Read the publisher's own exit code. Download served POM/JAR and compare bytes and SHA-256 with the local output.
- Replace all 27 consumer rc3 coordinates with rc4. Keep the already-fixed static references and all existing test sizes, heap caps and timeouts. Fetch/rebase consumer main before its local tests. Run exactly the 23 selected scenarios listed below, using explicit --local and no full consumer suite. Source in the child compaction harness has no downstream workaround.
- Commit with
git -c user.name=codexcoder21 -c user.email=admin.gmusic@gmail.com commit ..., ending the message with Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>. Check the PR remains open before push; push its named branch with --force-with-lease if rebased. Verify remote SHA. Update both descriptions through gh api --method PATCH ... --input <json-file>; gh pr edit is known not to work on this box. Record rc4 hashes, exact heads and counts, and explain that rc3 is superseded.
- Stop with PRs open and branch CI requested. Do not watch CI, enqueue, merge, deploy, restart or make another publication without explicit scope. The original brief's module-green requirement conflicts with its later no-watching stop rule; the worker followed the explicit no-watching instruction and reports the one-time state read honestly.
Consumer targets (the prior 21 already include the heap guard; two reader-retention checks were added):
projectionStartupKeepsAmortizedTailWithinTightHeap
projectionStartupFoldOfLongEvictedTailFitsTightHeap
projectionChangeFeedFollowUpCompactionSettlesAndResetsStrandedCursor
stressProjectionChangeFeedReadDoesNotWaitForCommittedRewrite
projectionChangeFeedRestartHonorsPersistedProtectedWindow
testProjectionStartupSealDecodesOnlyTailWindow
projectionChangeFeedCompactionReportsRoom
projectionPublicationKeepsFeedReadersLiveUnderSustainedWriters
testProjectionPersistentSourceReadFaultBoundsDescriptors
projectionChangeFeedReadRemainsAvailableDuringContinuousMaintenance
projectionRunDeletionDuringResetKeepsReaderProgress
projectionChangeFeedRecordCompactionBuysRoom
projectionChangeFeedCompactionRoomSurvivesRestart
projectionCompactionKeepsActiveRunSourcesAndDropsTerminalUtilization
projectionChangeFeedStartupCompactsLegacyJournalUnderTightHeap
projectionCompactionLargeDeltaStreamsUnderTightHeap
projectionCompactionDeltaAppendFitsDefaultHeapAtProductionRate
projectionChangeFeedCompactionRetainsCurrentRowsPerRun
projectionChangeFeedCompactedBaselineReplaysThroughPagedConsumerContract
projectionChangeFeedReadRemainsAvailableDuringRenewalCompactionUnderTightHeap
projectionStartupCompactionPublishesWhileAppendDrainerStaysActive
testProjectionIdleReaderStopsHoldingRetention
testProjectionResetReaderReleasesItsRetentionPosition
Operational notes and reproducibility
Use the caller-provided checkouts only; this job's roots were /tmp/claude-501/-code/59325ee8-a2d5-4074-bc13-a4a83499a3e5/scratchpad/workspace/btcpu-fix2/btpj and .../bte. A fresh pickup should clone the linked named branches inside its own workspace. Run build/test scripts from the corresponding repo cwd: --test . resolves against cwd even if the wrapper supplies -w, and an accidental parent-cwd launch was stopped during preparation before tests ran. Always pass --local; never use race mode or a remote client. At most one full local suite machine-wide: inspect actual script argv/PIDs, because grep on full prompt strings counted seven or nine nonexistent full-suite launches. Wait by tracked PID with a hard deadline, never by self-matching command text.
No global git identity exists. A rebase that needed to create commits initially stopped on missing identity and left no rebase message; aborting and rerunning git -c user.name=codexcoder21 -c user.email=admin.gmusic@gmail.com rebase origin/main completed cleanly. All final source changes are remote. No timeout, iteration, assertion or memory cap was loosened. Documentation read: both READMEs, reviewer findings, TESTING.md (especially TDD, public API only, OOM/streaming, Never Calibrate a Reproducer to a Machine) and PHILOSOPHY.md; skills github-repos, publish-maven-artifact, status-report and create-handoff.
The full incremental findings are preserved below so the next person needs neither this conversation nor local logs. Build outputs/caches were deliberately omitted. The public test code, fail-first commits and exact immutable artifact URLs make every key result independently reproducible.
Incremental findings record
Round 3 fixes — progress and evidence
2026-10-17 07:39:44 UTC — OBSERVED: Both assigned checkouts are clean, at the assigned heads, and fetched/rebased successfully on origin/main. Read module README, reviewer findings, skills github-repos / publish-maven-artifact / status-report, and retrieved TESTING.md and PHILOSOPHY.md. No repository AGENTS.md exists. INFER: The explicit one-publication rc3 grant overrides the generic later no-publication rule; the explicit no-CI-watching rule overrides the initial green-CI target for this worker. No delegation, merge, queue or production operation will occur.
Plan:
- [in progress] Read public contracts and relevant code; import and reproduce the two review failures; checkpoint tests.
- [pending] Fix reset-floor progress with public-API evidence; checkpoint.
- [pending] Bound startup decoded bytes and propagate load errors; checkpoint.
- [pending] Bump rc3, run final full module suite once, push, build/publish new candidate and byte-verify served files.
- [pending] Move 27 consumer pins, remove reflection, run 22 targeted tests, push.
- [pending] REST PATCH both PR descriptions, audit remote heads and background processes, stop.
2026-10-17T07:41:48.042032+00:00 — OBSERVED: F1 baseline run failed at its exact intended progress assertion: 720 baseline rows collected; 0 publications, 6 discarded attempts for 18 appended rows. Full output and stack trace: /tmp/btcpu-f1-before.log. Imported all five journalReview scripts with the requested glob; checkpoint push underway.
INFER: Mechanism to fix F1: liveness-filtered capture omits dormant advertised reset targets, then a read revives one and unconditionally discards a stage. Capture all reusable reset targets and the current committed end as a conservative floor; future newly created resets cannot precede that end. Retain concurrent suffix under existing ceiling policy instead of invalidating. The old reset-work test requires at least one discarded attempt, which asserts the defective behavior and must be changed to require zero discards.
Invariant table before implementation:
| Resource | Creator / owner / closer | Terminal / re-entry / restart contract | Public-API evidence |
|---|---|---|---|
| Committed feed and staged baseline | Journal owns committed generation; caller owns and closes baseline; compactor releases captured views | Reset callbacks can re-enter reads and append; a reset cannot force unbounded discarded stages; close prevents publication; restart preserves rows and fingerprints | journalReviewLongCollectionResetDoesNotStarveCompaction; journalLateResetReaderKeepsUnreadTail; journalResetReadersDuringStagingKeepWorkBounded; two streaming-close review tests |
| Startup decoded row window | Loader owns working cache, disk stage owns allowed tail; generation close releases stage | Working decoded bytes and count bounded; valid in-bound tail stays on disk; load Error propagates without changing cursor authority | journalReviewStartupLargeRowsStayBounded; added load-error identity preservation scenario planned |
2026-10-17T07:44:57.645464+00:00 — OBSERVED: F1 targeted validation PASS 3/3 (long collection/reset, bounded reset-work, late reset unread suffix), XML /tmp/btcpu-f1-after.xml. Strengthened long-collection check to require all six publications and zero discarded stages. F1 fix committed as da950ea and checkpoint-pushed with startup Error fail-first test at remote head d7db850.
INFER: Retaining every reusable reset target also in needsCompaction is necessary: capturing idle snapshots only during staging could retain an over-budget suffix while the maintenance trigger ignored the same floor, causing repeated useless rewrites. Both paths now use the same floor and existing ceilings. Tail-reader liveness still expires. No new timeout, retry cap, or larger heap was added.
OBSERVED: Added startup observation Error propagation test; first invocation found missing required constructor budgets (compile-only failure /tmp/btcpu-f2-error-before.log, then corrected); its intended baseline behavior run is underway. The independently running large-row baseline test is still in its 300-second per-test budget.
Plan: [done] import/checkpoint F1 failure; [done] implement/target-test/checkpoint F1; [in progress] prove F2 large-row and load-Error baseline failures; [pending] bounded decoded cache, final full module suite, rc3 publication, consumer pins/tests, descriptions.
2026-10-17T07:45:37.555836+00:00 — OBSERVED: F2 baseline FAIL 0/1 at exact sequence assertion (2050 -> 0), real OutOfMemoryError at line 1772, 57.5 seconds. Full stack/output /tmp/btcpu-f2-before.log. Startup callback Error baseline FAIL 0/1: the callback throws AssertionError but constructor returns successfully after resetting authority; full stack /tmp/btcpu-f2-error-before.log.
INFER: Mechanism to fix F2: recentTailRows caps only count, retaining over 96 MiB of payloads before 1024 rows. Add independent raw-byte accounting capped at min(fixed budget, 8 MiB); evict parsed cache entries only, leaving the permitted tail on disk. Narrow both openOrReplace and loadFingerprints recovery catches to Exception, allowing Error (including OOM) to propagate without constructing an empty authority. Callback test covers Error propagation without inventing an OOM; large-row test exercises the real heap condition.
2026-10-17T07:47:39.851943+00:00 — OBSERVED: F2 targeted tests PASS 3/3: large-row restart, startup callback Error propagation/authority preservation, amortized-bound reopen. XML /tmp/btcpu-f2-after.xml. F2 implementation checkpoint-pushed as 1745c41. Large-row test now additionally requires no compaction obligation and identical committed journal SHA-256 after reopen.
OBSERVED: Consumer edits are prepared: exactly 27 rc2 pins replaced with rc3; guard classpath uses only static class references including BuildTestEmbeddedService::class.java. Existing target-list file contains 21 tests including the guard, so the planned final consumer set adds ordinary reader-liveness and reset-release tests (23 unique checks) to cover the reset policy change. No consumer full suite will run.
2026-10-17T07:48:39.358991+00:00 — OBSERVED: Final module head is 0725bc377843c4657e9868f1b528b6e3123e3439, with committed rc3 version line and removal of obsolete reset liveness bookkeeping. POM and JAR rc3 preflight both HTTP 404. Full local suite started at this head, after checking no real full-suite script was active. The mandated grep count falsely included supervisor prompt strings (7); /proc argv inspection confirmed zero actual full-suite scripts.
OBSERVED: A first full-suite launch used the parent cwd with btpj/scripts/test.bash; kompile warned that --test . resolved outside its workspace and scanned sibling consumer metadata. Stopped this local preparation immediately (owned PIDs 21924,21925,21955,22608; terminal exit 143), before any test executed. Corrected launch runs with cwd btpj, explicit --local, tracked session 9382, 1800-second hard cap. No consumer full-suite result or run is claimed. Build candidate is a separate tracked local process, session 5801.
OBSERVED: Read TESTING.md sections TDD, public API only/no reflection, byte/row extremes, in-memory error injection, local test gates and Never Calibrate a Reproducer to a Machine, plus PHILOSOPHY.md and both README contracts/build instructions. Source-level independent test-coverage review: manual-clock ordering is deterministic; heap fixture forces real parsed retention at 128 MiB; Error test targets actual callback propagation and exact identity, not a synthetic OOM; streaming-close tests retain public API ownership coverage.
2026-10-17T07:50:20.418591+00:00 — OBSERVED: Local rc3 build succeeded (707617 output bytes, directory /tmp/btcpu-rc3-artifact). Local JAR SHA-256 d264f46baa03aff021e67af4195085bc8b99a0b3fb4afc6ba3f6fe2a2f53f18c; POM SHA-256 c85425602a920410b79ac0802697468363cdc90d560f41a9dc8580d00fd76ad1. Publication waits for the final module suite, so the one immutable candidate is not used before validation.
OBSERVED: Consumer rebase picked up newly merged main 8de16ac48cfa6837947c74811f2dff49b4710813. First rebase stopped because no global git identity exists; aborted and cleanly repeated with the required per-command identity, without conflict edits. Consumer is clean at a2285da0d4b28c596eb8c18bcd28f3c442880eb8. Added README public ratio/startup/reset contracts; candidate pin count remains 27.
INFER: Dedicated code review found the reset floor monotone during a compaction: every reusable slot is represented at capture; a later newly captured reset target is at or beyond the captured committed end; replacing a slot cannot introduce an earlier target. Existing ceilings bound snapshot-held tail, and needsCompaction uses the same floors so a retained suffix does not cause per-append rewrites. Startup byte accounting removes only working-cache references, subtracts each ordinal exactly once, and resets accounting at both oversized-record boundaries; retained disk rows are unchanged.
2026-10-17T07:50:46.041376+00:00 — OBSERVED: Final module full local suite PASS 77/77, 0 failed, at exact head 0725bc377843c4657e9868f1b528b6e3123e3439. Log /tmp/btcpu-module-final.log; XML /tmp/btcpu-module-final.xml. This is 71 existing + five imported review tests + one startup Error propagation test. All scripts used --local; full suite ran once to completion from cwd btpj. Final module push in progress after confirming unchanged rebase and PR open.
OBSERVED: Immediately before the one authorized publication, both rc3 served POM and JAR again returned HTTP 404. Invoked documented publisher 0.0.6 with --artifact-dir /tmp/btcpu-rc3-artifact, HTTP API URL and --force (skip prompt only); tracked session 69699 with a 900-second process cap. No manual upload, version overwrite or source-version temporary edit is used.
2026-10-17T07:51:19.836079+00:00 — OBSERVED: rc3 publisher exited 0. Downloaded served POM and JAR and compared bytes and SHA-256 against the local build: identical. JAR (705050 bytes) d264f46baa03aff021e67af4195085bc8b99a0b3fb4afc6ba3f6fe2a2f53f18c; POM (2295 bytes) c85425602a920410b79ac0802697468363cdc90d560f41a9dc8580d00fd76ad1. Coordinate buildtest.projection.journal:buildtest-projection-journal:0.0.14-cpuamortize2-rc3. Both files were 404 before publication; no existing version was overwritten. Committed version line already names rc3 and needed no temporary alteration. Module ls-remote confirms final tested head 0725bc377843c4657e9868f1b528b6e3123e3439.
OBSERVED: XML independently confirms module 77 passed, 0 failed, 0 failed build rules. Consumer targeted run is starting from final rebased head a2285da0d4b28c596eb8c18bcd28f3c442880eb8 against served rc3; target names saved to /tmp/btcpu-fix2-consumer-targets.txt.
Plan: [done] F1/F2 fail-first evidence, fixes and checkpoint pushes; [done] final module full 77/77 and push; [done] rc3 publication/served-byte verification; [in progress] consumer 23 targeted checks; [pending] consumer push, both descriptions, final remote/process audit.
2026-10-17T07:51:48.229019+00:00 — OBSERVED: Baseline failure stack traces below (the original exceptions, before runner duplicate wrappers). All three are expected fail-first evidence; none remains an unresolved test failure.
F1 progress
java.lang.AssertionError: Compaction must publish within six attempts despite long baseline collections and reset reads; published=0 discarded=6 collected=720 baseline rows for 18 appended rows.
at kotlin.test.DefaultAsserter.fail(DefaultAsserter.kt:16)
at kotlin.test.Asserter$DefaultImpls.assertTrue(Assertions.kt:652)
at kotlin.test.DefaultAsserter.assertTrue(DefaultAsserter.kt:11)
at kotlin.test.Asserter$DefaultImpls.assertTrue(Assertions.kt:662)
at kotlin.test.DefaultAsserter.assertTrue(DefaultAsserter.kt:11)
at kotlin.test.AssertionsKt__AssertionsKt.assertTrue(Assertions.kt:44)
at kotlin.test.AssertionsKt.assertTrue(Unknown Source)
at buildtest.projection.journal.JournalReviewLongCollectionResetDoesNotStarveCompactionKt.journalReviewLongCollectionResetDoesNotStarveCompaction(journalReviewLongCollectionResetDoesNotStarveCompaction.kt:90)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at kompile.TestRunner.executeTest(TestRunner.kt:49)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at community.kotlin.kompile.testrunner.bootstrap.BootstrapRunner.main(BootstrapRunner.java:449)
FAILED buildtest.projection.journal.journalReviewLongCollectionResetDoesNotStarveCompaction -> java.lang.AssertionError: Compaction must publish within six attempts despite long baseline collections and reset reads; published=0 discarded=6 collected=720 baseline rows for 18 appended rows. (in 1.0 seconds)
F2 large-row restart
java.lang.AssertionError: Reopening valid committed large rows must preserve their sequence under the default 128 MiB heap.. Expected <2050>, actual <0>.
at kotlin.test.DefaultAsserter.fail(DefaultAsserter.kt:16)
at kotlin.test.Asserter$DefaultImpls.assertTrue(Assertions.kt:652)
at kotlin.test.DefaultAsserter.assertTrue(DefaultAsserter.kt:11)
at kotlin.test.Asserter$DefaultImpls.assertEquals(Assertions.kt:671)
at kotlin.test.DefaultAsserter.assertEquals(DefaultAsserter.kt:11)
at kotlin.test.AssertionsKt__AssertionsKt.assertEquals(Assertions.kt:63)
at kotlin.test.AssertionsKt.assertEquals(Unknown Source)
at buildtest.projection.journal.JournalReviewStartupLargeRowsStayBoundedKt.journalReviewStartupLargeRowsStayBounded(journalReviewStartupLargeRowsStayBounded.kt:52)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at kompile.TestRunner.executeTest(TestRunner.kt:49)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at community.kotlin.kompile.testrunner.bootstrap.BootstrapRunner.main(BootstrapRunner.java:449)
FAILED buildtest.projection.journal.journalReviewStartupLargeRowsStayBounded -> java.lang.AssertionError: Reopening valid committed large rows must preserve their sequence under the default 128 MiB heap.. Expected <2050>, actual <0>. (in 57.5 seconds)
Startup callback Error
java.lang.AssertionError: Expected an exception of class java.lang.AssertionError to be thrown, but was completed successfully.
at kotlin.test.DefaultAsserter.fail(DefaultAsserter.kt:16)
at kotlin.test.AssertionsKt__AssertionsImplKt.checkResultIsFailure(AssertionsImpl.kt:22)
at kotlin.test.AssertionsKt.checkResultIsFailure(Unknown Source)
at buildtest.projection.journal.JournalStartupCallbackErrorPreservesCommittedAuthorityKt.journalStartupCallbackErrorPreservesCommittedAuthority(journalStartupCallbackErrorPreservesCommittedAuthority.kt:32)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at kompile.TestRunner.executeTest(TestRunner.kt:49)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at community.kotlin.kompile.testrunner.bootstrap.BootstrapRunner.main(BootstrapRunner.java:449)
FAILED buildtest.projection.journal.journalStartupCallbackErrorPreservesCommittedAuthority -> java.lang.AssertionError: Expected an exception of class java.lang.AssertionError to be thrown, but was completed successfully. (in 0.8 seconds)
2026-10-17T07:56:44.482013+00:00 — OBSERVED: Module description successfully REST PATCHed for Review round 3 at remote head 0725bc377843c4657e9868f1b528b6e3123e3439, still OPEN; it records module full 77/77, both mechanisms, published rc3 checksums, and consumer check count pending. Consumer local targeted run remains active with compiler/build output and no terminal result yet. Its resolved Coursier rc3 JAR hash is the exact served/local d264f46b... value. Both checkout trees are clean. No CI watcher, delegation, queue, merge or production operation has been started.
2026-10-17T07:57:11.075899+00:00 — OBSERVED: Consumer terminal result FAIL: 22/23 passed, projectionCompactionLargeDeltaStreamsUnderTightHeap fails with child exit 3 (real -XX:+ExitOnOutOfMemoryError) at 4500 distinct 12 KiB run rows. All other selected cases, including static-reference startup guard and both reader-retention checks, passed. Full XML /tmp/btcpu-fix2-consumer-targeted.xml; log /tmp/btcpu-fix2-consumer-targeted.log.
INFER (course-changing): Conservatively capturing the latest end as a possible future reset floor exposes a remaining materialized protected-suffix rebuild under the 32 MiB ceiling. This regression must be fixed upstream; a consumer workaround or larger heap would be wrong. rc3 is already immutable/published and cannot be overwritten. The brief grants exactly that one publication, so publishing a corrected replacement version would require new authorization after a concrete fixed result is ready. The rc3 landing requirement is not currently satisfied and will not be reported complete.
Consumer failure output
FAILED buildtest.embedded.projectionCompactionLargeDeltaStreamsUnderTightHeap -> java.lang.AssertionError: The projection-compaction child failed under -Xmx128m while processing 4500 distinct 12 KiB run records. Exit status 3 indicates -XX:+ExitOnOutOfMemoryError during large-delta rebuild. Output:
[BuildTestService] Loaded 0 build runs from disk (0 archived runs left on disk, paged on demand)
[BuildTestService] WARNING: Admission configuration: dropletLimit=0, active=false; fleet admission control is disabled
[BuildTestService] Retention configuration: archiveRetentionMs=86400000, logRetentionMs=604800000, runsPoolMaximumBytes=100000000000, runsPoolTargetBytes=90000000000, runRetentionIntervalMs=900000
[BuildTestService] Background recovery cycle complete (0 dirs scanned, 0 recovered, 0 already-recovered skipped, 0 remaining)
[BuildTestService] Projection-feed test-status rows backfill finished over 0 run directories: persisted=0, alreadyCurrent=0, skipped=0, failed=0
[BuildWatchdog] Stopped
[DeletionWorker] Shut down (remaining pending: 0)
[BuildTestService] Loaded 8 build runs from disk (0 archived runs left on disk, paged on demand)
[BuildTestService] WARNING: Admission configuration: dropletLimit=0, active=false; fleet admission control is disabled
[BuildTestService] Retention configuration: archiveRetentionMs=86400000, logRetentionMs=604800000, runsPoolMaximumBytes=100000000000, runsPoolTargetBytes=90000000000, runRetentionIntervalMs=900000
[BuildTestService] Run c36a90b9 has an interrupted chunked upload (chunks on disk, archive not yet assembled) after a service restart — leaving it PENDING for the client to resume the upload; the watchdog will reap it within 5 minutes if the client does not return.
[BuildTestService] Run c25307fc has an interrupted chunked upload (chunks on disk, archive not yet assembled) after a service restart — leaving it PENDING for the client to resume the upload; the watchdog will reap it within 5 minutes if the client does not return.
[BuildTestService] Run e468f3fc has an interrupted chunked upload (chunks on disk, archive not yet assembled) after a service restart — leaving it PENDING for the client to resume the upload; the watchdog will reap it within 5 minutes if the client does not return.
[BuildTestService] Run bb8b5bfb has an interrupted chunked upload (chunks on disk, archive not yet assembled) after a service restart — leaving it PENDING for the client to resume the upload; the watchdog will reap it within 5 minutes if the client does not return.
[BuildTestService] Run 9682315f has an interrupted chunked upload (chunks on disk, archive not yet assembled) after a service restart — leaving it PENDING for the client to resume the upload; the watchdog will reap it within 5 minutes if the client does not return.
[BuildTestService] Run b7f6b7f6 has an interrupted chunked upload (chunks on disk, archive not yet assembled) after a service restart — leaving it PENDING for the client to resume the upload; the watchdog will reap it within 5 minutes if the client does not return.
[BuildTestService] Run 9e30406a has an interrupted chunked upload (chunks on disk, archive not yet assembled) after a service restart — leaving it PENDING for the client to resume the upload; the watchdog will reap it within 5 minutes if the client does not return.
[BuildTestService] Run d46233af has an interrupted chunked upload (chunks on disk, archive not yet assembled) after a service restart — leaving it PENDING for the client to resume the upload; the watchdog will reap it within 5 minutes if the client does not return.
[BuildTestService] Projection-feed test-status rows backfill finished over 8 run directories: persisted=0, alreadyCurrent=0, skipped=8, failed=0
[BuildTestService] Background recovery cycle complete (8 dirs scanned, 0 recovered, 0 already-recovered skipped, 0 remaining)
LARGE_COMPACTION_REPLACEMENT_STAGED
LARGE_DELTA_REFRESH_STARTED
Terminating due to java.lang.OutOfMemoryError: Java heap space
. Expected <0>, actual <3>. (in 19.4 seconds)
--- Console output (buildtest.embedded.projectionCompactionLargeDeltaStreamsUnderTightHeap) ---
java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:118)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at community.kotlin.kompile.testrunner.bootstrap.BootstrapRunner.main(BootstrapRunner.java:449)
Caused by: java.lang.AssertionError: The projection-compaction child failed under -Xmx128m while processing 4500 distinct 12 KiB run records. Exit status 3 indicates -XX:+ExitOnOutOfMemoryError during large-delta rebuild. Output:
[BuildTestService] Loaded 0 build runs from disk (0 archived runs left on disk, paged on demand)
[BuildTestService] WARNING: Admission configuration: dropletLimit=0, active=false; fleet admission control is disabled
[BuildTestService] Retention configuration: archiveRetentionMs=86400000, logRetentionMs=604800000, runsPoolMaximumBytes=100000000000, runsPoolTargetBytes=90000000000, runRetentionIntervalMs=900000
[BuildTestService] Background recovery cycle complete (0 dirs scanned, 0 recovered, 0 already-recovered skipped, 0 remaining)
[BuildTestService] Projection-feed test-status rows backfill finished over 0 run directories: persisted=0, alreadyCurrent=0, skipped=0, failed=0
[BuildWatchdog] Stopped
[DeletionWorker] Shut down (remaining pending: 0)
[BuildTestService] Loaded 8 build runs from disk (0 archived runs left on disk, paged on demand)
[BuildTestService] WARNING: Admission configuration: dropletLimit=0, active=false; fleet admission control is disabled
[BuildTestService] Retention configuration: archiveRetentionMs=86400000, logRetentionMs=604800000, runsPoolMaximumBytes=100000000000, runsPoolTargetBytes=90000000000, runRetentionIntervalMs=900000
[BuildTestService] Run c36a90b9 has an interrupted chunked upload (chunks on disk, archive not yet assembled) after a service restart — leaving it PENDING for the client to resume the upload; the watchdog will reap it within 5 minutes if the client does not return.
[BuildTestService] Run c25307fc has an interrupted chunked upload (chunks on disk, archive not yet assembled) after a service restart — leaving it PENDING for the client to resume the upload; the watchdog will reap it within 5 minutes if the client does not return.
[BuildTestService] Run e468f3fc has an interrupted chunked upload (chunks on disk, archive not yet assembled) after a service restart — leaving it PENDING for the client to resume the upload; the watchdog will reap it within 5 minutes if the client does not return.
[BuildTestService] Run bb8b5bfb has an interrupted chunked upload (chunks on disk, archive not yet assembled) after a service restart — leaving it PENDING for the client to resume the upload; the watchdog will reap it within 5 minutes if the client does not return.
[BuildTestService] Run 9682315f has an interrupted chunked upload (chunks on disk, archive not yet assembled) after a service restart — leaving it PENDING for the client to resume the upload; the watchdog will reap it within 5 minutes if the client does not return.
[BuildTestService] Run b7f6b7f6 has an interrupted chunked upload (chunks on disk, archive not yet assembled) after a service restart — leaving it PENDING for the client to resume the upload; the watchdog will reap it within 5 minutes if the client does not return.
[BuildTestService] Run 9e30406a has an interrupted chunked upload (chunks on disk, archive not yet assembled) after a service restart — leaving it PENDING for the client to resume the upload; the watchdog will reap it within 5 minutes if the client does not return.
[BuildTestService] Run d46233af has an interrupted chunked upload (chunks on disk, archive not yet assembled) after a service restart — leaving it PENDING for the client to resume the upload; the watchdog will reap it within 5 minutes if the client does not return.
[BuildTestService] Projection-feed test-status rows backfill finished over 8 run directories: persisted=0, alreadyCurrent=0, skipped=8, failed=0
[BuildTestService] Background recovery cycle complete (8 dirs scanned, 0 recovered, 0 already-recovered skipped, 0 remaining)
LARGE_COMPACTION_REPLACEMENT_STAGED
LARGE_DELTA_REFRESH_STARTED
Terminating due to java.lang.OutOfMemoryError: Java heap space
. Expected <0>, actual <3>.
at kotlin.test.DefaultAsserter.fail(DefaultAsserter.kt:16)
at kotlin.test.Asserter$DefaultImpls.assertTrue(Assertions.kt:652)
at kotlin.test.DefaultAsserter.assertTrue(DefaultAsserter.kt:11)
at kotlin.test.Asserter$DefaultImpls.assertEquals(Assertions.kt:671)
at kotlin.test.DefaultAsserter.assertEquals(DefaultAsserter.kt:11)
at kotlin.test.AssertionsKt__AssertionsKt.assertEquals(Assertions.kt:63)
at kotlin.test.AssertionsKt.assertEquals(Unknown Source)
at buildtest.embedded.ProjectionCompactionLargeDeltaStreamsUnderTightHeapKt.projectionCompactionLargeDeltaStreamsUnderTightHeap(projectionCompactionLargeDeltaStreamsUnderTightHeap.kt:55)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at kompile.TestRunner.executeTest(TestRunner.kt:49)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
... 2 more
2026-10-17T08:01:07.354176+00:00 — RUNNING status report (20-minute checkpoint). OBSERVED: Original task is round-3 fixes on the module and consumer PRs, one immutable rc3 publication, final pushes and descriptions, no CI watching/merging. Module PR https://github.com/CodexCoder21Organization/BuildTestProjectionJournal/pull/8 remains OPEN per its last mutation/read; consumer PR https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1490 is OPEN at the last verified original head. No delegated work exists. No landing gate is idle: the newly exposed module regression is being fixed/tested before a corrected candidate can be prepared.
OBSERVED: New module public-API scenario journalNoResetDuringLargeRefreshDoesNotHoldTail fails deterministically on rc3 (0/1, 1.3 seconds): no reset was admitted, yet large-delta compaction retained all speculative suffix positions and needsCompaction remained true. Fail-first test checkpoint is remote bede4de. Full trace /tmp/btcpu-no-reset-before.log.
INFER: Exact mechanism is the artificial compactionRetentionFloor=latestSequence in 0725bc3. The consumer configures a 64 MiB fixed budget/256 MiB reader ceiling, so the 4500-row x 12 KiB burst all fits that speculative reader suffix and protectedSuffixAcross retains its decoded payloads, exhausting 128 MiB. The corrected design samples actual reusable reset targets together with every delta cut under the journal lock; a reset created after that cut targets the cut or later, so it needs only the later residual. No hypothetical reader is retained. Idle advertised snapshots still remain represented, preserving the long-collection starvation fix.
OBSERVED: Revised implementation is local 88d0d80, targeted five-scenario validation running in session 42546. Prepared next unused candidate version rc4, not published; this is a concrete replacement plan because rc3 cannot be overwritten. No second publication is authorized or attempted. Consumer root-cause and module reproducer are now established before the revised fix. Repeated lease-renewal stdout was omitted from the earlier copied failure body, retaining its actual error, stack trace and source log path.
INFER lessons: (1) Capture reader promises at the same boundary as the data cut instead of reserving data for hypothetical readers. (2) A count-bounded cache still needs a byte bound. (3) An immutable candidate may pass its module suite and fail a real consumer path; retain the consumer gate and never overwrite that candidate. Next: prove revised module, build replacement, verify consumer directly with local replacement classpath, push checkpoints and request only the extra publication authorization that is actually needed.
2026-10-17T08:02:05.221813+00:00 — OBSERVED: Revised actual-reset/delta-cut design PASS 5/5 targeted scenarios: no-reset large refresh (failed before), long-collection reset progress, reset work budget, late-reset exact unread suffix, and in-bound staging overlap. Log /tmp/btcpu-reset-cut-after.log, XML /tmp/btcpu-reset-cut-after.xml. Current module commit 88d0d80 prepares rc4 as an unpublished replacement; local build and final full suite are underway.
OBSERVED: Started a single consumer regression repeat against unchanged published rc3, to capture its real child classpath and independently validate a local rc4 artifact without publishing it. Its original -Xmx128m/SerialGC/ExitOnOOM arguments and 4500 x 12 KiB fixture are unchanged. An owned, capped read-only /proc capture watches only the exact child main with this checkout in its classpath, saving /tmp/btcpu-consumer-child-cp.json. This is local integration verification, not an additional Maven publication or a consumer workaround.
INFER: Final authorization question can be concrete after module full validation and this identical child path pass against local replacement: authorize publishing unused rc4 and moving the consumer from known-bad rc3. Existing rc3 will remain immutable.
2026-10-17T08:04:58.441120+00:00 — OBSERVED: Revised module final full local suite PASS 78/78, 0 failed, at exact head 88d0d806c5a01d6ed75a3e547a9574f4bfa1db10; XML /tmp/btcpu-module-revised-final.xml independently confirms 78 successes. Repeated full suite is justified by the newly discovered consumer regression and subsequent implementation/test/version change; the initial 77/77 applies only to the earlier published rc3 head. Revised head push underway.
OBSERVED: The unchanged consumer large-delta child, with the captured exact runtime dependencies and only the journal JAR replaced by local rc4, now prints OK largeDeltaRuns=4500 notesBytes=12288 handlerCalls=512 handlerCallsDuringCompaction=512 maxHandlerMs=7. All original 128 MiB/SerialGC settings and fixture counts are preserved. Class-load diagnostic confirms ProjectionChangeJournal loaded from /tmp/btcpu-rc4-artifact/buildtest-projection-journal-0.0.14-cpuamortize2-rc4.jar. Against published rc3 the identical existing test failed three times with real OOM. This is a local cross-repository integration result, not a served-rc4 consumer test claim.
OBSERVED: Local unpublished replacement rc4 JAR SHA-256 3a561d8d44952ba24ed22c0398373b8101532328338f3c0e9eee80231892dcb5, POM 99a521b58eda8c19eabbdf2b13be1110632d347f7aeb9fd0d90600262adda810. No second publication attempted. Next action that needs new authority: publish unused rc4 through the documented publisher, byte-verify it, move all 27 consumer pins rc3 -> rc4, rerun the 23 targets, and push/update both PRs.
2026-10-17T08:06:30.218613+00:00 — OBSERVED: Both checkpoint heads are now on their named PR branches: module 88d0d806c5a01d6ed75a3e547a9574f4bfa1db10; consumer a2285da0d4b28c596eb8c18bcd28f3c442880eb8. Both descriptions successfully REST PATCHed and verified OPEN at those heads. They plainly mark published rc3 as not ready (consumer 22/23) and corrected rc4 as unpublished, with revised module 78/78 and local consumer child pass. No failing result is represented as green.
OBSERVED: New publication approval question was sent through the asynchronous user-input tool: authorize one unused rc4 publication and finish the consumer repin, or stop after checkpointing. This question is required by the brief’s explicit single rc3 publication grant, already consumed; no response has been received or assumed. Both rc4 proposed served POM and JAR currently return 404. All authorized implementation and local proof needed to review that next step is concrete and pushed.