Remaining: resolve new-main conflict, recheck 28 carried-head failures, full local suite and PR/CI gates. Eight fixture repairs pushed and passed twice; 16 local passes and 1 baseline failure.
Rebased branch wip/1078-rebased-W77 @27d26067 is green on its gates and offered on PR 1078; waiting for the PR owner to incorporate it (supersede with a separate PR if no response by 17:00 UTC)
RE-VERIFY — 2026-10-02 19:39:13 UTC: this is a write-time snapshot. Re-check PR state/checks with gh pr view and branch SHA with git ls-remote. This update uses a fresh JSON get and carries forward the full existing Mission, findings chain, all branch/PR rows, and other lanes' content below. Earlier sections are historical snapshots.
Remaining first: resolve the latest-main rebase conflict, then re-check the other 28 failures from the carried-head CI run. Main advanced to f3efa847c6e78233cd006f42528bd6534a578c26 during the final fetch. Rebase failed on carried commit e7b1f28a3 in tests/projectionChangeFeedFollowUpCompactionSettlesAndResetsStrandedCursor.kts. Conflict evidence is pushed; rebase was aborted to preserve the tested tree. No local pass exists against this new main, and its changes may alter the old-run failure classes.
| Branch / PR | Verified SHA | State |
|---|---|---|
| https://github.com/CodexCoder21Organization/BuildTestEmbedded/tree/wip/bte1200-fix-2026-10-02 | e4433045ea51bd60d01ebd930e842fa9b8072df6 |
Pushed and ls-remote matched. Contains eight tested fixture repairs, all four shard logs, exact36-message classification, local logs, inventory/review and final rebase conflict evidence. Source/test/build tree matches twice-passing b12bf0f66f9011884da31c59622fa6543f2e26fc. |
| https://github.com/CodexCoder21Organization/BuildTestEmbedded/commit/b2f7041a05b40d2c82a9bdfee5f38988c67f3e7a | b2f7041a05b40d2c82a9bdfee5f38988c67f3e7a |
Eight fixture-only repairs; useful to apply separately after understanding the integration conflict. Assertions/counts/deadlines/limits unchanged. |
| https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1200 | dc8e3a54fe003ba6b9f3021511efa5360a850fa9 |
Still OPEN/DRAFT at carried head. Four shard checks and bld-build FAIL. No newer commits/comments from another session at exit verification; PR branch untouched. |
CI https://github.com/CodexCoder21Organization/BuildTestEmbedded/actions/runs/37033820110: 3320 PASS / 36 FAIL, all four shard logs read (832/8, 828/10, 829/11, 831/7). Base main CI https://github.com/CodexCoder21Organization/BuildTestEmbedded/actions/runs/37029468742 is SUCCESS at e04680a0429b5dad204adaa966fff19f7eedcd33. No local main comparison was repeated, per explicit lane rule.
Local testProjectionReaderRetentionIsBoundedByRecordCeiling first failed with the same CI precondition: expected tail cursor t, got baseline b (0 PASS / 1 FAIL, 19.3 seconds). Equal 1 ms lease renewals do not create rows under W77's 30-second observation contract. The fixture history setup was wrong; retention assertions remain correct.
Eight tail-cursor setups now use a 30-second primer, advance once per four-source rotation, and drain each renewal through public latestProjectionCursor. Thirty writes use 240 seconds, forty use 300 seconds, eight use 60 seconds, below the unchanged 600-second reader liveness window. Compaction preserves published observation times. Assertion text/arguments, counts, deadlines, ceilings, explicit idle expiration and cleanup are unchanged. Root and independent delegate review found no gaps.
First repaired group: 8 PASS / 0 FAIL. Required post-fetch/rebase pre-push repeat: 8 PASS / 0 FAIL. Cumulative local attempts: 16 PASS / 1 FAIL. XML confirmed all eight distinct selectors; names and timings are in lane-evidence/bte1200-fix/bte1200-fix-tail-group-verdict.json, alongside full logs/findings/inventory. These passes preceded the later main advance. Full local suite was not run.
New work ended at the 75-minute boundary. No active build and no new remote/CI submission. Result: INCOMPLETE, with eight useful repairs and proof pushed. Local findings: /tmp/claude-501/-code/4719c767-3557-4cb1-90f8-28975c4c3602/scratchpad/out/bte1200-fix-findings.md. Handoff: url://handoff/handoffs/hf-2026-09-24-finish-s1078e-projection-close-restart-and-regression-gates via https://www.handoff.wasmserver.com.
RE-VERIFY: 2026-10-02 18:30:10 UTC. https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1200 remains OPEN/DRAFT at dc8e3a54fe003ba6b9f3021511efa5360a850fa9, all five checks failed. No newer commit/comment indicates another session editing the PR. The repair lane has made no product/test edit or PR-head push.
| Repo | Branch | Remote head SHA | PR | What is on it | State |
|---|---|---|---|---|---|
| CodexCoder21Organization/BuildTestEmbedded | https://github.com/CodexCoder21Organization/BuildTestEmbedded/tree/wip/bte1200-fix-2026-10-02 | 8c35504c9c7bbad193cf1933f12f15ece5ae377c | no new PR; repairs target https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1200 | Complete 36-test failure list, exact messages grouped, full four job logs, findings in lane-evidence/bte1200-fix/; source/test/buildscripts identical to PR head after README-only main rebase | Evidence checkpoint; no local test verdict yet |
OBSERVED: Extracted exactly 36 FAILED records from direct GitHub job logs API after gh run view --log-failed returned empty output. Groups: 8 expected-tail/got-baseline cursor preconditions, 5 generic assertTrue failures, 4 missing new-baseline-after-40-writes, 4 30-second test deadlines at compaction setup, 3 unchanged reset-snapshot cursors, 3 residual/publication hook failures, and 9 distinct messages. Exact messages and full stacks are durable on the linked branch. Main at e04680a0 had passing CI; this lane will not repeat local main compilation.
INFER: Largest group's renewal primers advance ManualClock 1 ms while README.md:74 requires equal payloads to remain rowless below the 30-second observation boundary. This is a source-supported candidate explanation, awaiting public-test local reproduction. Selected testProjectionReaderRetentionIsBoundedByRecordCeiling via supplied withlock.sh; waiting for compile slot since 18:24:50 UTC. No timeout/iteration/assertion changes. Mechanical read-only inventory delegate is collecting source locations; root diagnosis remains in this lane.
Remaining: obtain local largest-group failure, prove and fix every group according to original design, run full local suite, update/ready the PR only after verified repair, push one CI head and watch all four shards plus bld-build. No merge, enqueue, deploy, publish, completion or CI retry by this lane. Prior history and mission follow intact.
RE-VERIFY: 2026-10-02 17:15:53 UTC. PR https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1200 is OPEN/DRAFT at dc8e3a54fe003ba6b9f3021511efa5360a850fa9. The only CI workflow https://github.com/CodexCoder21Organization/BuildTestEmbedded/actions/runs/37033820110 completed FAILED: all four shards failed, and required bld-build failed after they finished. bld-build was absent earlier because its needs dependency waits for all shards, not because dispatch was missing. The aggregate explicitly requires successful shards.
| Work | Branch / PR | Verified state |
|---|---|---|
| Review evidence | https://github.com/CodexCoder21Organization/BuildTestEmbedded/tree/wip/rev-bte1200-2026-10-02 | ddd5b0a48c1c75c770ffe160b27d9c9ea3f8b13b; findings, changed-test manifest, workflow jobs, full four-shard logs/stacks under review-evidence/; source identical to PR head |
| Replacement under review | https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1200 | Source unchanged by review lane; draft hold remains |
OBSERVED: CI executed 3356 test cases: 3320 PASS, 36 FAIL (shards: 832/840, 828/838, 829/840, 831/838 successful). Full assertion texts are in review findings and full stacks in logs. Multiple fixtures still require equal below-deadline lease renewals to create rows. ChangedRunInRenewalBatch also failed expected 1 versus actual 2. Four shard-3 selectors are running locally under slot 2; preparation advanced from workspace scan to Kotlin compilation. No local verdict yet. No retry, remote run, PR source edit, undraft, merge, enqueue, deployment, publication or completion.
INFER: Initial review-lane CI-green premise is refuted. Remains: compare shard-3 failures locally on PR/main, finish recovery-index proof/refutation and six close/two rowless gates, complete fail-first and three-repeat changed-test review, prove any corrections locally, then require new green CI before undrafting. Existing mission and history follow unchanged.
RE-VERIFY: 2026-10-02 16:30:28 UTC. The new draft PR's only GitHub workflow run is https://github.com/CodexCoder21Organization/BuildTestEmbedded/actions/runs/37033820110, IN_PROGRESS at dc8e3a54fe003ba6b9f3021511efa5360a850fa9. The lane's build-watchman JVM was stopped for exit; the workflow itself was not canceled or rerequested. Resume watching this same run/PR after resolving the recorded review hold. Working tree is clean, and the pushed branch remains the verified checkpoint below. The final report was uploaded RUNNING; the handoff is open.
RE-VERIFY: 2026-10-02 16:27:35 UTC. This is a write-time snapshot. Re-read the draft PR, its head, checks, and branch before resuming. No merge, enqueue, deployment, artifact publication, old-PR closure, or handoff completion occurred. This checkpoint supersedes the older incorporation instructions below; all historical mission, findings, and branch rows are retained.
| Work | Verified state |
|---|---|
| New main-targeted replacement https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1200 | DRAFT at dc8e3a54fe003ba6b9f3021511efa5360a850fa9, based on current main e04680a0429b5dad204adaa966fff19f7eedcd33. It explicitly supersedes https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1078 and links this handoff and the dependent handoff. Not ready for final review. |
| Durable branch https://github.com/CodexCoder21Organization/BuildTestEmbedded/tree/wip/bte-w77-2026-10-02 | git ls-remote verified dc8e3a54fe003ba6b9f3021511efa5360a850fa9, matching local HEAD. Fresh rebase carries 53 projection commits plus one recovery watermark correction. No tests are deleted relative to main. |
| Original PR https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1078 | Still OPEN at 1db31ca20484f2c1766fc8293de5ec624cb4bb30. Last commit September 27 05:15 UTC; last comment September 27 14:06 UTC. No recent actor or modification by this lane. Supervisor should decide closure after replacement review. |
| Local core gates | Final correction batch 4/4 PASS; post-fetch/rebase pre-push repeat 4/4 PASS on the same exact head. Eight successful executions of four unique public selectors; zero runtime failures. Earlier batch: four preparation failures at 0.0 seconds because the rebase omitted a required helper argument. First attempt: compile-lock wait expired with zero tests executed. |
| One CI watch | build-watchman 0.0.21 started for the new draft PR. Four GitHub test shards pending; required bld-build not reported yet. No buildtest run URL at the initial poll. The installed kotlin-build-ci-test app has queued zero-run suites on mature sampled main commits too; watchman classified it as not dispatching, not a lost dispatch, and did not rerequest anything. Continue only this CI run. |
The final four public selectors are projectionChangeFeedTaggedTailAtBaselineBoundaryContinues, projectionChangeFeedOldUnmarkedBaselineCursorResetsExactlyOnce, projectionBuildRuleDigestSurvivesCompactionAndRestart, and projectionUnchangedLeaseRenewalDoesNotRepeatRunUpsert. Both final batches used --local under the shared compile lock. No full local suite ran and no test deadline, iteration count, or assertion was relaxed.
Rebase resolution retained current main's startup-fold/reset behavior, source-loss records, unavailable-journal authority, reader retention, interrupted publication handling, repair-wake cancellation, and service resource release. Main's obsolete startup-wait fixture was already replaced on main by projectionListenerResetIsServedDuringStartupReconciliation; that newer contract remains. The first local compile exposed a missed new-main call: rebuildUnavailableState stages a fingerprint sidecar, while W77 requires its sequence watermark. The correction supplies replacementSequence, the watermark of the staged replacement journal, and every writeFingerprintsTemporary call was enumerated.
Review state: two independent passes were performed, but the later compile finding supersedes the initial no-issue conclusion. There is also an unproven recovery-index concern: rebuildUnavailableState replaces records/fingerprints without rebuilding payloadStates/payloadKeysByRunId, unlike other full-replacement paths. A candidate first-renewal guard was rejected before execution because a baseline with no observedAt legitimately permits a first observation. No index fix is guessed. A valid experiment should drive an equal later build-rule payload through the public runner path after its payload changed in a recovery baseline, or refute the concern with evidence. The rejected uncommitted draft is preserved locally outside the repository in out/bte-w77-recovery-payload-guard-rejected-draft.kts; no existing test was removed or skipped.
Remaining, in order: resolve or refute the recovery-index review question; run the six close/callback guards and rowless-compaction/restart neighbors on the final current-main head; address any locally reproduced failure; review the full rebased diff; update/undraft the replacement after the gates are satisfied; require green CI on the replacement. Only the supervisor decides merge and closure of the old PR. Keep this handoff open.
Lane evidence: out/bte-w77-findings.md, core-corrected.log/XML (complete compile failure), core-fixed.log/XML (4 PASS), prepush.log/XML (4 PASS), focused-gates.txt (19 verified selectors), watch.log, and three owned JVM thread dumps. These paths are local supervisor scratchpad evidence; the code checkpoint is durable in the branch linked above. Documents read: architecture/TESTING.md, PHILOSOPHY.md, PlanRepository/handoffs/README.md, both handoffs, repository README context, and the github-repos/create-handoff/status-report/watch-build skills. The lane's 75-minute stop prevents additional new work; unfinished gates are explicit.
The four selected tests failed preparation at 0.0 seconds; no test body ran. Complete command output and repeated wrapper stacks are in bte-w77-core-corrected.log and XML.
src/buildtest/embedded/ProjectionChangeJournal.kt:1228:56: error: no value passed for parameter 'sidecarLatestSequence'
java.lang.Error: Compile failure: Non-zero exit code: COMPILATION_ERROR
at build.kotlin.jvm.JvmBuildRulesKt$BuildKotlin$1.invoke(JvmBuildRules.kt:1810)
at build.kotlin.jvm.JvmBuildRulesKt$BuildKotlin$1.invoke(JvmBuildRules.kt:1755)
at build.kotlin.jvm.JvmBuildRulesKt$inlined$sam$i$java_util_concurrent_Callable$0.call(PrivilegedBuildAssistant.kt)
at kompile.executionenvironment.BuildscriptRunner.simpleCache(BuildscriptRunner.java:216)
at build.kotlin.jvm.JvmBuildRulesKt.BuildKotlin(JvmBuildRules.kt:2110)
at build.kotlin.jvm.JvmBuildRulesKt.BuildKotlinJar(JvmBuildRules.kt:1148)
at build.kotlin.jvm.JvmBuildRulesKt$buildSimpleKotlinMavenArtifact2$1.invoke(JvmBuildRules.kt:1505)
at build.kotlin.jvm.JvmBuildRulesKt$buildSimpleKotlinMavenArtifact2$1.invoke(JvmBuildRules.kt:1503)
at build.kotlin.jvm.JvmBuildRulesKt$inlined$sam$i$java_util_concurrent_Callable$0.call(PrivilegedBuildAssistant.kt)
at kompile.executionenvironment.BuildscriptRunner.simpleCache(BuildscriptRunner.java:216)
at build.kotlin.jvm.JvmBuildRulesKt.buildSimpleKotlinMavenArtifact2(JvmBuildRules.kt:2087)
at build.kotlin.jvm.JvmBuildRulesKt.buildSimpleKotlinMavenArtifact2$default(JvmBuildRules.kt:1492)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at kompile.executionenvironment.BuildscriptRunner.invokeInterceptedLocally(BuildscriptRunner.java:261)
at kompile.executionenvironment.BuildscriptRunner.visitMethodInsn(BuildscriptRunner.java:255)
at tools.kotlin.build.runtime.PrivilegedBuildAssistant.visitMethodInsn(PrivilegedBuildAssistant.kt:20)
at buildtest.embedded.Buildtest_embedded_buildktsKt.buildMaven$impl(buildtest_embedded_buildkts.kt:272)
at buildtest.embedded.Buildtest_embedded_buildktsKt.buildMaven(buildtest_embedded_buildkts.kt)
at CmdKt.command(cmd.kt:1)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at kompile.executionenvironment.BuildscriptRunner.main(BuildscriptRunner.java:509)
This section supersedes earlier snapshots. No merge, enqueue, deploy, publication, or push to the original PR branch occurred. The handoff remains open because the original PR has not incorporated the rebased branch and its required CI has not run there.
| Work | Verified state |
|---|---|
| Original PR https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1078 | OPEN/CLEAN at 1db31ca20484f2c1766fc8293de5ec624cb4bb30. Five checks on that older head are SUCCESS. The author last commented 2026-09-27 11:02 UTC. |
| Rebased branch https://github.com/CodexCoder21Organization/BuildTestEmbedded/tree/wip/1078-rebased-W77 | 27d260677c813dec15f9d04c564ba15ca8c6b5b5, based exactly on main 0eb3f0e1afc5b2b53f1f098d359c8dffe5fed320. Its final commit changes only two ManualClock steps from 1 ms to the existing 30,000 ms observation interval in the tagged-boundary fixture. |
| Targeted remote run https://buildtest.kotlin.build/run?id=239df875 | COMPLETED: 3 selected PASSED, 0 FAILED, 2,796 FILTERED_OUT. Read-only archive hashes matched branch 27d26067 for all three selected tests and both main projection source files, establishing that the passing run included the corrected fixture and rebased implementation. |
| Additional W80 run https://buildtest.kotlin.build/run?id=d42aa5e6 | Submitted from branch 27d26067 at 13:56 UTC before archive provenance was recovered; CANCELED at 14:21 UTC before any selected test executed. The local client exited 137 after sleep interrupted; the service recorded kompile-cli process interrupted. No rerun is needed to interpret the proven 239df875 result. |
| Coordination comment https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1078#issuecomment-5856559623 | Posted once, offering the rebased branch to the original author with the fixture mechanism, exact counts, and archive provenance. |
The mechanism of the fixture failure was unchanged renewals advancing ManualClock by 1 ms, before the 30-second observation boundary at which this projection intentionally emits a row. W77's original local batch on the pre-correction head had 6 passes, 1 tagged-boundary fixture failure, and 1 selector that did not execute after a build-rule compile deadline. The corrected fixture and the old unmarked boundary and build-rule digest tests all passed in run 239df875. W80 independently completed test-comprehensiveness and adversarial review of final head 27d26067; no further finding arose. See W80/findings.md for details.
Remaining: the original author may take the separate rebased branch, or the orchestrator may choose a main-targeted superseding PR if coordination stalls. Require all current PR checks on the incorporated head, including kotlin.build (remote) if configured. Only the orchestrator enqueues or merges. Do not push over the active author's branch and do not deploy.
This section supersedes the older snapshots below. No merge, enqueue, publication, or deployment occurred.
| Work | Verified state |
|---|---|
| Cursor PR https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1111 | MERGED 2026-09-27 11:25 UTC; main contains tagged-tail cursor support. |
| Projection PR https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1078 | OPEN/CLEAN at 1db31ca20484f2c1766fc8293de5ec624cb4bb30 with five successful checks on that old head. Author commented at 11:02 UTC; W77 did not alter this branch. |
| Rebased W77 branch https://github.com/CodexCoder21Organization/BuildTestEmbedded/tree/wip/1078-rebased-W77 | 27d260677c813dec15f9d04c564ba15ca8c6b5b5, based on main 0eb3f0e1afc5b2b53f1f098d359c8dffe5fed320. The 53 original commits rebased cleanly; W77 added one test-fixture commit. This is WIP, not a green PR. |
Mechanism and evidence: the local eight-selector batch on the first rebased head completed 6 passes, one real test failure, and one test that did not execute after the local build rule reached its existing 20-minute compile deadline. The failing test was projectionChangeFeedTaggedTailAtBaselineBoundaryContinues: three unchanged lease renewals moved ManualClock by 1 ms each and could not publish the rows the fixture required after PR 1078's intended row suppression. The failure was at its unchanged compaction latch. W77 changed only those two clock advances to the public 30,000 ms observation interval at 27d260677; no assertion, wait, or iteration count changed. The full error output is in W77/projection-batch.xml, and the exact prior-head results are six PASS, tagged-boundary FAIL, old-unmarked-boundary NOT EXECUTED due build-rule deadline. Both test-comprehensiveness and adversarial review passes were recorded in W77/findings.md for the rebased and final heads.
A remote three-selector run was started after that fixture correction using scripts/test.bash --remote with tagged boundary, old unmarked boundary, and build-rule digest selectors. It connected to url://buildtest/ and reached BUILDING by 13:39 UTC, but had returned no run ID or selected result by 13:42 UTC; its local log is W77/cursor-pair-remote.out and result target is W77/cursor-pair-remote.xml. Do not claim it passed. The original PR's old-head green CI does not certify the rebased W77 branch.
Remaining, in order: (1) obtain the remote targeted result, then diagnose any selected failure at its cause; (2) rerun the focused integration gate on the final head if the remote attempt never accepted a run; (3) after a passing gate, post ONE comment on the actively owned PR linking the separate rebased branch and exact test counts, offering it to the owner; (4) coordinate incorporation and require full required CI on the resulting PR head. If the owner becomes inactive for more than 24 hours, a main-targeted superseding PR from the W77 branch is permissible after a fresh rebase, final tests, and why-first description. Orchestrator alone decides enqueue or merge. Do not force-push the original owner's branch or deploy.
Cursor PR https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1111 is now at head e9a4b07eb7a03426cf276db2a7683dce484c217c on remote branch https://github.com/CodexCoder21Organization/BuildTestEmbedded/tree/wip/W64-tail-cursor-provenance. Its first CI shard on prior head 3a7f2d9a6 executed 686 tests: 685 passed and one fixture failed at tests/projectionConcurrentResetReadersStayIndependent.kts:72 because it parsed tagged authority:t:18 with substringAfter(':').toLong(). Full job: https://github.com/CodexCoder21Organization/BuildTestEmbedded/actions/runs/36303840179/job/108576479866. Commit e9a4b07eb changes the fixture to substringAfterLast(':').toLong(); its exact public selector passed locally 1/1 with no assertion weakened. Main remained 5125d3ba7 at the immediate pre-push rebase. The PR body records this result. The new-head CI verdict is pending; do not treat prior-head shard failure as a current-head verdict.
Remaining: watch all required checks for the new head to green, diagnose any new failure locally before changing code, then orchestrator reviews the cursor PR and decides merge order. After it lands, rebase projection PR https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1078 and rerun its final gate. No merge, enqueue, or deploy was done in W64.
This section supersedes the older W55 snapshot below. The user asked for the saved-cursor fix to be a separate main-targeted PR; W64 did not edit the concurrently owned projection PR branch.
| Work | Live state |
|---|---|
| Cursor PR https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1111 | OPEN, head 3a7f2d9a6ee27eb3572801808e66aa6a5ebf945b, base main 5125d3ba791a2790133f35b765d7d38aeab57f5b; four CI shards pending with no buildtest run URL at first watchman poll. |
| Cursor branch https://github.com/CodexCoder21Organization/BuildTestEmbedded/tree/wip/W64-tail-cursor-provenance | Same remote head 3a7f2d9a6. Source/README plus exact tagged, legacy and gap guards. |
| Projection PR https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1078 | OPEN/CLEAN at 1db31ca20484f2c1766fc8293de5ec624cb4bb30 with four successful test shards and bld-build; its forced saved-cursor guard failed 3/3 before the cursor source fix. |
| W55 diagnostic branch https://github.com/CodexCoder21Organization/BuildTestEmbedded/tree/wip/W55-rule-cursor-boundary | ffc75a66e63ccf90375e30bad52a0359d1e46e89; 5/5 focused integrated tests pass, including the forced boundary guard. |
On W64's main-based branch, the new tagged-boundary selector passed locally 1/1 and four cursor neighbors passed locally 4/4, each once. Review pass A covered the exact tagged and old unmarked boundary, retained suffix, dropped gap, protected window, and stale history. Review pass B checked cursor parsing, continuation, legacy compatibility, README, and all changed paths; no unresolved finding. The remote targeted attempt https://buildtest.kotlin.build/run?id=ee238a71 stayed PENDING for 30 minutes and executed no selected test before the client deadline. Local tests provide the final-head focused verdict.
Remaining steps: watch https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1111 to required CI green, address any actual test failure at its cause, then let the orchestrator review and decide merge order. After the cursor PR is merged, rebase https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1078 on main and run its final projection gate again. Do not merge, enqueue, or deploy from this worker.
Written 2026-09-27 06:22 UTC. Re-verify live state before acting. This is a checkpoint; it has not been merged, enqueued, or deployed.
W55 found and reproduced an intermittent public guard on https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1078. The PR is OPEN/CLEAN at 1db31ca20484f2c1766fc8293de5ec624cb4bb30. All four shard checks and bld-build on https://github.com/CodexCoder21Organization/BuildTestEmbedded/actions/runs/36296743910 succeeded; CI artifact XML reports 2778/2778 successes and includes all 179 unique evidence-manifest selectors. The build-rule cursor test nevertheless failed in 3 of 5 local runner attempts at that same head. Its CI pass was therefore insufficient to close this handoff.
The mechanism is cursor provenance. Newly issued non-baseline cursors on the PR head are unmarked authority:sequence. After restart compaction folds saved sequence 6 into the baseline and retains sequence 7, the journal conservatively resets an unmarked cursor at the baseline boundary, because a legacy unmarked baseline cursor there must reset. W55 changed the final reopen fixture to force this exact boundary; the unmodified source failed the public test on all 3 attempts. W55 then created a separate WIP branch that marks newly issued tail cursors authority:t:sequence, keeps old unmarked cursors accepted with their boundary reset, and permits a tagged tail to continue only when the next sequence is retained. The first post-fix five-selector run completed 4/5 successful: the forced rule-boundary guard and three neighbors passed; the startup retained-suffix guard expected the old literal token. W55 updated that expected literal in commit 70016e9fefb141147bb647af32d00e3cba898574. The repeat then completed 4/5 and exposed a gap-edge regression: an old unmarked cursor just before the retained suffix waited for reconciliation. Commit ffc75a66e63ccf90375e30bad52a0359d1e46e89 restores that continuation while retaining tagged baseline-boundary behavior. The third exact five-selector run on ffc75a66e exited 0: 5/5 tests successful, each with one execution and no flaky retry line. Its XML is in the W55 lane as cursor-fix-tests3.xml. No edit was made to the concurrently owned PR branch.
| Reference | Head | State |
|---|---|---|
| https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1078 | 1db31ca20484f2c1766fc8293de5ec624cb4bb30 | OPEN/CLEAN, five green checks, current public guard intermittently fails |
| https://github.com/CodexCoder21Organization/BuildTestEmbedded/tree/wip/W55-rule-cursor-boundary | ffc75a66e63ccf90375e30bad52a0359d1e46e89 | W55's failing-before fix test, source/README candidate, and gap-edge correction |
| https://github.com/CodexCoder21Organization/BuildTestEmbedded/tree/wip/F1078-2026-09-26 | 5ad73f9b8b6bf3d78ed78825c675040856fec97f | Older WIP checkpoint |
| https://github.com/CodexCoder21Organization/PlanRepository/tree/evidence/laneF1078-2026-09-26 | bdedf95f4b14c02cbe41c578de0b8431001d0ffd | Historical logs and manifest |
cursor-fix-tests3.xml (5/5, one execution each).Written 2026-09-27 05:39 UTC. RE-VERIFY every state below before acting: query PR https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1078 with GitHub GraphQL for state, head, base, merge status and checks; use git ls-remote for branches, and build-watchman for CI. This is a snapshot.
Finish the unchanged run/build-rule projection row change in https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1078, verify close/restart/compaction and the projection selector gate, and leave the PR green for the orchestrator. Do not merge, enqueue, or deploy.
The handoff's old PR head was stale. A concurrent Codex actor pushed head 1db31ca20484f2c1766fc8293de5ec624cb4bb30 at 05:15 UTC, rebased onto main 5ef4f9d38a6d822a537e49092e02509828418772, updated the body, and started four CI shards. W55 did not edit or push to that branch. The prior wip/F1078-2026-09-26 branch remains at 5ad73f9b8b6bf3d78ed78825c675040856fec97f. Main lacks the unchanged-row feature by content.
The evidence manifest has 60 + 60 + 59 = 179 distinct selectors, not 180. The current PR body has individual selected-pass lines for 91 of these; 88 lack such lines, including 12 changed tests. This does not mean they failed, but it means selected historical results do not certify the final head. At 05:30, 05:35, and 05:40 UTC build-watchman found all four PR shards pending without buildtest URLs; at 05:40 they were within recent runtime medians. Do not treat those polls as a green verdict.
The mechanism behind the build-rule cursor gate is specific: the test saves an unmarked cursor at sequence 6 with one protected recent position. After restart, compaction rewrites the baseline through sequence 6 and rule B is retained at sequence 7. ProjectionChangeJournal.pageInternal resets an unmarked cursor at the retained-prefix boundary. Historical selected run https://buildtest.kotlin.build/run?id=4ae8383b failed at this exact no-reset assertion. The existing public test projectionChangeFeedOldUnmarkedBaselineCursorResetsExactlyOnce requires the reset for legacy unmarked baseline cursors. A blanket change that lets unmarked boundary cursors continue would risk skipping baseline state. A distinct tail-cursor format could distinguish the two cases, but needs compatibility and stale-cursor tests; no such code was written here. Alternatively, if the existing retention contract governs, the build-rule gate's no-reset expectation needs an explicit correction. Do not silently weaken it.
W55 reviewed the final-at-write-time source diff and test coverage. It found the cursor-gate conflict above; no other proven code defect. The six close/callback guards and broad final-head gate still require execution. The source diff passed git diff --check. No code changes, publication, production action, merge, or enqueue occurred in this lane.
| Ref | Remote head | State |
|---|---|---|
| https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1078 | 1db31ca20484f2c1766fc8293de5ec624cb4bb30 | OPEN, four shards pending at 05:40 UTC; concurrently owned branch |
| https://github.com/CodexCoder21Organization/BuildTestEmbedded/tree/wip/F1078-2026-09-26 | 5ad73f9b8b6bf3d78ed78825c675040856fec97f | Older WIP checkpoint |
| https://github.com/CodexCoder21Organization/PlanRepository/tree/evidence/laneF1078-2026-09-26 | bdedf95f4b14c02cbe41c578de0b8431001d0ffd | Earlier logs and handoff evidence |
Written 2026-09-26, 22:51 UTC. RE-VERIFY all state before acting: use gh pr view 1078 --repo CodexCoder21Organization/BuildTestEmbedded --json state,headRefOid,mergeStateStatus,statusCheckRollup, git ls-remote, and direct build-client results. Public build run lists can remain PENDING after the execution client has reported failure.
Finish https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1078: suppress unchanged run/build-rule projection rows, finish round-4 restart/close/compaction guards, rebase onto main, verify the complete projection gate, then update the PR branch and start its four-shard CI. This is part of the user's goal to "solve the underlying root cause of any projection update slowness." The inherited investigation reports approximately 107 journal records per second, with unchanged run rows contributing 17% of bytes and increasing compaction frequency. This lane did not re-measure production.
No merge, enqueue, deploy, test heap/deadline/iteration changes, weakened assertions, or changes to sibling branches are authorized. The companion https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1079 is another lane's work on the same files. Re-verify main and preserve its tests when rebasing.
Update 23:19 UTC: ce1ffaa2 and 08095f60 both completed 4/4 PASSED on 5ad73f9b8. Corrected ContinuousMaintenance19,922ms, deletion reset4,323ms, startup gap6,505ms, original rule cursor6,697ms; unchanged renewal6,660ms, changed-row batch6,957ms, lazy-open close16,549ms, concurrent close/flush6,820ms. Full 60-selector projection gate batches 0 and 1 are accepted as f3dd5960 and 4c6da8f6 (mapping still to verify), both queued. Batch2 and 25 extra close/scheduling/main regressions have not been submitted yet, respecting the two-client limit. Local full projection gate remains unrun after the prohibited-to-retry cgroup kill. No PR branch/body update or new PR CI.
Update 23:12 UTC: core run https://buildtest.kotlin.build/run?id=f39fca6c is COMPLETED, 4/4 PASSED on f93340756: unchanged 128 MiB child84,050ms, original rule cursor6,921ms, process-stop10,968ms, collection-window7,410ms. Neighbor run https://buildtest.kotlin.build/run?id=5e08db2b passed ConcurrentReaders (17,272ms), but failed ContinuousMaintenance's staging latch plus the two main startup-gap/deletion-reset fixtures. Their source stimuli still relied on rowful unchanged renewals or on deletion padding that now grows the baseline. Commit https://github.com/CodexCoder21Organization/BuildTestEmbedded/commit/5ad73f9b8b6bf3d78ed78825c675040856fec97f corrects only those three fixtures to use due public observations, draining seed rows. No assertion, writer count, payload or wait bound changed. Verification ce1ffaa2 covers those three plus the original cursor guard; 08095f60 covers I-A and two close guards. Both BUILDING at this snapshot. The full 180-selector gate is not green or complete. Prepared PR body is a draft only; the PR head/body remain unchanged.
Handoff access recovered and revealed the later O1078b work missing from the supplied brief. Its original body is preserved at the end and in the evidence branch. Read it for its broader gate evidence.
The lane's latest pushed head is 5ad73f9b8b6bf3d78ed78825c675040856fec97f, based on main 3ce65cb66c8f6c641173d4494d187ddea00c87a4. It incorporates three later commits after independently verifying their source and historical failing/passing results:
All three imported commits applied without conflicts and carry the required Fable trailer. Separate pushes after I-D and I-F are durable. Source/test review found no assertion/limit change. Current-head execution is still outstanding.
Scanned every result page, with page-number coverage checked, from 7e7be0f2, 1f31fd15, and 0ae038e7. The unchanged 128 MiB / 4500-record heap guard PASSED in 7e7be0f2 (249,199 ms). This historical pass refutes a deterministic inability of that child to fit its heap; it does not turn the current local compile kill into a child result.
Do not import O1078b's 516d4dea4 rule-test change: it raises retention on final reopen from 2 to 64 to avoid the compaction, while the current user expressly requires fixing the code and keeping the cursor assertion. The original guard is unchanged on this lane. The saved cursor :6 becomes the end of a rewritten baseline (run/A/B at 4-6, B retained at 7), so the journal's unmarked-boundary check resets it. This still requires a product fix satisfying the protected-window and old-unmarked-baseline contracts; coordinate through main with the sibling effort, never edit its branches.
Additional inherited gate reds remain: projectionChangeFeedConcurrentReadersDoNotSeeReleasedStagedRun and projectionChangeFeedReadRemainsAvailableDuringContinuousMaintenance. They timed out twice on O1078b; the former was faster on main, the latter was marginal there. No guessed fix or relaxed limit has been applied.
At 22:47 UTC, after observing newly allocated runs reaching BUILDING following the earlier provider failures, submitted two batches from f93340756 (fetched/rebased immediately before submission): core I-E/I-B/I-D/I-F, and the two inherited gate failures plus main's startup-retention/deletion-tombstone tests. Runs f39fca6c and 5e08db2b were queued; their mapping to the batches and final verdicts must be recovered from the clients/results. Logs fixed-core-remote.log and fixed-neighbors-remote.log; watchers f39fca6c-watch.log and 5e08db2b-watch.log. These submissions have changed code and fresh allocation evidence; neither is a blind rerun. Do not duplicate them.
All rebased-head product-test verdicts remain unverified. I-D, I-E and I-F have verified historical passes, with I-D/I-F fixes now imported as stated above. These are exact existing public test selectors, prefixed with buildtest.embedded.
| Item | Guard | State |
|---|---|---|
| I-A unchanged run / changed row exactly once | projectionUnchangedLeaseRenewalDoesNotRepeatRunUpsert; projectionChangedRunInRenewalBatchEmitsOnce; projectionUnchangedLeaseRenewalStaysSilentAfterRestart | Not run on rebased head |
| I-B rules and saved cursor across restart | projectionBuildRuleDigestSurvivesCompactionAndRestart | Historical failure in https://buildtest.kotlin.build/run?id=4ae8383b; rebased execution blocked before tests |
| I-C terminal close/atomic flush | projectionCloseWaitsForInFlightJournalOpen; projectionJournalOpenCallbackCannotCloseService; projectionConcurrentCloseFlushesRowlessFingerprintOnce; remaining close guards in manifest | Inherited history has partial passes; all 17 still need current-head confirmation |
| I-D committed rowless renewal survives abrupt stop | projectionSuppressedRenewalProcessStopBeforeFlush | Harness shadowing fix inherited; rebased execution blocked |
| I-E bounded compaction memory | projectionChangeFeedReadRemainsAvailableDuringRenewalCompactionUnderTightHeap | Local compilation killed; remote allocation failed; child never ran |
| I-F collection-to-snapshot window | projectionRowlessRenewalDuringBaselineCollectionSurvivesRestart; projectionRowlessRenewalPreservesStagedCompaction | Existing guards inherited; rebased execution blocked |
The full gate consists of 180 exact selectors in S1078g-gate-0.selectors, -1.selectors, and -2.selectors (60 each). Do not pass more than four local selectors per command. None of the complete gate executed in this lane.
| Repo | Branch | Remote head | PR | Contents / state |
|---|---|---|---|---|
| BuildTestEmbedded | https://github.com/CodexCoder21Organization/BuildTestEmbedded/tree/wip/F1078-2026-09-26 | 5ad73f9b8b6bf3d78ed78825c675040856fec97f | https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1078 (not yet its head) | Rebased I-D/I-F fixes have current core passes; three fixture corrections under verification |
| BuildTestEmbedded | https://github.com/CodexCoder21Organization/BuildTestEmbedded/tree/wip/L2b-duplicate-emissions | 32da2a139e40c76122e40234196dc659f2fb3181 | https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1078 | Unchanged old PR head; all old shards failed |
| PlanRepository | https://github.com/CodexCoder21Organization/PlanRepository/tree/evidence/laneF1078-2026-09-26 | 798213842ab095d0caf41e04bd6998d147e6a8e9 (evidence branch continues to receive checkpoint updates) | No PR | Findings, complete failure logs/stacks, cgroup counters, gate manifests, and this handoff; no merge requested |
No deployment, artifact publication, merge, enqueue, or production service change was performed in this lane. No deployment commands were issued. Existing production versions were not investigated.
Findings and logs: https://github.com/CodexCoder21Organization/PlanRepository/tree/evidence/laneF1078-2026-09-26/handoffs/artifacts/laneF1078 . Full failure stacks are retained there; no assertion/limit was changed.
Local command used: jvm-slot.sh env JAVA_OPTS=-Xmx2g JDK_JAVA_OPTIONS=-Xmx2g scripts/test.bash --local --test buildtest.embedded.projectionChangeFeedReadRemainsAvailableDuringRenewalCompactionUnderTightHeap. scripts/test.bash overwrites JAVA_OPTS, so JDK_JAVA_OPTIONS also caps the parent; explicit child -Xmx128m remains unchanged and last. Remote used the same selector with --remote. Full local suite is forbidden by common.md; four-shard CI is the full repository gate.
Build-watchman reported "100 of 100 tests completed" against a PENDING run list; historical findings show this can count FILTERED_OUT first-page rows. Do not equate it with selected test success. Use complete selected results and direct client failure.
Handoff service initially failed getHandoffFile with EOF after request write; a later health command returned OK. No initial claim/write was attempted. report-challenge's CLI automatically enqueues/merges, so it was not invoked under this lane's no-merge rule; challenges are preserved here for the supervisor.
Written 2026-09-24 09:45 UTC by lane O1078b. RE-VERIFY: this is a snapshot of in-flight work. Read https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1078 with gh pr view --json state,headRefOid,statusCheckRollup. For buildtest runs, read every selected result row through the coordinator (paged getTestResultsPaginated, 500 rows a page); the public /api/test-results pages can still show PENDING/RUNNING for rows of a terminal run.
The user reported stale projections at https://buildtest.kotlin.build/runs, with the banners "Historical run backfill is continuing in the background; stored rows remain available while it catches up." and "The live projection feed is disconnected or has an unapplied gap; this page shows the last committed stored view.", and asked for the underlying cause to be found and fixed with TDD. Original evidence: https://github.com/CodexCoder21Organization/PlanRepository/blob/evidence/PROJRESET-2026-09-23/evidence/PROJRESET-2026-09-23/FINDINGS-orchestrator-2026-09-23.md.
Finish round 4 of https://github.com/CodexCoder21Organization/BuildTestEmbedded/pull/1078 (it stops the coordinator from writing a journal row for every lease renewal of an unchanged run). The invariants I1-I6 and groups A-D are in the S1078e/S1078f briefs on the PlanRepository evidence branch (evidence/PROJRESET-2026-09-23/session-2026-09-24/briefs). Never merge, enqueue, deploy, raise a deadline, reduce a stress bound or weaken an assertion.
wip/L2b-duplicate-emissions is UNCHANGED at 32da2a139 (red). The gate was not met, so nothing was pushed there and the PR body has no round-4 section yet.supplementalFingerprintRevision, and staged-compaction publication rejected any revision change before choosing a delta, so the held 128 MiB stage was discarded and never reached its large-delta refresh. Rowless versions now travel with the finite delta; real source loss still invalidates the stage. Tight-heap child PASSED in 9349e8b7 and 7e7be0f2 (249 s, unchanged 128 MiB / 4,500 records).git rebase origin/main conflicted on the first PR commit (76a15d91b, README.md), and main added projection changes since the PR base (#1089 live readers paging during startup reconciliation, #1092 deletion tombstones in reset baselines). Rebase, resolve, then re-run the round-4 targeted set and all three gate batches at the rebased head (keep each batch small enough that its build plus tests finish inside the client's 30-minute limit).wip/L2b-duplicate-emissions with --force-with-lease, and add a "Review round 4" section to the PR body with the mechanisms above, stating that the two follower-grace tests and dynamicRecoveryAcrossThreeGenerations fail identically on main.No status reports yet.
Moves it out of every priority list and into ArchiveArea.