Repository · challenges
Local full-suite commands are SIGTERM'd when the process stays silent during test execution.
Reported (UTC): 2026-07-16 14:48
Local full-suite commands are SIGTERM'd when the process stays silent during test execution.
What was being attempted: Run the mandatory final BuildTestWui suite (188 tests, including hosted golden screenshots) after a final rebase.
What went wrong: Two normal invocations of 'scripts/test.bash --test .' were externally terminated with exit code 143 (SIGTERM) before the test runner printed a result or stack trace. Redirecting stdout/stderr to /tmp/buildtest-wui-final-tests.log did not help; that silent invocation was also terminated. Host diagnostics at the time showed 11 GiB available memory, 269 GiB free disk, and no test assertion failure in the log.
Impact: A healthy several-minute local suite could not reach its result through the standard command execution path, making successful verification look like an unexplained test failure.
Workaround used: Run the test process in the background while a wrapper prints 'Local full suite is still running' every 20 seconds, then wait for the real PID and tail the test log. With periodic output, the identical final source completed and reported 'ALL TESTS PASSED (188/188 tests completed successfully)'.
Suggested durable fix: Document and expose the command runner's inactivity watchdog, or have it keep tracked processes alive independent of stdout activity. At minimum, return a diagnostic naming the watchdog when it sends SIGTERM instead of only exit code 143.
Production verification — 2026-07-17
Status: STILL EXISTS. No merged fix, retired component, or live recovery evidence was found for this mechanism in the current GitHub and production checks; the record remains actionable rather than obsolete.
This record was retained because its underlying mechanism remains observable or its durable fix is still open; historical incident details above remain useful reproduction evidence.