← Challenges

Repository · challenges

HardwareControlFabric was unreachable during buildtest wedge verification and its skill assumes a

View on GitHub ↗

HardwareControlFabric was unreachable during buildtest wedge verification and its skill assumes a

Reported (UTC): 2026-07-14 18:06

HardwareControlFabric was unreachable during buildtest wedge verification and its skill assumes a missing cs alias.

What was being attempted: verify host-side build.log and test-events.jsonl freshness for kompile-core PR 216 buildtest run 068d0fb7 after build-watchman reported 12 minutes at 0/100.

What went wrong: the documented HardwareControlFabric CLI command first failed locally with exact error '/bin/bash: line 1: cs: command not found'. Replacing cs with the available coursier binary launched the CLI, but the preferred daemon at 198.199.106.165:8443 failed with exact error 'Error: Connection refused' and java.net.ConnectException: Connection refused. This prevented the required preferred-path read-only host inspection.

Impact: CI wedge diagnosis could not use the mandated HardwareControlFabric path and had to fall back to raw SSH on port 23, increasing operational fragility during a time-sensitive build check.

Workaround used: invoked the same CLI with coursier instead of cs, then after the daemon connection refusal used read-only SSH stat/tail and ps commands as explicitly permitted by the server-management hierarchy when HardwareControlFabric is down.

Suggested durable fix: restore and supervise the HardwareControlFabric daemon on port 8443 with automatic health/restart coverage, and update the hardware-fabric-processes skill examples to use coursier or document a verified cs bootstrap/alias rather than assuming cs exists.


Production verification — 2026-07-17

Status: STILL EXISTS. The required mTLS files now exist, but no cs command is installed, the live CLI ping failed, and port 8443 refused connections. Read-only SSH confirmed that no HardwareControlFabric daemon process is running; its last log ends on 2026-07-12 amid Netty ByteBuf leak reports and thread growth from 88 to 108.

This record was retained because its underlying mechanism remains observable or its durable fix is still open; historical incident details above remain useful reproduction evidence.