Resolve the service connection notice before another ContainerNursery main deployment

Written: 2026-10-22T01:12:32.784003+00:00 RE-VERIFY: This is a write-time snapshot. Recheck CN PID, exact proc command, detailed health, routes and app processes through the documented CLIs before further production actions.

Mission summary

The authorized one-time main deployment was performed, and its required restoration was completed after the screenshot check did not pass. Current main is not left deployed. Another deployment requires new owner direction; the original deploy authorization has been used. The owner ruling remains: preserve the exact current JVM arguments, including -Xmx768m; no heap change is authorized.

What was found and done

1. Built current main 45324dc2b27b1d1562887b9023e463ecafc0e32e locally with the documented build command, exit 0, archive integrity passed. Both requested merges are ancestors. New jar SHA-256 ca6ddc710994cf182da19c1b0da38fa9a47397430428564eb90616cba5fd6eb6, size 133961905 bytes. Host hash matched before deployment.
2. Captured previous PID 16622, exact command and previous jar hash. Re-read live arguments immediately before TERM; every argument matched, including -Xmx768m. Stopped CN PID alone through HCF, waited for exit, and started main in the required systemd transient unit with identical JVM arguments, CN_RESTART_HANDOFF=1 and no automatic restart. Main PID was 3511242. No direct HCF CN launch or SSH was used.
3. Main passed detailed CLI health, all 120 identical route records, one CN JVM, no extra app JVMs, correct cgroup and unit settings. Idle TLS waited 35 seconds, returned 0 bytes and closed cleanly; a fresh request returned 200. Home loads 20/20 status 200 and freshness loads 20/20 status 200 on their first attempt, zero HTTP 503 and zero Read timed out bodies. Two additional sites returned 200.
4. The first-frame screenshot did not pass: service connection setup at 29776 ms, readings 2 h 30 m old (2026-10-21 22:37:53 UTC), awaiting-data placeholders. Navigation was 200 and browser page errors were 0. This lane did not establish the cause or whether this CN change introduced the notice.
5. Immediately restored the previous jar with the byte-identical previous command through a separate transient unit. Current PID 3522254. Detailed health true, 120 identical routes, 27 active containers and 26 managed app JVMs in the final complete snapshot; every app JVM matches an active container. PIDs 16622 and 3511242 are absent. Restored home, freshness, kotlin.directory and nursery UI all returned 200. No manual action on app containers was taken.
6. An early restored snapshot had one app not yet listed in active containers; the next complete snapshot matched all apps. No source, test, route, keepWarm or JVM-argument changes; no product PR, merge or Maven publication. All local tasks launched for this lane have exited; product checkout is clean. Both jars remain on the host.

Relevant refs

• Full report: https://github.com/CodexCoder21Organization/ContainerNursery/blob/wip/laneDCN-evidence/laneDCN2/rDCN-final.md
• Findings: https://github.com/CodexCoder21Organization/ContainerNursery/blob/wip/laneDCN-evidence/laneDCN2/findings.md
• Screenshot: https://github.com/CodexCoder21Organization/ContainerNursery/blob/wip/laneDCN-evidence/laneDCN2/droplets-first-frame.png
• Evidence branch: https://github.com/CodexCoder21Organization/ContainerNursery/tree/wip/laneDCN-evidence/laneDCN2
• Source: https://github.com/CodexCoder21Organization/ContainerNursery/commit/45324dc2b27b1d1562887b9023e463ecafc0e32e
• Requested merged work: https://github.com/CodexCoder21Organization/ContainerNursery/pull/695 and https://github.com/CodexCoder21Organization/ContainerNursery/pull/700

Current production command and jar

Previous jar /root/ContainerNursery/bin/container-nursery-7dd1ba45-20261021T141028Z.jar is restored, hash 9c1805c5e35bcf7e74c509810207b079ea5f43b4d489883c269c5865088410a1. previous-command.json and cmdline-restored.bin on the evidence branch preserve the complete command. PID 3522254 runs that exact command; all JVM arguments remain unchanged. The new-main jar and launch scripts remain alongside it under /root/ContainerNursery/bin; deployment-settings.json provides exact paths and unit names.

Next steps

Owner direction is needed to scope investigation of the WUI service-connection notice and authorize any additional deployment. Use the saved first frame, exact request timings and current service logs to identify why the WUI is still connecting before attempting another deploy. The main-deployment objective remains unmet, so this handoff stays open.

Operational knowledge

Use CN CLI 0.0.20 health-detailed --json. Use HCF CLI 0.2.6 with client_cert.pem, fabric_key.pkcs8 and server_cert.pem under ~/.config/hardware-control-fabric/. Read proc through HCF launch of /usr/bin/cat; read-file yields no proc contents. Upload a shell script for systemd-run because HCF argument strings are split simply. Use Chromium cached version 1228 with the documented aarch64 library directories. The old CN had no restart-handoff environment entry; the new and restored transient units set it to 1. Sequential process and container snapshots are not atomic; resolve unmatched app entries before declaring verification complete. No automatic restart policy was added.