Corrected record: DigitalOcean snapshot RPC interrupted by sandbox watchdog

Reported (UTC): 2026-07-15 15:19

What was being attempted: Rebake the BuildTestEmbedded warm CI image by snapshotting droplet 584840690, named buildtest-rebake-tmp in sfo3, as buildtest-ci-ubuntu2404-warmcache-v3-2026-07-15 through digitalocean-droplet-cli 0.0.9. The repository-prescribed BuildDriver.installDependencies path had completed and the full 126-entry buildtest-runner 0.0.62 graph, including kompile manager 0.0.122, was cached.

What went wrong: The timeout-wrapped snapshot command failed with exit 1. The exact leading failure was "foundation.url.protocol.sandbox.SandboxException: SJVM host-side RPC interrupted during method=createSnapshot (watchdog timeout, cancellation, or shutdown); the host-side intrinsic handler did not complete." The cause was "java.lang.InterruptedException" from "foundation.url.resolver.PersistentRpcConnection.doSendRequest". The CLI JSON message began "Sandboxed code threw an exception: java.lang.InternalError: Exception encountered in implementation of foundation/url/sjvm/intrinsics/ServiceBridge:rpc" and repeated that the host-side RPC was interrupted during method=createSnapshot.

Impact: The task explicitly required stopping if the droplet CLI or service appeared broken, so snapshot-list verification, sfo3 and nyc3 availability verification, droplet deletion, source pin update, pull request creation, and CI could not proceed. Because createSnapshot may have continued remotely after the client watchdog interrupted, the resulting snapshot state is unknown.

Workaround used: None. Retrying or bypassing the required URL-protocol CLI would violate the task stop condition. The local renew loop was stopped after a final successful renewal at 2026-07-15T15:17:36Z, leaving the service deadline at approximately 60 minutes. The one-time SSH public key had already been removed from the droplet before the failed snapshot request.

Suggested durable fix: Make DropletService.createSnapshot asynchronous or extend the sandbox watchdog for this long-running operation, return an operation ID immediately, and add an idempotent status or reconciliation command so clients can determine whether a snapshot was actually created after an interrupted RPC.

This supersedes the malformed challenge record created at 2026-07-15T15:18Z, whose exact error fragments were lost to shell interpolation.

Production verification — 2026-07-17

Status: STILL EXISTS. No merged fix, retired component, or live recovery evidence was found for this mechanism in the current GitHub and production checks; the record remains actionable rather than obsolete.

This record was retained because its underlying mechanism remains observable or its durable fix is still open; historical incident details above remain useful reproduction evidence.