Track and chart which agents and models work on each handoff (minutes and tokens, share and timeline)
RE-VERIFY: everything below is a snapshot written 2026-10-20 ~21:45 UTC while five implementation lanes were running. Before acting, re-check each repository's branches (git ls-remote https://github.com/CodexCoder21Organization/<Repo> 'refs/heads/*usage*'), open PRs (gh pr list --repo CodexCoder21Organization/<Repo>), and published versions (probe https://kotlin.directory/handoff/<artifact>/<v>/<artifact>-<v>.pom).
Design decisions in force
-
2026-10-20, the user, verbatim: "A handoff should keep track of which agent and model(s) are working on the handoff, and display the information both as a percentage chart (how many minutes and how many tokens for each model, in aggregate) and as a timeline (who was working when, so for example, we can see if a handoff was originally being handled by mostly opus but then graduated to be handled mostly by opus+sol). Actually, perhaps it should be shown as a stacked chart with time on the x axis and the percentage of time/tokens spent in a given model on the Y axis. This involves several changes, we will need a place to store/record the relevant information and also will need to change the API/cli/tools to enable reporting of this information, etc. We care which agent and also care which model (if agent is unknown)."
-
2026-10-20, the user, verbatim: "despite running as opus, Delegate the work to gpt 6.1-sol, maximize parallelization and race PRs." (Part of the original request, so carried over.)
-
2026-10-20, the user, verbatim: "Also remote CI has some issues which you can work around by doing as described in https://my.externalbrain.wasmserver.com/doc/github-builders-racing-remote-ci.md" — that procedure: pin kompile-cli >= 0.0.119 (0.0.124), add a GitHub Actions bld-build workflow that runs --local-and-remote (GitHub runner races the remote service, first result wins), and once it is green on main make bld-build the required check in the main ruleset in place of kotlin.build (remote). HandoffEmbedded and HandoffServiceServer already have this; HandoffApi, HandoffCli and HandoffWui still require kotlin.build (remote) and need the rollout (one PR each, branch ci-bld-build).
-
The settled design (orchestrator-written): CONTRACT.md. Summary: usage stored as 15-minute buckets per (handoff, harness, session id, agent, model, effort), upserted (replace, not add) so re-reporting a whole transcript is idempotent; recordUsageJson / getUsageJson on HandoffService with throwing defaults; Embedded persists with no retention pruning; the CLI computes buckets from the Claude Code / codex transcript located by exact session id and uploads automatically after every handoff-cli report; the WUI handoff detail page shows 100 %-stacked share bars (minutes, tokens) with absolute totals and a 100 %-stacked timeline, switchable by model vs agent+model.
-
Contract amendments 1–3 (2026-10-20, orchestrator, driven by review evidence): remote errors arrive wrapped (match on message text); the CLI's attribution window never shrinks and codex counter resets are handled; the usage projection is bounded — a server-chosen timeline of at most 2000 points instead of raw buckets. Full text in CONTRACT.md on the PlanRepository branch.
Mission summary
Five repositories change, each with its own PR on branch usage-tracking, assigned versions: HandoffApi 0.0.28, HandoffEmbedded 0.0.40, HandoffServiceServer 0.0.31 (+client), HandoffCli 0.0.9, HandoffWui 0.0.30. The PRs race: downstream lanes build against release candidates published from checkpoint branches (handoff:api:0.0.28-usage-rcN, handoff:embedded:0.0.40-usage-rcN, handoff:serviceserver(-client):0.0.31-usage-rcN) and switch their pins to final versions once the upstream PR merges and the final version is published (lines marked TODO(usage)).
What was found and done
- Explored the five repos (HandoffApi, HandoffEmbedded, HandoffServiceServer, HandoffCli, HandoffWui). Claims and status reports already carry a conversation reference (harness + native session id) discovered from
CLAUDE_CODE_SESSION_ID / CODEX_THREAD_ID; that is the key used to find the transcript to measure.
- Transcript formats verified on the /code box: Claude Code
~/.claude/projects/<proj>/<session>.jsonl (assistant lines carry message.model, message.id, message.usage; one API message is split across several lines repeating the same id and usage — must be counted once), subagents in <session>/subagents/agent-*.jsonl + .meta.json; codex ~/.codex/sessions/YYYY/MM/DD/rollout-*-<threadId>.jsonl with turn_context (model, effort) and cumulative token_count events (total_token_usage{input_tokens, cached_input_tokens, cache_write_input_tokens, output_tokens, reasoning_output_tokens}).
- HandoffWui already draws server-side SVG charts on its dashboard; the usage charts follow that pattern.
- Five implementation lanes launched 2026-10-20 ~21:40 UTC; briefs are in the artifacts directory linked above.
Relevant PRs / refs
| Repo |
Branch |
Remote head SHA |
PR |
What is on it |
State |
| CodexCoder21Organization/PlanRepository |
wip/handoff-usage-tracking-2026-10-20 |
see branch |
no PR |
contract + lane briefs |
reference only, not to merge |
| HandoffApi / HandoffEmbedded / HandoffServiceServer / HandoffCli / HandoffWui |
usage-tracking (PR), wip/usage-<lane>-ckpt (checkpoints) |
not yet pushed at write time |
none yet |
lane work |
in progress |
Deployed or published by this effort at write time: nothing (checked by .pom probe: handoff:api:0.0.28 → 404). Production url://handoff/ is unchanged; a production deploy waits for the user to ask.
Next steps
- As each lane finishes: independent test-comprehensiveness review and independent defect review of the exact PR head, fixes back to the author, final review, merge.
- Merge order for publication: Api → publish 0.0.28 → Embedded/ServiceServer/Cli/Wui swap
TODO(usage) pins to final versions → Embedded → publish 0.0.40 → ServiceServer → publish 0.0.31 → Cli and Wui.
- Update the
status-report / create-handoff skills and the delegation playbook so orchestrators report delegate usage: handoff-cli usage report <handoff> --conversation codex:<threadId> --agent <laneName>.
- Ask the user before deploying the ServiceServer and WUI to production.
Reusable knowledge
- The /code box disk was at 100 % at the start (other containers); pruning
~/.codex/sessions older than a day freed 2 GB; it later recovered to 51 GB free on its own.
~/bin/cs (x86_64 coursier) was missing after the container rebuild; reinstalled from the coursier releases page. /usr/local/bin/coursier is the wrong architecture.