Workstream: MyLittleAgi

Status: Planned · Component: Enable agentic swarms

This is the end-user console for the agent system — the consumer-friendly face a person uses to create and talk to their agents. Its companion is the AiCliHostSupervisor workstream, which is the durable home of the agent (definition, permissions, history, mailbox, runtime config, conversation catalog — everything) and actually runs its conversations. Read the two together: the supervisor is where an agent lives; MyLittleAgi is how an end user reaches it — without needing to know which supervisor that is. Its admin-facing sibling console — same supervisors underneath, different audience — is AiCliSupervisorManager.

Goal

Make MyLittleAgi the place a user creates their own agents and has conversations with them — without knowing or caring where those agents run. A user should be able to define an agent — give it a persona, grant it tools, data, and credentials, and decide which other agents it may talk to — and then start a conversation that runs that agent live, with full continuity of history across sessions. MyLittleAgi is the end-user console: a consumer-friendly frontend that drives one or more AiCliHostSupervisor instances — where the agents actually live (their definitions, permissions, history, and mailbox) — and presents all of it through a chat-style UI. The backing supervisors are an implementation detail the user sees only deep in a settings page.

And it meets the user on their own devices: that console is a native LittleAGI app on desktop and Android — driven by voice, summoned by a spoken hotword, the command line, the system-tray icon, or a file's right-click menu — so an agent is something that lives with you and can act on your real local files, not only a server-side workspace (clients & invocation).

The heavy lifting of running and storing an agent happens in AiCliHostSupervisor: the supervisor durably owns the agent in full (definition, permissions, runtime config, conversation catalog, transcript history, mailbox) and runs its conversations. MyLittleAgi issues create-agent and start/continue-conversation commands to the agent's home supervisor and renders the read models it exposes; it holds no agent state of its own, only thin user-facing metadata (which supervisors back it, human-friendly names/preferences, a cached aggregate). In the default local install the supervisor it drives is bundled right into the app, so "where the agent lives" is simply your own machine.

Note — repositioning. This workstream previously framed MyLittleAgi narrowly as "the reusable template for agent-facing services" (a trivial getSecretPassword() demo across all layers), and later as the durable brain that stored agents. That all-layers demo is the seed this grows from and still serves as the standard-architecture reference, but the goal has since sharpened twice: MyLittleAgi is now the end-user console over a fleet of supervisors — the consumer-friendly frontend that creates and converses with agents — not a copyable scaffold and not a durable agent store (the agent lives on its home supervisor). The "agent-facing service template" idea lives on as its own Agent-facing service template workstream.

What an agent is, here

• An agent is a persistent entity owned by a user and stored durably by its home AiCliHostSupervisor: a name/persona, which AI CLI it runs, the tools it may use, the data it can read/search, the credentials it may exercise, the peers it may message, and its accumulated history. It exists whether or not a conversation is currently running; MyLittleAgi is where a user creates, configures, and reaches it.
• A conversation is a running PTY of that agent on its home AiCliHostSupervisor. Ephemeral: launched from the supervisor's stored definition, paused or terminated on the user's request, with transcript and outbound messages committed to the supervisor's own durable store as events — which MyLittleAgi renders.
• Tools are LambdaServer functions the agent is permitted to invoke, provided to the conversation as CLI executables mounted into its container (not MCP, for now) — including built-ins like SearchableBucket for the agent's searchable task context and SimpleFileSystem for its workspace.
• Permissions are W3Wallet capabilities. Granting a tool, a data bucket, a credential, or the right to message a peer is delegating a capability; revoking it removes the ability. The agent never holds raw secrets. MyLittleAgi is the UI to configure these grants; at conversation launch the home supervisor delegates the agent's granted capabilities (on behalf of the owner's wallet) to the W3Wallet identity the supervisor and guest negotiate for that run (see the engine design).
• Agents talk via an async mailbox. Sending a message is a command (an event); an agent's inbox is an Observable projection it reads when it runs — see the supervisor design.

The LittleAGI app — clients, voice, and how you summon it

The end-user console is a native LittleAGI app that lives on the user's own machines, so an agent is something that is with you — on your desktop and in your pocket — and acts on your real files and applications, not only a remote workspace. The existing MyLittleAgiGui (Compose Desktop/Wasm) is the seed of the desktop client; an Android app extends the same agent surface to the phone; the browser MyLittleAgiWui remains the zero-install web face.

You summon an agent however is closest to hand:

• Voice, hands-free. A realtime speech-to-speech API (something like the OpenAI Realtime API, or an equivalent) drives the frontend conversationally — you talk to your agent and it talks back, over the same conversation it would show on screen.
• A spoken hotword wakes it without touching the device — "Ok little agi", "ok my little agi", "ok my ai", "ok my little ai", or "hey little agi".
• The command line. littleagi convert all these files to jpegs runs an instruction straight from a shell, in the current directory's context.
• The system tray. Clicking the tray icon opens the LittleAGI window for a quick conversation.
• The file manager. Right-clicking files and choosing "LittleAGI" hands those files to an agent as the subject of the next instruction.

How the app is composed

LittleAGI separates three roles that are bundled together — run in one process on a single device — by default (the standard install is fully local, which is what makes the agent "live on your computer"):

• the user interface — the clients above;
• the model/engine — an AiCliHostSupervisor running the agent's conversation;
• file access — the tool functions available to the agent are proxied to the user's local machine, so "convert these files" acts on the real files you pointed at. This is the platform's existing delegated-capability / reference-proxy model (engine §5) applied to the local environment: the agent invokes a function, the work runs where the files are, only the result comes back.

Because the three roles are separable, the topology is the user's choice. Advanced users can run AiCliHostSupervisor on other hardware and configure LittleAGI to use it — keeping the UI local while the agent lives (and runs) on a beefier machine, for example. LittleAGI can likewise be pointed at one or more HardwareControlFabricDaemon instances, giving an agent the ability to reach and work with files on those machines too. The console / supervisor split is preserved throughout: the agent — definition, history, mailbox and all — lives on whichever supervisor homes it (the bundled-local one by default, or a configured remote one), and MyLittleAgi is the UI over it. Where a conversation runs is simply where the agent is homed (engine §8); MyLittleAgi keeps no agent state of its own, so switching devices or redeploying the console loses nothing.

Current state

MyLittleAgi exists as a complete, deployed project whose domain is still intentionally trivial (a single getSecretPassword() operation) — but which already demonstrates the full standard stack and an MCP surface:

Layer	Repository	Notes
Api	MyLittleAgiApi	MyLittleAgiService interface
Embedded	MyLittleAgiEmbedded	in-process domain logic
ServiceServer	MyLittleAgiServiceServer	hosts url://my-little-agi/
WUI	MyLittleAgiWui	chat UI at my-little-agi.wasmserver.com
GUI	MyLittleAgiGui	Compose Desktop/Wasm + embedded MCP server exposing a tool

Its interface is already powered by AiCliHostSupervisor behind the scenes, and it works end to end across every layer plus a frontend-and-MCP surface. So the shape is proven; what is missing is the durable agent model (which now lands on the supervisor, per the engine workstream) and the console over it: today there is one fixed operation, not user-defined, persistent, capability-scoped agents with history and a mailbox that a user browses and drives. This workstream is about growing the deployed demo into the end-user console described above.

Why it matters for swarms

• It's the human entry point for individual agents. A person defines an agent and converses with it; oversight sits at the level of "what may this agent do," expressed as capabilities. (SpawningPool is the corresponding surface for farms of mostly-autonomous agents — overlapping, but a different emphasis.)
• It's how a person reaches their agents, wherever they live. The agent — definition, permissions, history, mailbox — is durably owned by its home supervisor; MyLittleAgi is the consumer console that lets a user create, configure, and converse with it without tracking which supervisor that is. Because it holds no agent state of its own, a console redeploy or a move between devices loses nothing — the agent stays put on its home supervisor. (Its admin-facing sibling, AiCliSupervisorManager, is the same kind of console for fleet operators.)
• It composes the whole platform. An agent's tools are LambdaServer functions, its memory is SearchableBucket, its files are SimpleFileSystem, its authority is W3Wallet, and its history/mailbox are Event Log + Observables — these compose into an agent on its home supervisor, and MyLittleAgi is where a user meets that agent.

Plan / roadmap

These are proposed and need sharpening before work starts; they assume the AiCliHostSupervisor engine deltas land in parallel.

• [ ] Agent-creation & configuration UI. Create and edit agents (CLI choice, persona/system prompt, granted capabilities, allowed peers) by driving the home supervisor's durable definition store, and browse the agents across the backing supervisors as one unified roster. (The event-sourced definition model itself is the supervisor's deliverable — engine.)
• [ ] Conversation history views. Render the read models the home supervisor derives and exposes — a rendered transcript for the chat UI and query-based recall — for display and lookback; continuity itself is the AI CLI's native resume over the supervisor's durable home (engine §9).
• [ ] Permissions UI over W3Wallet. Let a user grant/revoke an agent's tools, data buckets, credentials, and peer-messaging rights as W3Wallet capabilities on its home supervisor — and W3Wallet-gate MyLittleAgi itself for identity, authorization, and quota. (The supervisor delegates the granted set at launch; this is the UI to configure it.)
• [ ] Tool catalog ↔ LambdaServer. Browse/attach LambdaServer functions as an agent's tools, plus built-ins (SearchableBucket recall, SimpleFileSystem workspace, mailbox); the home supervisor assembles the selected set into the per-conversation CLI tools mounted into the container.
• [ ] Mailbox UI (same-supervisor roster). Surface an agent's inbox and let a user see and manage mail among their agents homed on the same supervisor, with the "all peers vs. subset" permission model. The message_sent stream and inbox projections live on the supervisor (engine §6); mail is delivered into the next human-opened conversation — in MyLittleAgi no mail wakes an agent, because "conversations are human-initiated" is this console's product policy (the engine itself supports trigger-woken conversations — monitor agents in engine §12, and SpawningPool's autonomy). No cross-user messaging; cross-supervisor federation is deferred in the engine.
• [ ] Conversation lifecycle UI. Create an agent, start/continue/observe a human-driven conversation (terminal + BUSY/IDLE) — human initiation being this console's own policy over an engine that also supports trigger-wake (engine §8) — with at most one live conversation per agent, and let the user pause or terminate a live session — sessions stay warm until then, and a paused one is continued later via the CLI's native resume. Durable pause/resume custody lives in the supervisor's own conversation catalog; MyLittleAgi relays the pause/terminate/resume command to the home supervisor (engine §8).
• [ ] Agent placement + create/converse commands. Discover supervisors on the url:// fabric, choose which supervisor a new agent is created on, and issue create/converse commands (the home supervisor assembles the rehydration bundle from its own durable state — engine §3); set the agent's default-deny egress policy in its definition (engine §10 — enforcement deliberately deferred there). An existing agent is pinned to its home supervisor.
• [ ] Native LittleAGI clients. Grow MyLittleAgiGui into a full desktop app and add an Android client over the same agent surface; the WUI stays the zero-install web face.
• [ ] Voice frontend. Drive a conversation hands-free with a realtime speech-to-speech API, plus a wake-word hotword ("Ok little agi", "ok my little agi", "ok my ai", "ok my little ai", "hey little agi") that opens an agent without touching the device.
• [ ] OS-integrated invocation. The littleagi command-line entry point (an instruction run in the current directory's context), a system-tray window, and a file-manager right-click "LittleAGI" action that targets the selected files.
• [ ] Local-execution topology. Bundle UI + a local AiCliHostSupervisor + local-file-proxy in one process by default, while letting a user point LittleAGI at AiCliHostSupervisor instances on other hardware, or at HardwareControlFabricDaemon instances, to work with files on those machines.
• [ ] End-to-end tests of define-agent → converse → invoke a granted tool → recall from SearchableBucket → message a permitted peer, against a real local supervisor + real backing services, per the testing standards.

Open questions

• Relationship to SpawningPool. Boundary now drawn at who drives, as console policy over a common engine capability: MyLittleAgi conversations are human-initiated (this console's product policy — the AiCliHostSupervisor engine itself supports trigger-woken conversations, which it must to power SpawningPool); SpawningPool embraces autonomous/woken execution and large farms. Both ride the same engine. Open: how much of the agent-definition model is shared vs. SpawningPool-specific.
• Whether (and how) to expose monitor agents. The engine's monitor agents (engine §12) are fully available to MyLittleAgi's users — an agent watched by a keyword-woken advisor is an end-user-friendly idea — but whether this console surfaces monitor creation (and in what simplified form, relative to the Manager's attach-monitor flow) is an open UX decision for this workstream.
• How much domain in the seed demo. Keep the getSecretPassword() demo as the standard-architecture reference, or retire it once real agent definitions exist?
• Agent-facing service template. The original "copyable scaffold" idea is now a separate Agent-facing service template workstream; does it remain a MyLittleAgi responsibility or split out entirely?

Graduation

MyLittleAgi already has a Documentation Repository project page. This workstream graduates when a user can define a persistent, capability-scoped agent in the console and hold a conversation with it — the conversation runs on the agent's home AiCliHostSupervisor, invokes a granted LambdaServer tool mounted as a CLI command, recalls context from SearchableBucket, and exchanges a mailbox message with a permitted peer homed on the same supervisor; the whole agent — definition, history, and mailbox — lives durably on that home supervisor (the bundled-local one by default), so it survives a console redeploy or a move between the user's devices because MyLittleAgi holds no agent state; and that agent is reachable from the native LittleAGI app — by voice, a hotword, the littleagi CLI, the system tray, or a file's right-click menu — and can act on the user's own local files through a bundled-local (or configured-remote) supervisor — at which point update the Documentation Repository project page.