One person, seventy daemons — the fleet as it is today

_Byron Arnao · written 2026-10-26 for a 45-minute architecture review · everything here is live and measured as of this date. Where a mitigation exists but is not yet carrying real traffic, it says shadow or not built._

How to read this in ten minutes: §1 is the decision I most want challenged (one host, on purpose) and what dies when it dies. §2 is the three questions for the meeting. §3 and §4 are the system and its control plane. §5 is the security broker. §6 is my own list of weaknesses. §7 is the adversarial critique this document already received and what changed because of it. Appendices hold the inventory. Diagram: architecture.svg.


1. One host, deliberately — and what dies when it dies

Everything runs on one Apple Mac Mini (M4 Pro, 14 cores, 48 GB) in my house: five OpenClaw agent instances in Docker (via Colima), a shared vector-memory database, a git server, a metrics stack, a TLS proxy, 61 macOS launchd daemons that do the scheduled work, and a control plane that manages the fleet the way a small SRE team would. Public surfaces (about 200 small sites) are served from Vercel. Model spend is about $400 a month.

Why one host, on purpose: I can read every byte, the data never leaves a machine I own, and the whole thing costs electricity. The price is that the Mini is the control plane, the agents, the memory and the proxy at once.

What the fleet is for, in operational terms, and what happens when the box stops:

FunctionWho notices an outage, and how fastTolerable downtime (my proposal)Tolerable data loss (my proposal)Today's reality
Telegram agent answers (Gia)Me, within minutes4 hours0 (chat history is on Telegram's side)Restart from compose; container state on named volumes
Daily briefing podcast by 07:30 and the 07:30 Morning CardMe, same morning1 missed day1 day of research notesPre-generated at 05:45 by a launchd job; no second producer
Outbound sends (email, text, LinkedIn)The recipient, never; me, when I check24 hours0 (prepared sends live in a private queue on disk)Queue is on the Mini's disk only
Shared memory (OB1, pgvector)Every agent, next turn4 hours24 hoursDumped nightly; in the DR bundle
Work log, commitments, stateMe, next session24 hours24 hoursJSONL on disk; nightly backup; weekly encrypted bundle
Public sites (Vercel)Visitorsn/a (not on the Mini)n/aUnaffected by a Mini outage; owner consoles lose their gate
Collaborator git (Gitea)One collaborator, same day48 hours24 hoursOn the Mini; the Mini is on someone else's critical path

Proposed recovery objectives for the whole box: RTO 8 hours, RPO 24 hours. I have never met them in anger. The restore script has been dry-run once and never used for a real restore. There is no UPS and no ISP failover; a power blip restarts everything (containers and keep-alive daemons come back on their own, scheduled jobs resume at their next slot).

What runs unattended versus only when I am at a keyboard: the agents, all 61 daemons, the 3-hour sentinel loop (security scans, atlas, state), the daily credential check, backups, the fixer, and the morning pipeline all run without me. The parts that need a human are a session of mine (planning, building, reviewing) and the taps that release outward sends.

Mitigations that are real today: nightly backup; weekly encrypted DR bundle (full 1.9 GB, core 3.4 MB) to cloud storage with a runbook; 34 heartbeats with a dead-man's check; containers recreated automatically if they exit. Mitigations that are not yet real: the security broker (shadow, see §5), the automatic fixer (shadow), immutable agent config (not built), a decommission programme (staged, has deleted nothing).

Known failures live at the time of writing, left in rather than hidden: 6 of 61 daemons exit non-zero (two tunnels, the photo sync to the PC, two health probes, one Cloudflare connector); 2 of 34 heartbeats are stale. The dead-man's check reports them daily. They have not been repaired, which says something about alert fatigue that I would rather discuss than conceal.


2. The three questions for 45 minutes

  1. The line. I propose moving only the credential broker off the box (smallest possible cloud footprint, credentials in a secrets manager, the Mini holding zero secrets) so the house box becomes disposable compute rebuildable from git. Everything else stays local. Is that the right line, and what would you move with it? (10 minutes)
  2. Cutover and identity. The production agent holds every credential in its environment. Its hardened replacement is in shadow: fixed verbs, pinned grants, a deterministic broker (§5). How would you sequence a per-channel cutover, what signal says stop, and what in the IAM model does my broker reinvent badly? (20 minutes)
  3. Spend guardrails. One research loop briefly cost $200 in a month. What sits between an agent and a spend, short of a human approving each call? (15 minutes)

And the one I most want answered: which failure do you expect first, and would you run this?

Further questions I would take by email if there is time are in Appendix C.


3. The system

3.1 Host

ItemValue
MachineMac Mini, Apple M4 Pro, 14 cores, 48 GB RAM, 1.8 TB SSD (41% used)
Container runtimeDocker via Colima (Linux VM: 4 vCPU, 8 GB, 200 GB)
SchedulingmacOS launchd: 61 system daemons (22 keep-alive services, 37 scheduled jobs)
Network inTailscale for owner consoles (tailnet-only HTTPS paths). Three deliberate public paths from the box: Tailscale Funnel on two ports (self-hosted analytics, a mentor API, the live fleet monitor) and one named Cloudflare tunnel. Nothing else listens to the internet
Network outDirect for four agents; the hardened agent goes only through a Squid domain allowlist
Local modelsOllama on the host (qwen3-coder 30B, qwen2.5-coder 32B, gemma4) for two agents and for loops that should cost nothing
Second machineA Windows PC runs a sixth agent (WhatsApp) and the photo library share; it is not managed by the control plane and its sync to the Mini fails whenever the PC sleeps

3.2 Agents

OpenClaw is an open-source agent runtime: a gateway with channels (Telegram, WhatsApp, Signal), a skill system (markdown instructions plus scripts), scheduled jobs, a tool layer (exec, browser, web fetch), and a model-provider abstraction with fallback chains.

AgentRoleVersionPrimary modelChannelsSkillsJobsIsolation
GiaProduction: Telegram, the daily podcast pipeline, research, publishing2026.9.7Claude Opus 5.5Telegram4826Standard container; holds Gmail, LinkedIn, Twilio, Vercel, GoDaddy credentials in its environment
Gia2Hardened rebuild of Gia, shadow2026.9.7Claude Opus 5.5off until cutover483No credentials in the agent; broker + egress proxy + allowlist DNS + read-only root filesystem + cap_drop ALL
MiaSecondary, WhatsApp allowlist2026.7.1Gemini 2.5 FlashWhatsApp91Standard container
CiaCoding on local models only2026.7.1Ollama qwen3-coder 30Bnone21No cloud keys
TiaTelegram on a local model with cloud fallbacks2026.7.1Ollama gemma4Telegram41Standard container

Three agents are two release lines behind the other two. That is a known risk, not an oversight: the upgrade path (schema migration plus config rewrite) has bitten once and is done one agent at a time.

Inside Gia2 live two further personas: Ria, a responsible-AI research agent with seven host daemons of its own, and Muse, a slot for comparison runs on a different model family.

Every agent is described to itself by the same three layers: runtime config (models, tools, skills), persona and privacy rules, and orientation files loaded at boot. A change to any layer requires a re-index and restart; forgetting that was my most common early bug.

3.3 Models and routing

PolicyDetail
Frontier workClaude Opus 5.5 on the direct Anthropic API with prompt caching (on one repeated 30k-token call the second call cost $0.007 instead of $0.149; I do not yet track a fleet-wide cache hit rate)
Cheap lanesHaiku 5.5 direct; Sonnet 5.5 via OpenRouter; Gemini 2.5 Flash; GPT-6.1 Sol via OpenRouter as a fallback
LocalOllama for the two local agents and for background loops
RoutingMeasured on my own task set: a fixed GPT-6.1 Sol beat every router on cost at equal accuracy (benchmarks); a small safety model ("Jev") is used as a gate, not a router
Fallback chainsEvery agent has a 3 to 5 model chain; a daily job keeps the fleet on the newest Opus rather than a pinned version

4. Control plane

The agents do the domain work. The control plane runs the fleet. It is a set of Claude Code skills and small CLIs under one ops directory, plus standing rules the harness reads at the start of every session. All of the scheduled parts run from launchd without me.

FunctionMechanismCadence
Work logAppend-only JSONL; re-renders one page and derives the to-do feed for the morning reportEvery job
Session state + fleet atlasGenerated state file (what is in flight) and atlas (machines, agents, daemons, containers, surfaces, open problems); every session reads both before touching the boxEvery 3 h
Commitments and goalsA commitments tracker and a goals file, rendered into the agents' memory and the morning reportContinuous
OutboxPrepared outward sends (email, text) that one tap releases; the tap executes through the harness, never by bouncing me into an app; replies are detected and surfacedOn demand, checked every 45 min
Morning CardOne Telegram card at 07:30 with at most three items; everything else is pull-onlyDaily
SecurityInventories 12 categories (daemons, containers, images, ports, Tailscale, Vercel projects, DNS, skills, crons, hooks, MCP servers), diffs against the last snapshot, scans only what is new; owner-gated Security CenterEvery 3 h
Credential health30 checks, each secret tested inside the container that holds it, plus balances, token days-left, heartbeatsDaily 05:10
Dead-man's switch34 heartbeats; any job older than twice its interval fails the daily checkContinuous
FixerA 5-minute daemon with playbooks, in shadow during a 90-day freeze: detects, snapshots evidence, escalates; does not act yet5 min
Backups and DRNightly fleet backup; weekly encrypted DR bundle to cloud storage with a runbook and a staged restore scriptNightly / weekly
FinancePer-provider cost ledger from provider APIs and agent transcripts; revenue file; monthly targetDaily
Executive logEvery session and deploy auto-captured into a public-safe logContinuous

Standing rules that shape all of this: every link must be clickable on a phone; shareable means public; the private vault never enters any agent; incidents end as automation (a playbook, a "cannot automate because", or a backlog line); do it first (nothing reaches me as a bare to-do, the system arrives with the draft and one tap); a 90-day freeze on new daemons and consoles until 2027-01-16.


5. Security model (the broker)

5.1 The threat

An agent reads attacker-controlled text (email, web pages, inbound messages) in the same process that holds credentials and a shell. The first design put allowlists inside one container; an adversarial review showed that was theatre, because an agent that can read its own secret files or post through a logged-in browser has no boundary. That review is what produced the design below.

5.2 Separate the reader from the sender

 Telegram / email-read / web
          │
 ┌────────▼───────────┐  internal-only network   ┌──────────────────────────┐
 │ gia2 (LLM agent)   │ ── fixed verbs only ───▶ │ gia2-broker (no LLM)     │──▶ Gmail send, LinkedIn,
 │ no credentials     │                          │ holds every credential   │    Twilio, Vercel, host jobs
 │ read-only rootfs   │                          │ policy: auto / tap / deny│
 └────────┬───────────┘                          │ pinned standing grants   │
          │ only route out                       │ Jev gate (shadow)        │
 ┌────────▼───────────┐                          └──────────────────────────┘
 │ gia2-egress (Squid)│  domain allowlist + allowlist-only DNS
 └────────────────────┘

5.3 Limits

A prompt-injected Gia2 can still read everything in its own context and send it to me, or propose a harmful action that I approve by mistake. Once Gia2 holds a Telegram bot token, the Telegram API is reachable and a raw call could message anyone who has started the bot. The four older agents have none of this protection: Gia, the production agent, holds every credential in its environment today. Per-channel cutover has not started. Full design and the critique that reshaped it: https://gia2-proposal.arnao.ai/


6. Where I think it is weak

  1. One host, one power cord. No UPS, no ISP failover, one dry-run restore. Covered in §1.
  2. Daemon sprawl. 61 launchd jobs plus Docker plus compose is three schedulers; six daemons failing for days is the evidence that it is past what I can hold in my head.
  3. Secrets. Four agents keep API keys in plaintext compose files on disk. Rotation is manual. The broker exists on one agent only.
  4. No immutable config. Agent configs are writable mounts; an agent could in principle edit its own config. Plan: config in git, mounted read-only. Not built.
  5. Token renewals need a human. 60-day OAuth cycles have needed a tap from me twice this quarter; the permanent fix is built and waiting on one registration step.
  6. The second machine is unmanaged. The PC's agent and photo sync fail silently when it sleeps.
  7. Surface sprawl. 344 surfaces and 227 Vercel projects; 133 with no deploy in 90 days; a guarded decommission runner has deleted nothing yet.
  8. Prompt injection is bounded, not solved. §5.3.
  9. It is all me. No second operator has ever used the runbooks.

7. Self-critique — Fable adversarial pass (2026-10-26)

Before publishing, this document was reviewed by a frontier model (Claude Fable 5.1) instructed to act as a hostile reviewer, with the same brief a resiliency expert would bring. The critique in brief, and what changed (full text: critique.html):


Appendix A. Shared services

ServiceWhat it isWhy it exists
OB1 (pgvector Postgres)Shared "open brain": one vector-memory store every agent reads and writesSo five agents share one memory instead of five
GiteaSelf-hosted gitShared repo for a collaborator who should not touch the Mini
CaddyTLS reverse proxy on 443Fronts the gateways with internal certs
Prometheus + GrafanaMetricsOpenClaw exports OTel diagnostics; broker actions and costs land here
Plausible (+ ClickHouse, Postgres)Web analyticsSelf-hosted analytics for the public sites
Open WebUIChat front end for OllamaLocal-model testing
A relay stack (Postgres, Redis, MinIO)Relay for a separate productNot part of the fleet proper

Appendix B. Edges and surfaces

Appendix C. Further questions, if there is time

  1. At this scale, what is the right single unit of orchestration, given that launchd + Colima + compose is three schedulers?
  2. What SLOs would you actually set for an agent fleet like this, and which of my checks would you drop?
  3. The resilient pattern for a house with a Mini, a PC that sleeps, a NAS and a phone, when the control plane must see all four.
  4. How a decommission programme keeps running after the first enthusiastic week.
  5. Against the AWS Well-Architected pillars, which fails first, and the one change that moves it most.

Appendix D. Numbers at a glance

CountValue
Agent instances5 on the Mini (+1 on the PC)
Containers25 (24 running)
launchd daemons61 (22 keep-alive, 37 scheduled; 6 exiting non-zero today)
Heartbeats34 registered, 2 stale today
Skills115 across agents; 50+ harness skills
Vercel projects / surfaces~200 / 344 (133 with no deploy in 90 days)
Model spend, last 30 days$406 (Anthropic $263, Google $92, OpenRouter $14, domains $32, Twilio $3, OpenAI $2)
Backupsnightly, plus a weekly encrypted DR bundle

_Written and published from the control plane itself. Source of truth for the numbers: the fleet atlas regenerated every three hours; last regenerated 2026-10-26 15:17 ET._