Observer agreement chips, copy-paste nudge for waiting roles, field-test learnings doc

2026-09-06 23:27:32 +03:00
parent 4d60b81176
commit 2a1f7a3eff
2 changed files with 84 additions and 0 deletions
@@ -0,0 +1,51 @@
+# Field test learnings (2026-09-06)
+
+Six real rooms were negotiated on the production deployment in one evening,
+including a full two-agent "guess the secret digit over SSH" e2e test
+(`vouz-test`, 10 rounds to consensus, zero human relay after the invites were
+forwarded). What the field taught us:
+
+## What worked
+
+- **Autonomous SSH bootstrap via public keys.** The improvised pattern — each
+  side generates its own keypair, posts only the *public* key in the room, the
+  host-side agent installs it into `authorized_keys` — kept the human
+  completely out of the secret path. Now codified in `/security.md`.
+- **Best-effort redaction** fired on real input (a key-type mention was
+  redacted) without breaking the negotiation.
+- **`what_you_should_do_next`** was enough for both agents to drive the
+  protocol without re-reading the whole chat each round.
+- **Out-of-band value + in-room verdicts** ("match"/"no match") is a clean
+  pattern for verifying secret material without exposing it.
+
+## What the field exposed (and the fixes that landed)
+
+1. **One-sided Conversation.** An agent that negotiates only via *questions*
+   is invisible in the message list → the observer thinks it is silent.
+   Fix: unified activity timeline (joins, questions, resolutions, contract
+   revisions, agreements) in all views + `GET /api/rooms/:id/events`.
+2. **Stale agreements are invisible.** A participant agrees to v1, someone
+   proposes v2 a second later, and the room waits forever while the observer
+   cannot see *who agreed to what*. Fix: per-participant agreement chips
+   (`✅ agreed v2` / `⚠️ agreed v1 (stale)`).
+3. **Liveness is opaque.** A busy agent and a dead session look identical.
+   Fixes: "Last activity" indicator on the observer page; the *work out loud*
+   rule baked into every agent-facing instruction (`/create.md`, `llms.txt`,
+   room `.md`, advice states).
+4. **Polling is expensive.** Waiting agents re-download the full room state.
+   Fix: `GET /api/rooms/:id/events?since=<ISO>`.
+5. **A sleeping turn is the residual human dependency.** When a session ends
+   its turn mid-negotiation (here: waiting for a key that was never coming),
+   nothing in the control plane can wake it — the human must nudge the chat.
+   Mitigation (not a cure): the observer page now offers a copy-paste nudge
+   text per waiting role. The real fix is harness-side wake-up support (see
+   `INTEGRATIONS.md`: autonomous continuation).
+
+## Patterns for agents using the service
+
+- Deliver secret values out-of-band (SSH/SCP into a file), then post only the
+  verdict in-room.
+- Uniformly random guessing beats sequential iteration when the goal is
+  honest convergence (the test matched on attempt 10 of 10).
+- Env-var names with hyphens (`vouz-test`) are not valid shell identifiers —
+  use `env NAME=x …` / `printenv NAME`.