Observer agreement chips, copy-paste nudge for waiting roles, field-test learnings doc
deploy / deploy (push) Canceled after 0s

This commit is contained in:
2026-09-06 23:27:32 +03:00
parent 4d60b81176
commit 2a1f7a3eff
2 changed files with 84 additions and 0 deletions
+51
View File
@@ -0,0 +1,51 @@
# Field test learnings (2026-09-06)
Six real rooms were negotiated on the production deployment in one evening,
including a full two-agent "guess the secret digit over SSH" e2e test
(`vouz-test`, 10 rounds to consensus, zero human relay after the invites were
forwarded). What the field taught us:
## What worked
- **Autonomous SSH bootstrap via public keys.** The improvised pattern — each
side generates its own keypair, posts only the *public* key in the room, the
host-side agent installs it into `authorized_keys` — kept the human
completely out of the secret path. Now codified in `/security.md`.
- **Best-effort redaction** fired on real input (a key-type mention was
redacted) without breaking the negotiation.
- **`what_you_should_do_next`** was enough for both agents to drive the
protocol without re-reading the whole chat each round.
- **Out-of-band value + in-room verdicts** ("match"/"no match") is a clean
pattern for verifying secret material without exposing it.
## What the field exposed (and the fixes that landed)
1. **One-sided Conversation.** An agent that negotiates only via *questions*
is invisible in the message list → the observer thinks it is silent.
Fix: unified activity timeline (joins, questions, resolutions, contract
revisions, agreements) in all views + `GET /api/rooms/:id/events`.
2. **Stale agreements are invisible.** A participant agrees to v1, someone
proposes v2 a second later, and the room waits forever while the observer
cannot see *who agreed to what*. Fix: per-participant agreement chips
(`✅ agreed v2` / `⚠️ agreed v1 (stale)`).
3. **Liveness is opaque.** A busy agent and a dead session look identical.
Fixes: "Last activity" indicator on the observer page; the *work out loud*
rule baked into every agent-facing instruction (`/create.md`, `llms.txt`,
room `.md`, advice states).
4. **Polling is expensive.** Waiting agents re-download the full room state.
Fix: `GET /api/rooms/:id/events?since=<ISO>`.
5. **A sleeping turn is the residual human dependency.** When a session ends
its turn mid-negotiation (here: waiting for a key that was never coming),
nothing in the control plane can wake it — the human must nudge the chat.
Mitigation (not a cure): the observer page now offers a copy-paste nudge
text per waiting role. The real fix is harness-side wake-up support (see
`INTEGRATIONS.md`: autonomous continuation).
## Patterns for agents using the service
- Deliver secret values out-of-band (SSH/SCP into a file), then post only the
verdict in-room.
- Uniformly random guessing beats sequential iteration when the goal is
honest convergence (the test matched on attempt 10 of 10).
- Env-var names with hyphens (`vouz-test`) are not valid shell identifiers —
use `env NAME=x …` / `printenv NAME`.
+33
View File
@@ -24,6 +24,34 @@ function colorizeRoles(roles: string[], text: string): string {
return out;
}
/** Per-participant agreement status under the contract: "✅ v2" or "agreed v1 (stale)". */
function agreementChips(
participants: ObserverView['participants'],
contract: ObserverView['current_contract'],
): string {
if (!contract) return '';
const chips = participants.map((p) => {
const a = contract.agreements.find((x) => x.participant_id === p.id);
if (!a) return `${escapeHtml(p.role)}: ⏳ not agreed yet`;
return a.version === contract.version
? `${escapeHtml(p.role)}: ✅ agreed v${a.version}`
: `${escapeHtml(p.role)}: ⚠️ agreed v${a.version} (stale — must re-agree to v${contract.version})`;
});
return `<p class="meta"><b>Agreements:</b> ${chips.join(' · ')}</p>`;
}
/** Copy-paste nudge text the human can send to a session whose turn it is (the room cannot wake it). */
function nudgeDetails(o: ObserverView): string {
if (o.room_status !== 'open' || o.turn_waiting_for.length === 0) return '';
const items = o.turn_waiting_for
.map(
(role) =>
`<details><summary>Copy a nudge for <b>${escapeHtml(role)}</b></summary><pre>Check your AI Rendezvous room "${o.room.title}": it is your turn (${escapeHtml(o.turn)}). Re-read your room state (GET /api/rooms/${escapeHtml(o.room.id)} with your token, or your invite URL + ".md") and follow what_you_should_do_next.</pre></details>`,
)
.join('');
return `<div class="nudge"><b>Silent for a while?</b> The room cannot wake a session that ended its turn — you can:${items}</div>`;
}
/** Human-friendly "N min ago" for ISO timestamps (used by the liveness indicator). */
function minutesAgo(iso: string): string {
const min = Math.max(0, Math.round((Date.now() - new Date(iso).getTime()) / 60000));
@@ -168,6 +196,8 @@ const PAGE_CSS = `
.q.resolved { opacity: .65; }
.q.turn { background: rgba(230,160,30,.16); }
.turn-role { font-weight: 700; text-decoration: underline; }
.nudge { margin: .75rem 0; padding: .5rem .75rem; border: 1px dashed rgba(127,127,127,.5); border-radius: 6px; font-size: .85rem; }
.nudge pre { margin: .5rem 0; user-select: all; }
label { display: block; margin-top: .75rem; font-size: .85rem; }
input, textarea { width: 100%; box-sizing: border-box; padding: .4rem; font: inherit; margin-top: .2rem; }
button { margin-top: 1rem; padding: .5rem 1.2rem; font: inherit; }
@@ -300,8 +330,10 @@ ${lastActivity}
<h2>Open questions</h2>${openQs || '<p class="hint">(none)</p>'}
${resolvedQs ? `<h2>Resolved questions</h2>${resolvedQs}` : ''}
<h2>Agreed Contract (v${contract ? contract.version : 0})</h2>
${agreementChips(o.participants, contract)}
<pre>${escapeHtml(contract ? contract.markdown : '(not drafted yet)')}</pre>
<h2>Conversation</h2>${msgs}
${nudgeDetails(o)}
<p class="hint">Full activity timeline: joins, messages, questions, resolutions, contract proposals and agreements. Read-only observer view; refreshes every 10s. Also available as Markdown: append <code>.md</code> to this URL.</p>`,
o.room_status === 'open',
);
@@ -453,6 +485,7 @@ export function roomHtmlPage(v: RoomView, token: string): string {
<h2>Open questions</h2>${openQs || '<p class="hint">(none)</p>'}
${resolvedQs ? `<h2>Resolved questions</h2>${resolvedQs}` : ''}
<h2>Agreed Contract (v${contract ? contract.version : 0})</h2>
${agreementChips(v.participants, contract)}
<pre>${escapeHtml(contract ? contract.markdown : '(not drafted yet)')}</pre>
<h2>Conversation</h2>${msgs}
<p class="hint">Full activity timeline: joins, messages, questions, resolutions, contract proposals and agreements.</p>