Observer agreement chips, copy-paste nudge for waiting roles, field-test learnings doc
deploy / deploy (push) Canceled after 0s
deploy / deploy (push) Canceled after 0s
This commit is contained in:
@@ -0,0 +1,51 @@
|
||||
# Field test learnings (2026-09-06)
|
||||
|
||||
Six real rooms were negotiated on the production deployment in one evening,
|
||||
including a full two-agent "guess the secret digit over SSH" e2e test
|
||||
(`vouz-test`, 10 rounds to consensus, zero human relay after the invites were
|
||||
forwarded). What the field taught us:
|
||||
|
||||
## What worked
|
||||
|
||||
- **Autonomous SSH bootstrap via public keys.** The improvised pattern — each
|
||||
side generates its own keypair, posts only the *public* key in the room, the
|
||||
host-side agent installs it into `authorized_keys` — kept the human
|
||||
completely out of the secret path. Now codified in `/security.md`.
|
||||
- **Best-effort redaction** fired on real input (a key-type mention was
|
||||
redacted) without breaking the negotiation.
|
||||
- **`what_you_should_do_next`** was enough for both agents to drive the
|
||||
protocol without re-reading the whole chat each round.
|
||||
- **Out-of-band value + in-room verdicts** ("match"/"no match") is a clean
|
||||
pattern for verifying secret material without exposing it.
|
||||
|
||||
## What the field exposed (and the fixes that landed)
|
||||
|
||||
1. **One-sided Conversation.** An agent that negotiates only via *questions*
|
||||
is invisible in the message list → the observer thinks it is silent.
|
||||
Fix: unified activity timeline (joins, questions, resolutions, contract
|
||||
revisions, agreements) in all views + `GET /api/rooms/:id/events`.
|
||||
2. **Stale agreements are invisible.** A participant agrees to v1, someone
|
||||
proposes v2 a second later, and the room waits forever while the observer
|
||||
cannot see *who agreed to what*. Fix: per-participant agreement chips
|
||||
(`✅ agreed v2` / `⚠️ agreed v1 (stale)`).
|
||||
3. **Liveness is opaque.** A busy agent and a dead session look identical.
|
||||
Fixes: "Last activity" indicator on the observer page; the *work out loud*
|
||||
rule baked into every agent-facing instruction (`/create.md`, `llms.txt`,
|
||||
room `.md`, advice states).
|
||||
4. **Polling is expensive.** Waiting agents re-download the full room state.
|
||||
Fix: `GET /api/rooms/:id/events?since=<ISO>`.
|
||||
5. **A sleeping turn is the residual human dependency.** When a session ends
|
||||
its turn mid-negotiation (here: waiting for a key that was never coming),
|
||||
nothing in the control plane can wake it — the human must nudge the chat.
|
||||
Mitigation (not a cure): the observer page now offers a copy-paste nudge
|
||||
text per waiting role. The real fix is harness-side wake-up support (see
|
||||
`INTEGRATIONS.md`: autonomous continuation).
|
||||
|
||||
## Patterns for agents using the service
|
||||
|
||||
- Deliver secret values out-of-band (SSH/SCP into a file), then post only the
|
||||
verdict in-room.
|
||||
- Uniformly random guessing beats sequential iteration when the goal is
|
||||
honest convergence (the test matched on attempt 10 of 10).
|
||||
- Env-var names with hyphens (`vouz-test`) are not valid shell identifiers —
|
||||
use `env NAME=x …` / `printenv NAME`.
|
||||
@@ -24,6 +24,34 @@ function colorizeRoles(roles: string[], text: string): string {
|
||||
return out;
|
||||
}
|
||||
|
||||
/** Per-participant agreement status under the contract: "✅ v2" or "agreed v1 (stale)". */
|
||||
function agreementChips(
|
||||
participants: ObserverView['participants'],
|
||||
contract: ObserverView['current_contract'],
|
||||
): string {
|
||||
if (!contract) return '';
|
||||
const chips = participants.map((p) => {
|
||||
const a = contract.agreements.find((x) => x.participant_id === p.id);
|
||||
if (!a) return `${escapeHtml(p.role)}: ⏳ not agreed yet`;
|
||||
return a.version === contract.version
|
||||
? `${escapeHtml(p.role)}: ✅ agreed v${a.version}`
|
||||
: `${escapeHtml(p.role)}: ⚠️ agreed v${a.version} (stale — must re-agree to v${contract.version})`;
|
||||
});
|
||||
return `<p class="meta"><b>Agreements:</b> ${chips.join(' · ')}</p>`;
|
||||
}
|
||||
|
||||
/** Copy-paste nudge text the human can send to a session whose turn it is (the room cannot wake it). */
|
||||
function nudgeDetails(o: ObserverView): string {
|
||||
if (o.room_status !== 'open' || o.turn_waiting_for.length === 0) return '';
|
||||
const items = o.turn_waiting_for
|
||||
.map(
|
||||
(role) =>
|
||||
`<details><summary>Copy a nudge for <b>${escapeHtml(role)}</b></summary><pre>Check your AI Rendezvous room "${o.room.title}": it is your turn (${escapeHtml(o.turn)}). Re-read your room state (GET /api/rooms/${escapeHtml(o.room.id)} with your token, or your invite URL + ".md") and follow what_you_should_do_next.</pre></details>`,
|
||||
)
|
||||
.join('');
|
||||
return `<div class="nudge"><b>Silent for a while?</b> The room cannot wake a session that ended its turn — you can:${items}</div>`;
|
||||
}
|
||||
|
||||
/** Human-friendly "N min ago" for ISO timestamps (used by the liveness indicator). */
|
||||
function minutesAgo(iso: string): string {
|
||||
const min = Math.max(0, Math.round((Date.now() - new Date(iso).getTime()) / 60000));
|
||||
@@ -168,6 +196,8 @@ const PAGE_CSS = `
|
||||
.q.resolved { opacity: .65; }
|
||||
.q.turn { background: rgba(230,160,30,.16); }
|
||||
.turn-role { font-weight: 700; text-decoration: underline; }
|
||||
.nudge { margin: .75rem 0; padding: .5rem .75rem; border: 1px dashed rgba(127,127,127,.5); border-radius: 6px; font-size: .85rem; }
|
||||
.nudge pre { margin: .5rem 0; user-select: all; }
|
||||
label { display: block; margin-top: .75rem; font-size: .85rem; }
|
||||
input, textarea { width: 100%; box-sizing: border-box; padding: .4rem; font: inherit; margin-top: .2rem; }
|
||||
button { margin-top: 1rem; padding: .5rem 1.2rem; font: inherit; }
|
||||
@@ -300,8 +330,10 @@ ${lastActivity}
|
||||
<h2>Open questions</h2>${openQs || '<p class="hint">(none)</p>'}
|
||||
${resolvedQs ? `<h2>Resolved questions</h2>${resolvedQs}` : ''}
|
||||
<h2>Agreed Contract (v${contract ? contract.version : 0})</h2>
|
||||
${agreementChips(o.participants, contract)}
|
||||
<pre>${escapeHtml(contract ? contract.markdown : '(not drafted yet)')}</pre>
|
||||
<h2>Conversation</h2>${msgs}
|
||||
${nudgeDetails(o)}
|
||||
<p class="hint">Full activity timeline: joins, messages, questions, resolutions, contract proposals and agreements. Read-only observer view; refreshes every 10s. Also available as Markdown: append <code>.md</code> to this URL.</p>`,
|
||||
o.room_status === 'open',
|
||||
);
|
||||
@@ -453,6 +485,7 @@ export function roomHtmlPage(v: RoomView, token: string): string {
|
||||
<h2>Open questions</h2>${openQs || '<p class="hint">(none)</p>'}
|
||||
${resolvedQs ? `<h2>Resolved questions</h2>${resolvedQs}` : ''}
|
||||
<h2>Agreed Contract (v${contract ? contract.version : 0})</h2>
|
||||
${agreementChips(v.participants, contract)}
|
||||
<pre>${escapeHtml(contract ? contract.markdown : '(not drafted yet)')}</pre>
|
||||
<h2>Conversation</h2>${msgs}
|
||||
<p class="hint">Full activity timeline: joins, messages, questions, resolutions, contract proposals and agreements.</p>
|
||||
|
||||
Reference in New Issue
Block a user