Ticket #241: Vera regression and cutover verification
- Date: 2026-11-11
- Branch:
ticket/241-review-fixes(review-fix pass onticket/241-vera-verification) - Base:
integration/native-first-defaultsat514ac86(includes #236–#240 and the original #241 merge) - Spec: parent #235; ticket #241
- ADRs: 0006, 0011
- Unit gate:
npm run verify— strict typecheck plus 1,868 tests, 0 failed, 0 skipped - Live gate:
npm run test:live:vera— seed 1/1, run 54/54, restart 18/18, exit 0 - Host matrix: OpenCode 2.2.7, Node 26.12.1, npm 12.2.1, tmux 3.5a, Debian 13
- Plugin:
ensemble0.1.0, shared-service version 3, contract version 6 - Provider: isolated local
ensemble-fakeOpenAI-compatible provider; the real suite also ran on the local catalog, no external network
Scenario
scripts/live-vera-regression.sh (npm run test:live:vera) runs one isolated fake host three times against a real temp workspace:
- Seed. Create Vera (leader), Clark, Kakiko, and Phoenix with stored titles and roles, create a workspace whose
memberIDsorder differs from agent creation order, warm the group binding, and seed a stale v5AGENTS.mdblock plus a project-owned line. - Upgrade shape. Stop the host and remove only the persisted
nativeFirstkey from the isolated plugin store. The fixture reports row counts and the removed key only; it never reads or copies message content. - Run. Restart, assert the absent value resolved to
onand was persisted, assert the generated v6 roster table, drive the leader turn, and exercise rollback and restore. - Restart. Restart again and assert durable turns, ledger, roster, and the no-completion result.
The explicit delegation uses the documented :tools legacy surface while native-first stays on: the native tool list deliberately omits ensemble_delegate, so the migration override is the only way one native turn can call it. Native ingestion, routing, the contact ledger, and completion stay on the native path.
Review-fix pass
The two-axis review of the #241 cutover produced these code and harness fixes. Behavior changes are covered by the unit/service suites; the live results below were re-recorded on this branch.
- One native-first default.
DEFAULT_NATIVE_FIRST_MODEis the single exportedonconstant used byDEFAULT_SETTINGS,nativeFirstMode, and the one-time store migration. An explicit persistedoff/shadowis never rewritten; absent or invalid values migrate to the default. - Explicit-tool dedupe. After the existing idempotency lookup, delegate, handoff, and group-post/approval paths check the native turn's
contactedledger. A recipient already contacted in the turn gets a clear "already contacted this turn" result and no second child or approval. - Anchor validation. The pure anchor resolver now requires the candidate task to belong to the calling workspace and agent (including a workspace direct
dm:<agent>:<workspace>turn). The group-channel rule that a workspace post must not borrow a DM turn remains a separate documented condition insend_message. - Anchor cleanup. The unused
taskMaxHopsfield was removed fromActiveTurnAnchor; the contact ledger remains and is the dedupe source. - Typed contact source.
NativeContactSourceis a domain union; the durable ledger stores recipient IDs only and the source code stays in thenative.contact.recordedlog event. - One focus model. Card focus registers every rendered card at successive
WORK_CARD_ROWS, matching the renderer'ssection.cards; one-card pagination makes this a no-visual-change agreement. - One command catalog. Command arity lives on
CommandSpec; parsing and completions derive fromCOMMANDS.:native-first onchanges only the mode and never clears an explicittoolSurface: legacy;:tools nativeis the separate restore. - Harness exactness. The Vera driver imports the product
teamRosterTablerenderer (run with--experimental-strip-types) for the expected table, keeps an independent leader-first/memberIDs ordering assertion, and applies the realnativeFirstCommand/toolsCommandpatches throughsettings.update. The TUI key handler is a thin wrapper covered bysrc/v2/tui/commands.test.ts. - Default-on service coverage. A fresh store with no seeded settings starts
on, persists the default, and registers the native surface.
The broad test:live:fake walkthrough predates default-on and asserts the legacy direct, DM-queue, and statusline surfaces. It now runs that scenario on the explicit legacy path; the Vera regression covers the default-on cutover, and the unit/service suites cover the default itself. This keeps the legacy assertions meaningful instead of rewriting them to the new surface.
The dispatch-budget probe inside test:live:multilocation also predates default-on and drives the explicit ensemble_handoff/ensemble_delegate tools, which the native surface deliberately does not register. It now pins nativeFirst: "off" before each scenario and waits for the mode to land, so the budget ledger is exercised on the legacy surface it was written for.
Regression assertions
Default-on restart from an unset store
- Before restart: one
v2/metarow, a persistednativeFirst, exactly one removed key, and nonativeFirstkey afterwards. - After restart: the store contains
nativeFirst: "on", the diagnostic reportson, andagent.sessionBindingreportsmode: "on". - The
native.modelog reportsmode: "on"andmigrated: true.
Generated roster table
The real workspace AGENTS.md contains one v6 block and this exact table (fixture titles/roles; leader first and marked, then memberIDs order):
| Member | Title | Role |
|---|---|---|
| Vera (leader) | Vera Lead | Owns the Vera regression and release sign-off. |
| Clark | Route Checker | Confirms each mentioned member receives exactly one turn. |
| Kakiko | Completion Guard | Checks that no second completion turn reaches the leader. |
| Phoenix | Roster Keeper | Keeps the generated team roster exact. |
The expected table is the product teamRosterTable renderer's exact output (imported by the driver under --experimental-strip-types), and a separate assertion parses the rendered member names to prove leader-first then memberIDs order. The project-owned line outside the Ensemble markers survives and the stale v5 text is gone.
One leader turn, no duplicate route, no completion bounce
| Observation | Result |
|---|---|
| Leader native turns for the prompt | 1, workKind: "turn", phase replied |
| Successful execution events on the parent | 1 |
| Parent user rows / final agent rows | 1 / 1 |
| Final answer length | 76 chars (length only; no text recorded) |
| New work items | 3 |
| Phoenix (explicit delegation) | 1 work item, workKind: "delegation", 0 peer turns |
| Clark (final-text mention) | 1 peer turn |
| Kakiko (final-text mention) | 1 peer turn |
| Parentage / dispatch group | every route parented to the leader turn in its group |
| Contact ledger | 3 recipients in one read-back |
native.route.suppressed | 1 already-routed for Phoenix |
completion.skipped | 3 total: 1 parent-is-leader, 2 asker-is-leader |
| Completion tasks anywhere | 0 |
| Leader-authored rows after the turn | 1 (the parent final row) |
Rollback and restore
nativeFirst: "off"plustoolSurface: "legacy"reportsoffand runs one non-native legacy task toreplied; the provider request carries all 11 legacy Ensemble tool schemas. The driver importsnativeFirstCommandandtoolsCommand, validates each, and sends their patches throughsettings.update, so the live scenario exercises the same command catalog the TUI uses. The TUI key handler is a thin wrapper covered bysrc/v2/tui/commands.test.ts(#240).- Every durable native turn, the parent result row, and the three-entry contact ledger survive the rollback.
- Restoring runs
nativeFirstCommand(["on"])andtoolsCommand(["native"]); the mode command alone never clears an explicit legacy surface. The restore reportsonand keeps the same turns and ledger. - After the second host restart: mode
on, native synclive, parent phasereplied, ledger unchanged, one successful execution event, one Phoenix work item, no completion task, no stuck turn, the exact roster table, one parent row, and one leader-authored row after the turn (the intentional legacy rollback row is excluded). Total work rows are unchanged.
AGENTS block prompt/token delta
Values use the documented chars/4 estimate. The generated Vera block is read from the real workspace file; the generic v6 value strips the roster section.
| Measurement | Chars | Est. tokens |
|---|---|---|
| Generated v6 block with the 4-member Vera roster | 4,509 | 1,128 |
| Generic v6 block without the roster section | 4,058 | 1,015 |
| Generated roster table | 367 | 92 |
| #238 fixed 12-member filled v6 block | 5,667 | 1,417 |
The Vera block is 1,158 chars / 289 tokens smaller than #238's 12-member fixture because it carries four members, not twelve. The roster table costs 367 chars / 92 tokens in this workspace.
#240's default-on native surface is still the five-tool decision set; the rollback request measured the 11-tool legacy list. npm run benchmark:tools on this branch reports 1,206 → 461 tool-schema tokens (11 → 5 tools), 3,347 → 1,879 total prompt tokens, 1 → 0 sender tool calls, and 2 → 1 sender round-trips.
Work pane size matrix
npm run test:live:sizes passed at 80x24, 100x30, 120x40, and 160x50, including the NO_COLOR=1 check. The new assertions extend the existing matrix:
- At 160x50, page one renders exactly four card rows; every top border starts at the Work content edge, ends at the content right edge, spans the full content width, and stands alone on its row.
- At 120x40, 100x30, and the 80x24 drawer, the pane renders six/five compact rows, zero card borders, and no full-width card rows.
One targeted 160x50-only probe hit the pre-existing NO_COLOR still-motion assertion transiently (a spinner glyph was captured); the full matrix rerun passed all sizes and the NO_COLOR check.
Commands
| Command | Result |
|---|---|
npm run verify | PASS — strict typecheck; 1,868 tests, 0 failed, 0 skipped |
npm run test:live:native-first | PASS — main 66/66, restart 18/18, 0 gaps, exit 0 |
npm run test:live:sizes | PASS — 80x24, 100x30, 120x40, 160x50, NO_COLOR=1, exit 0 |
npm run test:live:vera | PASS — seed 1/1, run 54/54, restart 18/18, exit 0 |
npm run test:live:fake:all | PASS — fake 463s, fake-empty 9s; 2 passed, 0 failed, 0 skipped |
npm run test:live:all | PASS — fake 461s, fake-empty 8s, real 57s; 3 passed, 0 failed, 0 skipped |
npm run test:live:multilocation | PASS — 22/22 budget hops, 1 continuation, 1 exhausted group, 0 mismatches |
npm run benchmark:tools | PASS — measurement only; values above |
Skips
- None. The real-model live suite ran on the local catalog and passed (
test:live:all, real 57s); no scenario reported a skip. - The broad fake walkthrough runs the explicit legacy path as described in the review-fix section; it is a pass for that configured scenario, not a claim that the legacy assertions cover the default-on surface.
Privacy
The committed record contains fixture names, stored roster titles/roles, counts, lengths, versions, and token estimates only. It contains no prompt or answer text, conversation transcript, credentials, file bytes, or absolute private paths. The live scenario gates its driver outputs for the harness root, project directory, password, prompt markers, and answer markers; it gates service logs for the harness home paths and the prompt marker.
Notes
- The original #241 cutover changed test infrastructure only. The review-fix pass adds narrow product changes (the canonical default constant, explicit tool dedupe, anchor validation, typed contact source, card-focus agreement, command arity, and the mode/tool-surface separation) plus the service and pure tests listed above.
- The expected
scripts/lib/native-first-driver.mjsconflict did not occur: the original branch started from integration at7b405ad, which already contains #240's fresh-store default assertion, and that file was not modified. The review-fix branch is based on integration at514ac86. - The fake walkthrough's explicit legacy setting is a fixture choice, not a product default:
DEFAULT_NATIVE_FIRST_MODEremainson, and the Vera regression plus the default-on service test prove the effective default.