Ticket #261: full deterministic E2E in CI — verification
- Branch:
ticket/261-ci-matrix - Base:
integration/prod-partials(4a84aa8, merge of #260) - Spec: #255, ticket #261
- Host: Linux/tmux, Node 26.12.1, npm 12.2.1, OpenCode 2.2.7, tmux 3.5a, fd 10.2.0
- Gate:
npm run verify(typecheck + unit suite),npm run docs:check
What changed
verify.ymlis split by cost intoverify,live-fake,live-native,live-multilocation, andlatency-smoke. Every live job runs in parallel, setsENSEMBLE_LIVE_REQUIRE=1, writes redacted artifacts to$RUNNER_TEMP/ensemble-live-artifacts, and uploads them on failure. The workflow has aconcurrencygroup that cancels superseded PR runs.nightly.ymlownslatency-reference(the strict 50/250 ms profile) andreal(test:livewith the real host catalog). The real job is opt-in via therealdispatch input orENSEMBLE_NIGHTLY_REAL=1; the schedule is17 4 * * *withworkflow_dispatchavailable..github/actions/live-host/action.ymlinstalls the pinned toolset once: Node, tmux, fd 10.2.0, OpenCode from the input version, andnpm ci. Every live job in both workflows uses it, so the host pin cannot drift job by job.verify.ymlpins OpenCode 2.2.7, the latest host inSUPPORTED_OPENCODE_VERSIONS;compatibility.ymlcontinues to run 2.2.6 and 2.2.7. The oldlatency.ymlis removed: the smoke profile is inverify.ymland the reference gate is innightly.yml.src/v2/ci-workflows.tsis the pure job-graph checker. It owns the job budgets, suite ownership, host-pin check, and repository check.src/v2/ci-workflows.test.tsmutates workflow text to prove a missing suite, a duplicated suite, a missing require mode, a missing artifact upload, an over-budget or missing timeout, a stale host pin, a missing concurrency block, a non-reference nightly profile, a non-opt-in real job, a PR-triggered nightly workflow, and a hardcoded action version all fail.scripts/live-lifecycle-fake.shnow matches the compact Work header (WORK ACTIVE │ N group(s) · N active) introduced by #263's 53-column card-mode floor. This is a test-harness assertion fix, not a product change; the group label still carries the active/done counts.
No product behavior changed. The skip-versus-fail contract is unchanged: local prerequisites skip, ENSEMBLE_LIVE_REQUIRE=1 turns a skip into a failure, and only a classified timing failure may retry once.
Job graph and time budget
PR and push (verify.yml)
| Job | Suites | Timeout | Observed |
|---|---|---|---|
verify | npm run verify | 15 min | ~1 min |
live-fake | test:live:fake:all, test:live:sizes, test:live:lifecycle, test:live:subagent | 45 min | ~16 min |
live-native | test:live:native-first, test:live:vera, test:live:tool-surface | 20 min | ~3 min |
live-multilocation | test:live:multilocation | 20 min | ~3 min |
latency-smoke | test:live:latency (smoke: 500 ms rail, 5000 ms card) | 20 min | ~3 min |
The live jobs run in parallel, so the PR critical path is live-fake plus setup: target 20 minutes, budget 30 minutes wall clock. The deterministic suite sum is about 15 minutes locally (fake 419 s, lifecycle 334 s, sizes ~57 s, multilocation 71 s, subagent 19 s, native-first 10 s, Vera 19 s, tool-surface 5 s, latency 8 s); tool setup and npm ci add the rest.
Nightly (nightly.yml)
| Job | Suites | Timeout | Observed |
|---|---|---|---|
latency-reference | test:live:latency (reference: 50 ms rail, 250 ms card) | 30 min | ~3 min |
real | test:live with the real host catalog, opt-in | 30 min | ~3 min |
Both nightly jobs together stay within a 60-minute wall-clock budget.
Kept separate
compatibility.yml runs the host probe and fake smoke on OpenCode 2.2.6 and 2.2.7; node-matrix.yml runs npm run verify on Node 22.23.4, 24.22.1, and 26.12.1; docs.yml runs npm run docs:check.
Evidence
All live commands ran with ENSEMBLE_LIVE_REQUIRE=1 on the isolated fake host.
| Check | Result | Notes |
|---|---|---|
npm run verify | PASS | strict typecheck; 2,024 tests, 0 failed/skipped |
npm run docs:check | PASS | contract v7; OpenCode 2.2.6, 2.2.7; Node >=22.23.4 |
npm run test:live:preflight -- all | PASS | Node 26.12.1, OpenCode 2.2.7, tmux 3.5a, fd 10.2.0 |
npm run test:live:fake:all | PASS | 419 s |
npm run test:live:sizes | PASS | 80x24/100x30/120x40/160x50 + NO_COLOR |
npm run test:live:lifecycle | PASS | 334 s after the compact-header assertion fix |
npm run test:live:subagent | PASS | 19 s |
npm run test:live:native-first | PASS | 10 s |
npm run test:live:vera | PASS | 19 s |
npm run test:live:tool-surface | PASS | native 6 tools vs legacy 11 |
npm run test:live:multilocation | PASS | 71 s |
npm run test:live:latency | PASS | 8 s, smoke profile |
npm run test:live:latency:reference | PASS | rail p95 12 ms, patch-card p95 39 ms; first-token 680 ms report only |
The CI job-graph check runs as part of npm test (17 cases, including the repository check). The workflow files were not executed on GitHub Actions from this worktree; the local suites above are the deterministic evidence.
First attempt
The first test:live:lifecycle run failed after 20 s waiting for WORK ACTIVE │ 1 group · 1 active · 0 done. The captured pane showed the compact post-#263 header (WORK ACTIVE │ 1 group · 1 active │ page 1/1) with the group label GROUP 1 · 1 active · 0 done directly below. The assertion expected the pre-#263 full header. The harness now matches the common prefix; the rerun passed in 334 s. No product code or layout changed.
Skips
test:live(real model) was not run: it is the opt-in nightly job and needs the runner's real catalog.- The compatibility matrix was not re-run locally: it needs OpenCode 2.2.6 in addition to the installed 2.2.7. The workflow and its probe/fake steps are unchanged.
- GitHub Actions itself was not executed; the job graph is validated by the pure checker and the suites by the local runs above.
Privacy
No conversation content, credentials, file bytes, or private paths are committed. Failure artifacts use the #256 redaction path; the live fixtures carry IDs, counts, lengths, and digests only.
Merge notes and conflict risk
integration/prod-partials is merged into the ticket branch before commit. Shared files with the other partials: scripts/README.md, README.md, docs/README.md, and the workflow directory. latency.yml is deleted, so a later edit to that file conflicts by design; #260's smoke/reference jobs now live in verify.yml and nightly.yml. The lifecycle assertion change is a one-line-per-assertion edit in a file #263/#264 also touched; conflict risk is low but non-zero.