Skip to content

v5.9 T2 verification — CI fake-live guardrail ​

Status: verify and deterministic live suite passed locally; hosted CI not run Date: 2026-11-05 Ticket: #107 Parent specification: #105 Branch: feat/v59-107 Feature branch base: bf2c54b; integration merge commit: 27ae7f0

This record covers the CI live job, the pinned host toolset, and the skip-versus-fail contract. It does not close the parent specification or this ticket. No product runtime behavior changed.

Environment ​

ItemValue
OSLinux
Terminal hosttmux 3.5a
Node26.12.1
npm12.2.1
OpenCode2.2.5
fdfdfind 10.2.0
Git2.47.3

The repository still type-checks against @opencode/plugin 2.2.1. The live checks exercise the host binary, which is 2.2.5 locally and is the version the CI live job installs.

Delivered ​

Acceptance areaResult
Verify job remains.github/workflows/verify.yml keeps the verify job (npm ci, npm run verify) unchanged apart from sharing the pinned Node version.
Deterministic fake/empty suite in CIA new live job runs on push and pull request and executes npm run test:live:fake:all (fake then empty catalog).
Pinned tool setupThe job pins Ubuntu 24.04, Node 26.12.1, OpenCode 2.2.5, and fd 10.2.0, and installs tmux from the runner image.
Missing tool fails, not skipsENSEMBLE_LIVE_REQUIRE=1 turns harness skips into failures; the aggregate runner also treats a SKIP row as a failure. Local runs keep the skip-and-exit-zero contract.
Preflightscripts/live-preflight.sh and npm run test:live:preflight resolve the binaries, print versions, fail on missing tools in require mode, and fail on a present-but-wrong pinned Node/OpenCode version.
Broken assertion fails the jobA temporary wrong TUI assertion made the require-mode matrix exit 1 with a FAIL summary; the file was restored immediately.
Real-model checks stay localThe live job runs only the fake and empty-catalog scenarios. test:live/test:live:all are not invoked in CI.
Documentationscripts/README.md documents tool setup, runtime, the CI flow, the skip-vs-fail contract, and local duration; the runtime guide and CONTRIBUTING.md link the same contract.

Commands ​

CommandResult
python3 -c 'import yaml; ...' .github/workflows/verify.ymlPass — YAML parses; jobs are verify and live; triggers are push and pull_request.
bash -n scripts/live-preflight.sh scripts/live-matrix.sh scripts/lib/live-harness.shPass — no shell syntax errors.
git diff --checkPass — no whitespace errors.
npm run verifyPass — strict type check, then 666 tests passed, 0 failed, 0 skipped.
ENSEMBLE_LIVE_REQUIRE=1 npm run test:live:preflightPass — node v26.12.1, opencode v2.2.5, tmux 3.5a, fdfind 10.2.0.
ENSEMBLE_LIVE_REQUIRE=1 npm run test:live:fake:allPass — fake 167s and fake-empty 8s, 2 passed, 0 failed, 0 skipped, 0 not run in 175s. A second run passed in 176s (168s + 8s).
Missing-tool require mode: HOME=<empty> PATH=<no opencode> ENSEMBLE_LIVE_REQUIRE=1 bash scripts/live-preflight.shExit 1 — fails with opencode is not installed or not on PATH (required; ENSEMBLE_LIVE_REQUIRE=1).
Missing-tool local mode: same command without ENSEMBLE_LIVE_REQUIREExit 0 — prints a skipping line, preserving local behavior.
Missing-tool require mode: same sandbox with bash scripts/live-matrix.sh fakeExit 1 — FAIL fake, NOT RUN fake-empty, no green summary.
Missing-tool local mode: same sandbox with bash scripts/live-matrix.sh fakeExit 0 — two SKIP rows, 2 skipped, as before.
Deliberately broken assertion: temporary DELIBERATELY-BROKEN-ASSERTION in scripts/live-check-fake.sh, then ENSEMBLE_LIVE_REQUIRE=1 bash scripts/live-matrix.sh fakeExit 1 — FAIL fake: TUI did not show: Ensemble page; NOT RUN fake-empty. The file was restored with no diff.

The local missing-tool and skip-mode checks used a temporary HOME without the nvm OpenCode binary while keeping the shell and system tools available. That exercises the same resolution path as an absent host tool; no real tool was removed or modified.

Intermediate run ​

The first ENSEMBLE_LIVE_REQUIRE=1 matrix run failed at the agent wizard (timed out after 10s waiting for agent title step) while the pane showed the next Agent role step. No assertion related to this ticket had run yet. The two subsequent full runs passed every scenario. This is recorded as an intermediate live interaction flake, not as evidence of a product regression; it was not retried automatically and no live assertion was weakened.

Not run ​

  • Hosted GitHub Actions: not triggered or executed from this environment. The workflow syntax, job steps, and equivalent local commands were checked; this record does not claim a passing hosted CI run.
  • Real-model live checks: intentionally not run for this ticket and not required in CI.
  • macOS/Herdr: unchanged manual check; this work is Linux/CI tooling.

Released under the MIT License.