Ticket #256: live harness flake policy — verification
- Branch:
ticket/256-flake-policy - Base:
integration/prod-partials(188fe5e, including #257/#258/#259/#264) - Spec: #255, ticket #256
- Policy: flake-policy.md
- Verified: 2026-11-16 on Node 26.12.1 / OpenCode 2.2.7
What changed
- The wizard alignment check waits for step readiness (
live_require_ready_text) and the typed field value (live_type) before Enter. Fixed sleeps around wizard transitions were removed; Enter is always a separate input event. live_typeobserves the last four non-space characters of the typed text plus a pane change, so a wrapped long line still proves the field committed.scripts/live-matrix.shclassifies a failed attempt from the harnessfailure.jsonrecord. Only atimingfailure retries, once. Product, unknown, and infrastructure failures never retry. Skips keep the existing local-skip / require-mode-fail contract.- The harness writes redacted pane captures, logs,
failure.json, andfailure.txtunderENSEMBLE_LIVE_ARTIFACTSon failure. CI uploads the directory. scripts/flake-quarantine.jsonentries require a linked issue and an owner; the matrix refuses to start with an invalid entry.src/v2/flake-policy.tsowns the pure classifier, retry decision, quarantine validation, and report.src/v2/flake-policy.test.tsandsrc/v2/live-matrix.test.tscover the policy and the matrix loop with stub scenarios.- The mixed-anchor assignment check reads the durable system message instead of the visible chat cell: a long streamed reply can scroll the older assignment line out of the viewport even though routing is correct. The Work rows, replies, and exact instruction remain asserted separately.
- Header assertions use the compact
#263prefix (… · N active) because the merged three-pane width may drop thedonesegment while keeping the page and counts.
Evidence
| Check | Result | Notes |
|---|---|---|
| Alignment scenario, 20 consecutive runs | PASS 20/20 | Merged tree; exact readiness/Enter block |
ENSEMBLE_LIVE_REQUIRE=1 npm run test:live:fake:all | PASS | fake + empty suites; 0 retries, 0 failures; budget=ok |
npm run verify | PASS | typecheck + 1,989 unit tests |
| Synthetic timing failure | PASS | classified timing; retried once; report names suite/attempt/classification/artifact |
| Synthetic product failure | PASS | classified product; never retried |
| Invalid quarantine | PASS | missing issue/owner refused before the run |
| Failure artifacts | PASS | redacted pane/log capture plus failure.json |
Merge notes
integration/prod-partials was merged into the ticket branch. The only conflict was the docs/README.md verification index; both the integration entries and the flake-policy entries were kept. The merged header compaction (#263) required the live header assertions above to accept the compact form.
Skips
- None in the final fake matrix.
ENSEMBLE_LIVE_REQUIRE=1makes any skip a failure.
Privacy
- Artifacts are ephemeral local/CI diagnostics, not committed fixtures.
- The harness redacts the live base, live home, live project, caller home, and the server password; credential-looking lines are dropped.
- No conversation content, credentials, file bytes, or private paths are committed.