Documentation quality gates and quickstart audit
Status: PASS for the automated docs contract, the packed quickstart smoke, the PR docs-rationale guard, and the reference/media/site gates. The human newcomer 10-minute usability audit remains unrun and is not claimed. Date: 2026-11-17 Ticket: #310 Parent spec: #301 Branch: ticket/310-docs-quality
This record covers the final documentation quality ticket: the extended docs:check contract, deterministic packed-install quickstart E2E, separate docs CI and live CI placement, the docs-not-needed pull-request guard, and the launch audit evidence. It is a docs/tooling change; no product behavior changed.
Environment
| Item | Value |
|---|---|
| Host | Linux, detached isolated tmux sessions; no desktop, pointer, or screen focus |
| Node | 26.12.1 |
| npm | 12.2.1 |
| OpenCode | 2.3.0 |
| Provider | local deterministic fake provider; no model request, API key, or real credential |
| Package | packed @heyimteee/ensemble@0.1.0 served by a loopback registry |
| VitePress | 1.6.4 with local search |
Docs contract
npm run docs:check now validates:
- the required page inventory and every user page's reachability from its section index;
- title, description, version scope, and H1/title agreement on every user-facing page, including historical release pages with their own version;
- every internal Markdown link and heading anchor across
docs/and the root Markdown files, plus raw HTML links in the README; - balanced code fences, a language on user-page fences, and
bash -nsyntax on everysh/bashblock with style-guide placeholders substituted; - canonical package, contract, host, and Node claims, including install versions and the site's stable-version label;
- generated-reference freshness, media, evidence-index links, and the VitePress docs-site build.
The default command runs the site build. Unit fixtures use the explicit --skip-site-build internal mode so a VitePress build does not run dozens of times inside the unit suite. CI runs the full default. A missing required page, link, anchor, or frontmatter field is a validation failure (exit 1), not an uncaught error; regression fixtures cover that distinction.
Quickstart E2E
npm run test:live:quickstart packs the package, serves only @heyimteee/ensemble from a loopback registry, installs it with opencode plugin add, and checks the installed dist entry and runtime dependency resolution in the isolated plugin cache. It then drives the TUI:
/ensemblereaches the emptyENSEMBLEpage.- The agent wizard creates Nova and then Rook.
- The workspace wizard creates
acme-noteswith both agents and Nova as leader; both move-home prompts are accepted. - The composer sends the first group message.
- The Work pane shows the card and settles it as
REPLIED; the durable snapshot contains one matching replied work item.
The local registry, pack, isolated HOME/XDG directories, and fake provider are harness setup. Every product transition above was driven and observed through the TUI. The synthetic screenshot/replay fixture from #309 is not this proof.
| Phase | Measured |
|---|---|
| pack | 4,216 ms |
| local registry | 272 ms |
| install | 13,583 ms |
| host start | 1,456 ms |
/ensemble | 1,684 ms |
| Nova wizard | 1,650 ms |
| Rook wizard | 2,105 ms |
| workspace wizard | 1,904 ms |
| message + Work card | 2,478 ms |
| total | 29,429 ms |
This is a deterministic timed smoke, not a human-usability study. A second run completed the same path in 34,858 ms, so the result is stable rather than a one-off.
CI and PR guard
.github/workflows/docs.ymlinstalls dependencies and runs the fullnpm run docs:check, including the site build, as a separate docs job.verify.ymllive-fakeownstest:live:quickstart; the CI job-graph test asserts the suite belongs to exactly that job..github/workflows/docs-rationale.ymlruns onpull_request_target, checks out the trusted base commit withpersist-credentials: false, requests onlycontents: readandpull-requests: read, reads the PR body and changed-file list as data, and never executes pull-request code or reads secrets.scripts/docs-rationale.mjsrequires a docs update or adocs-not-needed: <reason>line for user-visible source changes. Test-only, workflow, and docs-tooling changes are explicit exemptions. Empty rationales fail.
Final audit
| Check | Result |
|---|---|
npm run verify | 2175 tests pass, 0 fail, 0 skip; typecheck pass (baseline 2144) |
npm run docs:check | ok; full default includes the VitePress build; zero broken links or anchors |
npm run docs:media:check | ok (manifest, hashes, screenshots, cast, alt text, captions, Mermaid, headings) |
npm run docs:reference:check | ok: 6 native tools, 11 legacy tools, 17 : commands, 15 / commands, 9 generated pages; every catalogued surface documented |
npm run docs:site:build | 184 pages, 51 redirects, local search, sitemap, llms.txt, and llms-full.txt built |
npm run test:live:quickstart | PASS twice: 29,429 ms and 34,858 ms; sanitized logs and timing JSON preserved under /tmp/opencode |
| Private/credential scan | no credentials or private paths in new or user-facing docs; historical verification records retain pre-existing local workspace paths and were not rewritten |
| Human newcomer timing | not assessed; no human usability session was run, so the under-10-minute claim is not made |
Limits and exclusions
- The 29-second figure measures the automated packed-install path. Install, npm registry, and scripted typing latency can differ for a human; this record does not convert the smoke into a human usability result.
- macOS, Herdr, screen capture, pointer interaction, browser axe, and real-model checks were not rerun for this ticket. The #309 browser/accessibility record remains the evidence for that layer.
- The repository development tree's
npm ciaudit output (34 advisories: 2 low, 27 moderate, 1 high, 4 critical) is not a consumer package security claim. No dependency or audit result was changed here. - The blocked #296 upgrade work was not cherry-picked or merged; no unrelated dependency or runtime change is included.
- This branch adds the
docs-rationaleworkflow job but does not change the repository ruleset or mark it as a required status check. A maintainer must do that separately after merge if the policy requires it; the job itself fails the PR run when the rule is violated. - Historical verification records are public evidence and were not rewritten. They contain pre-existing developer workspace paths; this ticket adds no new private path, credential, conversation, or real user data.