Skip to content

Documentation quality gates and quickstart audit ​

Status: PASS for the automated docs contract, the packed quickstart smoke, the PR docs-rationale guard, and the reference/media/site gates. The human newcomer 10-minute usability audit remains unrun and is not claimed. Date: 2026-11-17 Ticket: #310 Parent spec: #301 Branch: ticket/310-docs-quality

This record covers the final documentation quality ticket: the extended docs:check contract, deterministic packed-install quickstart E2E, separate docs CI and live CI placement, the docs-not-needed pull-request guard, and the launch audit evidence. It is a docs/tooling change; no product behavior changed.

Environment ​

ItemValue
HostLinux, detached isolated tmux sessions; no desktop, pointer, or screen focus
Node26.12.1
npm12.2.1
OpenCode2.3.0
Providerlocal deterministic fake provider; no model request, API key, or real credential
Packagepacked @heyimteee/ensemble@0.1.0 served by a loopback registry
VitePress1.6.4 with local search

Docs contract ​

npm run docs:check now validates:

  • the required page inventory and every user page's reachability from its section index;
  • title, description, version scope, and H1/title agreement on every user-facing page, including historical release pages with their own version;
  • every internal Markdown link and heading anchor across docs/ and the root Markdown files, plus raw HTML links in the README;
  • balanced code fences, a language on user-page fences, and bash -n syntax on every sh/bash block with style-guide placeholders substituted;
  • canonical package, contract, host, and Node claims, including install versions and the site's stable-version label;
  • generated-reference freshness, media, evidence-index links, and the VitePress docs-site build.

The default command runs the site build. Unit fixtures use the explicit --skip-site-build internal mode so a VitePress build does not run dozens of times inside the unit suite. CI runs the full default. A missing required page, link, anchor, or frontmatter field is a validation failure (exit 1), not an uncaught error; regression fixtures cover that distinction.

Quickstart E2E ​

npm run test:live:quickstart packs the package, serves only @heyimteee/ensemble from a loopback registry, installs it with opencode plugin add, and checks the installed dist entry and runtime dependency resolution in the isolated plugin cache. It then drives the TUI:

  1. /ensemble reaches the empty ENSEMBLE page.
  2. The agent wizard creates Nova and then Rook.
  3. The workspace wizard creates acme-notes with both agents and Nova as leader; both move-home prompts are accepted.
  4. The composer sends the first group message.
  5. The Work pane shows the card and settles it as REPLIED; the durable snapshot contains one matching replied work item.

The local registry, pack, isolated HOME/XDG directories, and fake provider are harness setup. Every product transition above was driven and observed through the TUI. The synthetic screenshot/replay fixture from #309 is not this proof.

PhaseMeasured
pack4,216 ms
local registry272 ms
install13,583 ms
host start1,456 ms
/ensemble1,684 ms
Nova wizard1,650 ms
Rook wizard2,105 ms
workspace wizard1,904 ms
message + Work card2,478 ms
total29,429 ms

This is a deterministic timed smoke, not a human-usability study. A second run completed the same path in 34,858 ms, so the result is stable rather than a one-off.

CI and PR guard ​

  • .github/workflows/docs.yml installs dependencies and runs the full npm run docs:check, including the site build, as a separate docs job.
  • verify.yml live-fake owns test:live:quickstart; the CI job-graph test asserts the suite belongs to exactly that job.
  • .github/workflows/docs-rationale.yml runs on pull_request_target, checks out the trusted base commit with persist-credentials: false, requests only contents: read and pull-requests: read, reads the PR body and changed-file list as data, and never executes pull-request code or reads secrets.
  • scripts/docs-rationale.mjs requires a docs update or a docs-not-needed: <reason> line for user-visible source changes. Test-only, workflow, and docs-tooling changes are explicit exemptions. Empty rationales fail.

Final audit ​

CheckResult
npm run verify2175 tests pass, 0 fail, 0 skip; typecheck pass (baseline 2144)
npm run docs:checkok; full default includes the VitePress build; zero broken links or anchors
npm run docs:media:checkok (manifest, hashes, screenshots, cast, alt text, captions, Mermaid, headings)
npm run docs:reference:checkok: 6 native tools, 11 legacy tools, 17 : commands, 15 / commands, 9 generated pages; every catalogued surface documented
npm run docs:site:build184 pages, 51 redirects, local search, sitemap, llms.txt, and llms-full.txt built
npm run test:live:quickstartPASS twice: 29,429 ms and 34,858 ms; sanitized logs and timing JSON preserved under /tmp/opencode
Private/credential scanno credentials or private paths in new or user-facing docs; historical verification records retain pre-existing local workspace paths and were not rewritten
Human newcomer timingnot assessed; no human usability session was run, so the under-10-minute claim is not made

Limits and exclusions ​

  • The 29-second figure measures the automated packed-install path. Install, npm registry, and scripted typing latency can differ for a human; this record does not convert the smoke into a human usability result.
  • macOS, Herdr, screen capture, pointer interaction, browser axe, and real-model checks were not rerun for this ticket. The #309 browser/accessibility record remains the evidence for that layer.
  • The repository development tree's npm ci audit output (34 advisories: 2 low, 27 moderate, 1 high, 4 critical) is not a consumer package security claim. No dependency or audit result was changed here.
  • The blocked #296 upgrade work was not cherry-picked or merged; no unrelated dependency or runtime change is included.
  • This branch adds the docs-rationale workflow job but does not change the repository ruleset or mark it as a required status check. A maintainer must do that separately after merge if the policy requires it; the job itself fails the PR run when the rule is violated.
  • Historical verification records are public evidence and were not rewritten. They contain pre-existing developer workspace paths; this ticket adds no new private path, credential, conversation, or real user data.

Released under the MIT License.