Skip to content
CodeVetter docs
Esc
↑↓navigate↵open⌘Jpreview
On this page

Testing

The test and code-health surfaces and how to run each one.

Testing

CodeVetter has four test surfaces. CI runs all of them in .github/workflows/ci.yml.

1. Frontend unit tests (Node node:test + tsx)

cd apps/desktop
pnpm test:unit                                   # all src/**/*.test.ts
pnpm test:review-proof                           # review-proof only
pnpm test:agent-fix-packet                       # agent-fix-packet only
pnpm test:synthetic-qa                           # synthetic-qa fixtures
pnpm test:intent-debugger                        # intent-debugger report
pnpm test:coverage                               # c8 coverage

Runner is the built-in Node test runner via tsx — no Jest/Vitest. Add a *.test.ts next to the module it tests.

2. Playwright e2e (chromium)

cd apps/desktop
npx playwright install chromium   # first time only
pnpm test                         # full suite (starts Vite dev server)
pnpm test:e2e:ui                  # interactive UI mode
npx playwright test tests/e2e/smoke.spec.ts
npx playwright test -g "App loads without crashing"
  • Config: apps/desktop/playwright.config.ts
  • Browser: chromium only (single CI project)
  • Base URL: http://localhost:1420 (the Vite dev server)
  • Tests live in apps/desktop/tests/

Native Tauri UI qualification (manual)

Manual native qualification must not reuse the normal CodeVetter database. Start Vite in one terminal, then point the Tauri process at a fresh, explicit application-data directory in another:

cd apps/desktop
pnpm exec vite --port 1420 --strictPort

cv_qualification_dir="$(mktemp -d /tmp/codevetter-native-qualification.XXXXXX)"
CODEVETTER_NATIVE_QUALIFICATION=1 \
  CODEVETTER_APP_DATA_DIR="$cv_qualification_dir" \
  pnpm exec tauri dev --no-watch \
  --config '{"build":{"beforeDevCommand":"true","devUrl":"http://localhost:1420/review"}}'

The qualification flag disables transcript indexing and background watchers; CODEVETTER_APP_DATA_DIR isolates SQLite migrations and any explicit fixture writes. Neither replaces HOME nor isolates provider discovery, so keep screenshots scoped to the route under qualification and inspect them for local account metadata before retaining them.

3. Rust tests (cargo test)

cd apps/desktop/src-tauri
cargo test                                  # all unit + integration tests
cargo test mcp::                            # MCP protocol + safety tests
cargo test --release --test mcp_stdio       # release-mode stdio lifecycle
cargo test --release perf_bench -- --ignored --nocapture --test-threads=1   # benches
  • ~385 Rust tests + the MCP binary + real offline stdio integration.
  • Benches are #[ignore]d so they never gate normal cargo test.
  • CV_ENFORCE_GRAPH_BUDGETS=1 makes the real-repo structural bench enforce the release envelope — see performance.md.

4. Benchmark tests (Node node --test)

pnpm test:benchmark    # scripts/run-catch-rate-benchmark.test.mjs
pnpm bench:public      # 27 public cases, catch-rate/precision/F1

See benchmark.md.

Native Agent Island

On macOS, the Apple-framework-only helper has a framework-independent protocol self-test and focused Rust coverage:

cd apps/desktop
pnpm test:agent-island
cargo test --manifest-path src-tauri/Cargo.toml native_agent_island --lib
cargo test --manifest-path src-tauri/Cargo.toml agent_stream --lib
cargo test --manifest-path src-tauri/Cargo.toml claude_hook --lib

Architecture, privacy boundaries, and remaining release qualification are in native-agent-island.md.

CI order

ci.yml runs, in order: lint → code health → typecheck → unit tests → MCP sidecar build smoke → desktop build → MCP protocol/safety tests → MCP release-mode stdio lifecycle. A failure stops the pipeline.

Strictness gates

  • Biome is the linter/formatter (biome.json, root pnpm lint).
  • tsc --noEmit typecheck in apps/desktop.
  • Knip is the unused-code and unused-dependency authority (pnpm knip:strict).
  • Biome changed-file complexity caps new or modified functions at cognitive complexity 20 (pnpm quality:complexity). Historical over-threshold functions remain explicit in GitHub issue #117 for review by 2026-09-11 rather than being hidden by global suppressions.
  • Biome’s project scanner rejects runtime dependency cycles (pnpm quality:cycles). Type-only imports are excluded because they are erased before runtime.
  • jscpd prevents clone growth across authored desktop, landing, and script sources (pnpm quality:duplication). Generated artifacts and fixtures are excluded. The 2026-08-11 baseline is 0.75% duplicated lines: 1,000 of 133,900 lines across 356 files, with 72 clones at the 8-line/60-token floor.
  • Dependency audit blocks high and critical advisories across production and development dependencies (pnpm quality:dependencies). The remaining Astro 6 low/moderate baseline is owned by GitHub issue #116 for review by 2026-09-11 because its fix requires the Astro 7 major upgrade.
  • Clippy zero-warning in release qualification.
  • Bundle budgets via apps/desktop/scripts/bundle-budget.mjs.
  • OpenSpec strict validation for specs under openspec/.
  • Pre-commit (.husky/pre-commit): lint-staged runs biome check --write on staged apps/desktop/src/**/*.{ts,tsx}.
  • Pre-push (.husky/pre-push): runs pnpm lint + a secret-pattern scan over tracked files (with anchored exclusions for fixtures, benchmarks, and secret_policy.rs).

Adding tests

  • Pure logic → *.test.ts next to the module, Node test runner.
  • UI flow → tests/e2e/*.spec.ts, Playwright.
  • Rust behavior → #[test] in the module or tests/ integration target.
  • Benchmark regression → extend scripts/run-catch-rate-benchmark.test.mjs or add a #[ignore]d Rust bench in perf_bench.rs.

Was this page helpful?