v0.2.0 (2026-06-19)
v0.2.0 is a large structural release. Where v0.1.0 was the minimum needed to make The Test Cabinet work as a CLI plus a public gallery, v0.2.0 fills in most of the surrounding system: a centralized backend, remote execution, interactive runner/reporter consoles, reviewer workflows, and end-to-end observability.
This is still pre-1.0 software intended primarily for project developers, and much of the new surface area is rough. The goal for v0.2.0 is breadth — getting the major components in place and talking to each other — not polish.
Features
Section titled “Features”Centralized backend
Section titled “Centralized backend”The earlier “git-as-a-db” design — where run records were committed straight into the site’s dataset — has been replaced by a single private backend. It serves as the canonical source of test case definitions for runners and records published run results. The public site stays fully static: publishing exports a public snapshot that the gallery builds from, so the site keeps no live dependency on the backend. The backend stores definitions and results through a pluggable store — SQLite for local and development use, PostgreSQL for deployments (which keeps the backend stateless, with no single-replica or attached-volume requirement).
Remote execution with workers
Section titled “Remote execution with workers”A new worker component exposes the core’s run functionality over an HTTP API, so test cases can be executed on a remote machine instead of only on the operator’s own machine. Workers are the execution backend for the new web console.
Interactive runner/reporter consoles
Section titled “Interactive runner/reporter consoles”The Tauri desktop app is now a full runner and reporter: it launches runs, streams them live, reviews them, and publishes — all in one place. A new browser-based web console provides the same console experience backed by remote workers rather than a built-in local runner. Both the desktop and web UIs are now aligned with the public gallery’s look and feel.
Shared UI library
Section titled “Shared UI library”The frontend has been reorganized around a shared
@test-cabinet/ui package that hosts the full routed
gallery application plus the presentational primitives. The public site, the web
console, and the desktop app are now thin hosts over this shared code, which is
what keeps their UIs consistent.
Reviewer checklists
Section titled “Reviewer checklists”Test cases can now carry reviewer checklists. The desktop and web consoles present these as a guided review with a completeness gate, and verdicts are published inline with a run so they show up on the gallery’s verdict view. See test cases.
Live run experience
Section titled “Live run experience”Runs can now be watched as they happen: a live, virtualized harness-event feed with follow-along scrolling, a listing of in-progress runs, and a notifications bell. Completed runs surface their recorded events on the run detail page.
Harnesses installed at run time
Section titled “Harnesses installed at run time”Per-harness container images have been retired. Every run now executes in a
single shared base image (pulled by digest from GHCR) and installs the selected
harness’s CLI into the running container just before the session. Installing at
run time means a run always picks up the harness’s most recently published
version, with no image to rebuild when a harness ships an update. Each harness is
declared by a harness.toml manifest under
harnesses/
(slug, name, binary, install command), and run records pin the resolved
base-image digest for reproducibility. See
harnesses.
Per-run token and cost metrics
Section titled “Per-run token and cost metrics”Every run now records token usage and cost, extracted per harness from the harness’s own reporting and broken down by class — uncached versus cached input, output, and, where the harness reports it, reasoning. Cost uses the harness-reported figure when available and otherwise falls back to an OpenRouter price lookup. The gallery surfaces these as per-run metric tiles and a per-test-case token chart, so runs can be compared across harnesses and models. A token class a harness does not report is recorded as unknown rather than zero, so partial data is never charted as if it were complete. See metrics.
Normalized harness events
Section titled “Normalized harness events”The live and recorded event feeds now classify harness output consistently across every supported harness (Claude, Cline, Goose, Kilo Code, OpenCode, Pi). Model reasoning is captured as its own event kind — kept apart from visible messages and collapsed by default — and a system event type was added. Read, write, list, and command events are mapped uniformly even where a harness splits a single tool call across multiple stream events.
Proof of implementation
Section titled “Proof of implementation”Test cases can now ask an agent to submit evidence that a feature works — a screenshot or a short video clip written to a declared path. Validation records each proof as present or missing (informational only; a missing proof never changes a run’s status), and the reworked review UI walks the reviewer through one item at a time, pairing each submitted proof with the reference it targets. See test cases.
Starter workspaces and init commands
Section titled “Starter workspaces and init commands”A test case can now ship a starter workspace — files seeded into the run root before the specs — and an init command that runs inside the container, after seeding and before the harness starts. This lets a run begin against a prepared project rather than an empty repository; pong and snake use it to provide Playwright as a project-local dev dependency rather than a globally installed tool. See test cases.
Observability
Section titled “Observability”The Test Cabinet now ships opt-in OpenTelemetry instrumentation through a shared telemetry crate. The core, CLI, worker, backend, and desktop app emit traces and metrics with context propagated across process boundaries, the web console adds browser tracing, and the devcontainer can spin up a local Grafana LGTM stack for inspecting it all.
Deployment
Section titled “Deployment”v0.2.0 introduces documentation and templates for running The Test Cabinet’s always-on services — the backend and workers — as remote staging and production deployments, alongside guides for running everything locally, backups, and telemetry in production.
Bug Fixes
Section titled “Bug Fixes”A number of issues found while building out the above were fixed along the way, including run start/visibility failures, live-feed crashes and an infinite loop on run completion, serialization errors, reference images not displaying, auto-scroll disabling itself, a leaked path separator on Windows, and assorted readability and layout problems in the consoles.
A round of corrections to per-harness token accounting and event mapping also landed. Usage that was under- or over-counted was fixed for OpenCode and Pi (cached input was being dropped and Pi was capturing a single message instead of the run total), Kilo Code (dropped cached reads), and Cline (a cache double-count that roughly doubled both tokens and cost); OpenRouter price lookups for OpenCode and Kilo Code were corrected so comparable cost no longer silently degraded; and events that previously fell through as unknown were classified for Goose, Kilo Code, and Pi.
Known Issues
Section titled “Known Issues”Mobile support
Section titled “Mobile support”The shared gallery UI now carries responsive styling, but the console-heavy views (live runs, review) are still tuned for the desktop and remain rough on small screens.
Resolved from v0.1.0
Section titled “Resolved from v0.1.0”The Playwright browser-download hang reported in v0.1.0 is resolved: the project has moved to Playwright 1.61.0 and dropped the browser-download workaround.