Evaluation

A full-stack run is scored exactly as an end-to-end run is: an automated validation pass that catches gross failures cheaply, followed by a hand-written review by a person who plays the build and assigns the numbers — a score in points and a quality rating per scoring domain. The one thing full-stack adds is that the reviewer also judges the assets the model produced, because in a full-stack run the model made them.

Because the mechanics are the same as end-to-end, this page covers only how full-stack differs; read End to End → Evaluation for the shared detail, and the Core docs it points at — Validation and Results — for the underlying mechanisms.

Validation is unchanged

The automated pass is identical to end-to-end. The load check builds the produced implementation with the manifest’s required [build] commands, serves the static output, loads it in a headless browser, and records whether it runs at all — a program that never loads is the clearest possible negative signal. Checks (opt-in reference comparison), proofs (submitted evidence, recorded but not graded), and instrumentation — the debug API a run is driven through, whose contract a build must satisfy or fail the run — all work exactly as they do for an end-to-end case.

Crucially, there is no asset-generation-style validation of the produced assets — no action-log regeneration and no cheat detection. Those exist in an asset-generation run because the asset is the scored output there. In a full-stack run the produced files are build inputs: they are exercised only by being loaded and played inside the running program, and they are judged as part of that program by the reviewer.

Review

The real evaluation is the review, carrying the same three things as an end-to-end review: a writeup, a rating per scoring domain (one of flawless, great, passable, scuffed, or broken, for each [[domain]] in the run’s effective set, the overall rating being the worst across them), and a checklist of binary pass/fail verdicts, one per [[review_item]] (or one per sub-item for an item that declares them).

What full-stack adds is that the reviewer, playing the build, is judging assets the model produced rather than assets a case provided. The quality of the produced assets — the art, the sprite motion, the particle effects, and the sound — is part of the experience being rated, not a separate score. A case makes this explicit by wording its review items and scoring domains to cover the asset dimensions that matter (a domain or item for the art direction, for the feel of the effects, for the audio), the same way an end-to-end case words items for the mechanics it cares about. A build whose code is solid but whose assets are placeholder rectangles or silence — the very thing the quality directive warns against — is unfinished work and should be rated as such.

Scoring

Scoring is identical to end-to-end: each [[review_item]] carries a point weight, a pass earns it and a fail earns none, and the run’s score is the earned weight over the total declared weight. The score and the overall rating are shown together on the run, and the case’s leaderboard ranks models by points. Publishing refuses a run with no review, so every published full-stack implementation is both scored and framed by a human who played it and judged the whole product — code and assets alike.