Adding or Updating a Model

Overview

The model catalog is the list of subjects a run can be attributed to: the models the suite drives through a harness. This guide covers how the catalog is owned and served, how a model comes to appear, and how to curate one. The Add or Update a Model quickstart is the same task reduced to its steps.

Where the catalog lives

The catalog is owned by the backend. Model records, their aliases, and the catalog facts observed about them live in the backend store as the model, model_alias, and model_price tables. The backend serves the catalog at GET /models and bakes it into the public R2 snapshot as models.json, pointed to by the snapshot index.json’s modelsKey, so the static gallery renders model metadata and list prices without a backend round-trip.

Curating a model is an in-app edit that takes effect immediately, with no recompile and no release.

Curated and derived models

Every model with at least one recorded run appears in the Models section automatically:

• A derived model shows under its canonical model id, resolved from the run record. The openrouter/ routing prefix is stripped for the harnesses that emit it (OpenCode and Kilo Code), and a trailing OpenRouter variant tag such as :free is stripped for the OpenRouter-routed harnesses (every harness except Codex, Claude Code, and Antigravity). That normalization collapses one underlying model onto a single entry.
• A curated model carries a Test-Cabinet-defined display name, provider, logo, description, OpenRouter slug, and one or more aliases. Its aliases attribute runs to it, so a curated entry absorbs the derived ids it covers.

Curating a model in the app

Curated configuration is edited in the Models section of the web console. It is a write, so it requires sign-in.

A model record has these fields:

• Display name, the Test-Cabinet name shown across the site and the web console. It is required and never auto-generated.
• Aliases, one or more run-record model ids this entry covers, each paired with the harness family it is usable with. An alias is globally unique: an id belongs to at most one curated model.
• Provider, for example Anthropic, OpenAI, or Google.
• Provider logo, supplied as an svgl.app https:// URL. The backend fetches and sanitizes the SVG server-side.
• Description, markdown prose shown on the model’s page.
• OpenRouter slug, the id OpenRouter lists the model under, used to read the model’s list price and catalog facts. It is separate from the aliases, so a model can carry it without ever being run through an OpenRouter harness.
• List price: the uncached input, cached input, and output rates per Mtok a run’s comparable cost is priced from, with the date the figures were taken. The backend keeps it current from OpenRouter (see List price), and Fill from OpenRouter seeds the three rates on the form. The three rates are saved together with their date or not at all, and a blank set on an existing model keeps the stored one.
• Developer provider, the OpenRouter provider name of the model developer’s own endpoint. Blank takes the provider whose provider_name on the endpoints listing matches the author segment of the model id, ignoring case and punctuation. Set it where the two differ (qwen/… served by Alibaba). Its standard endpoint’s rates are the first choice for the model’s list price.
• Quantization filter, on by default. Off accepts every endpoint at whatever level it declares, unknown included, for a model every provider serves at one precision nobody discloses: a closed model’s providers declare none, so the filter would otherwise leave it no candidate. The other filters still apply.
• Native quantization, the level every provider a gg run uses must serve the model at while the filter is on. Blank takes the highest level any endpoint of the model declares.
• Price ceiling, the input and output rates in USD per million tokens a provider must be at or below when OpenRouter lists no developer endpoint. It is unused while the developer endpoint is listed, since that endpoint’s own rates are the ceiling.
• Banned providers, the providers a gg run of the model never uses.
• Unknown-quantization providers, the providers accepted despite declaring unknown quantization. Every other unknown endpoint is left out.

The last six fields decide a gg run’s candidate list. See Choosing the providers a gg run uses.

Alias harness families

A single model is reported under different ids depending on the harness that ran it. Most harnesses route through OpenRouter and report the slug unchanged. OpenCode and Kilo Code prefix it with openrouter/, and Claude Code, Codex, and Antigravity report a provider-native id.

A slug is meaningful only to the harnesses that speak its namespace, so each alias is tagged with a harness family:

• Claude Code, for provider-native Anthropic ids.
• Codex, for provider-native OpenAI ids.
• Antigravity, for provider-native Google ids.
• Others (OpenRouter), for OpenRouter ids (provider/model), shared by Cline, Goose, Kilo Code, OpenCode, Pi, and gg.

Claude Opus 4.8, for example, carries claude-opus-4-8 under Claude Code and anthropic/claude-opus-4.8 under Others. Listing every form maps each run record’s subject.modelId back to the one curated entry. The family also lets the New Run and Coverage forms filter the model dropdown to the slugs the selected harness can launch. See Harnesses for the reporting details.

Filling the form in from OpenRouter

The form’s first field is the OpenRouter slug, and Fill from OpenRouter beside it looks the slug up in OpenRouter’s catalog and fills in the display fields:

• Display name and Provider, split out of OpenRouter’s own Provider: Model name, so Anthropic: Claude Sonnet 4.5 becomes the name Claude Sonnet 4.5 under the provider Anthropic. A name carrying no such prefix falls back to the slug’s author segment.
• Description, as OpenRouter publishes it. OpenRouter truncates long blurbs itself, so what lands in the field is what it serves.

The fill also seeds the three list-price rates by the list price resolution, which tries the developer provider currently entered on the form first. Enter the date the figures were taken.

When the id list is still untouched, the fill also claims the slug as the entry’s first alias under Others (OpenRouter). A list you have already put an id into is left alone.

Fill replaces the display fields rather than filling only the blanks, and runs only on an explicit press. Nothing persists until Save, so leaving the page discards a fill you did not want.

Fill leaves one thing alone: the provider logo comes from svgl.app and is picked by hand. The context window, release date, and input modalities are recorded by the backend itself. A slug OpenRouter does not list reports inline and changes nothing.

Two ways to add a model

1. Blank form. In the Models section, click Add model and fill the form, usually by entering the OpenRouter slug and pressing Fill from OpenRouter, then adjusting.
2. Seed from a run. Open a run of a derived model and click Add this model. The form is pre-seeded from that run’s model id as a starting alias, already tagged with the family of the harness that ran it.

Either way, adding goes through the form and an explicit Save, and the display name is required.

List price

A run’s comparable cost is computed from the list price on the model’s catalog entry and from no other price (see Metrics). The backend reads the list price from OpenRouter, taking the first of these that lists a complete rate:

1. The developer provider set on the entry.
2. The provider named by the model id’s author segment, anthropic for anthropic/claude-haiku-5.5.
3. The model’s own price in OpenRouter’s models listing (the pricing field of its /models entry).

The first two steps read the provider’s standard endpoint in the model’s endpoints listing, and a Flex endpoint listed beside it is skipped.

The third step’s figure is the one OpenRouter’s API reports for the model, which can differ from the “In / Out Price” on OpenRouter’s web page for the model. It is a third party’s rate. Where OpenRouter spells the developer differently from the model id’s author segment, such as qwen/… served by Alibaba or mistralai/… by Mistral, set the developer provider to get the developer’s own price.

A developer provider you set always decides the price while its standard endpoint lists a complete rate. The 24-hour refresh and the Refresh button both re-read that provider’s rate, and move to the later steps only when yours returns nothing.

The backend refreshes the list price of every curated model shortly after it starts and every 24 hours after that. An entry saved without a list price also has one filled the first time a run binding it is enqueued. A launch naming a model the catalog has no entry for, or one OpenRouter yields no rate for, is refused at enqueue with the reason named.

The Refresh button, beside Add model in the Models section, runs the same refresh on demand. The button is disabled while the refresh runs. When it finishes the model list reloads and the page reports how many models were updated, how many were confirmed unchanged, and how many OpenRouter had no rate for. A refresh that fails reports the error.

A list price entered by hand on the form is saved as entered and stands until the next refresh resolves a rate for the model from OpenRouter. A model OpenRouter does not list keeps its hand-entered price through every refresh. Saving the form without changing the list price fields leaves the price, its date, and its source as they were. The model’s Stats tab shows the list price with its date and source.

A refresh changes the catalog entry only. Runs keep the cost they were scored at.

Catalog facts

The backend records the model’s context window, release date, input modalities, and developer provider as OpenRouter reports them, which makes the catalog the single store of those facts:

• It observes them when a run completes, and again on the 24-hour periodic task that refreshes list prices.
• It records a first observation the moment a model first appears: when you save it here with an OpenRouter slug, when a run that binds it is enqueued, and at backend startup for every known model still missing one. This seeding is missing-only, so a model already on record is left to the two paths above. A model on record with no developer provider counts as missing.
• An observation is appended only when one of the facts changed.
• A :free-tagged OpenRouter run observes the model’s base listing. The free variant is a routing tag.

The context window is what a gg run’s window-fullness accounting and compaction trigger are measured against. When a gg run is enqueued the backend looks the window up here for every model the run’s capability set binds and pushes the figures onto the launch, so gg keeps no model table of its own.

A model with no observation yet is seeded at enqueue by the launch-time fetch. Should that not answer, the launch falls back to a per-model lookup for that one model. If neither can answer, the launch is rejected: gg assumes no default window, because a run measured against a guessed one reports the wrong thing while looking healthy. See Runs with no resolved window.

The input modalities (text, image, file, …) are shown on the model’s Stats tab under Specs as both the raw list and a plain Vision line reading either “accepts images” or “text only”. They travel to a gg run on the same launch as the context window and decide whether the agent may be shown the reference images a test case’s specs ship. See Reading images.

An unknown modality list is recorded as unknown and the run proceeds. gg treats an unannotated model optimistically and recovers if the provider refuses the image, so an empty list in the console means the modalities have yet to be observed.

Choosing the providers a gg run uses

OpenRouter is a gg run’s gateway and bill, and gg chooses which of a model’s providers serves each request. The backend builds each bound model’s candidate list when the run is enqueued, from the model’s endpoints listing read at that moment, filtered and ordered as the candidate list describes. The list rides on the launch, and gg moves down it when a provider fails a request past its retries or keeps missing its own cache.

The model’s Stats tab shows the list the next enqueue would build, with each candidate’s quantization, prices and recorded fault rate, or the reason it would refuse. The list shown assumes an agent that sets no reasoning; an enqueue whose agents set one also drops the providers that do not support reasoning.

A provider’s fault rate is its stalls, its unexpected cache misses and the turns that ended on its failed model calls (model_retry_exhausted, model_timeout, model_parse), over its calls, across every recorded gg run of the model. A provider no recorded run used has a rate of zero. The rate orders the candidates; it removes none. Removing a provider for good is the ban list’s job.

A model with no candidate refuses the enqueue with the reason, which names the filter that emptied the list. The usual fixes are on the model’s form: set the developer provider when the listing spells it differently, set the native level when the observed one is wrong, name a provider under unknown-quantization providers when its endpoint is known to serve the native level, switch the quantization filter off when no endpoint declares a level because the model’s precision is undisclosed, or set a price ceiling when OpenRouter lists no developer endpoint.

Updating an existing model

Open the model in the Models section, click Edit, change any field, and Save. The change is live at once, and the snapshot picks it up on the next publish.

Probing a model

A model probe is a responses-as-code readiness check, run from the model’s Probes tab. It answers whether the model can drive gg’s RaC mode at all: the backend replays gg’s RaC opening request against the model’s OpenRouter slug — the submit_program tool offered and forced, as gg sends it — across cases that pair two scenarios with several task prompts, per program-language arm. A baseline case checks that the model calls the functions whose documentation views are open; a missing-docview case checks that it opens a missing function’s documentation and stops rather than calling the function unread. A model can hold the call shape and still fail either discipline and waste every RaC run; the verdict says whether to run the model in RaC mode or keep it out.

Run a probe when a new model is introduced, and again after pinning a new provider or before committing to a language arm, since providers can serve the same model differently. The steps are in the Probe a Model quickstart, and the wire contract is at Model probes.

Next steps

• Run a Test Case accepts the model as a valid --model argument.
• Harnesses documents how each harness reports the model id an alias maps back to a curated entry.