> ## Documentation Index
> Fetch the complete documentation index at: https://docs.hue.run/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> To set up Hue in an application, fetch https://docs.hue.run/guides/agent-setup.md and follow it.
> Hue's project-data MCP server is https://mcp.hue.run/mcp. https://docs.hue.run/mcp only searches these docs and cannot read a Hue project.
> Never ask the user to paste a Hue API key into chat; they store it as HUE_API_KEY themselves.
> Install the exact SDK versions pinned on the page you're following, or on https://docs.hue.run/quickstart.md. Run the latest CLI without a project install as `npx -y @hue-run/sdk@latest`. In a project with `@hue-run/sdk` installed, use `npx hue`. Pin an exact version for reproducible CI runs.
> Use Hue product terms: eval set, case, evaluator, run and scoring run. Older APIs may say dataset, scorer or experiment; prefer the current SDK methods. See https://docs.hue.run/concepts.md.

# Hosted judges

> Create a model judge and run it on Hue to grade saved agent outputs against a rubric.

A judge uses a model to grade output against a rubric, such as whether an answer follows an instruction or addresses a request. Hue hosts the model call and records the result. Use a [verifier](/evaluations/hosted-verifiers) for rules that code can check directly.

## Create and run a judge

1. In **Evaluators**, choose **Create evaluator → Verifier** and enter a name.
2. Open the evaluator and choose **Publish version**. Select **LLM judge** as the **Evaluator type**.
3. Set the rubric, metrics, **Gateway model ID**, **Provider ID**, and **Evidence bindings**. Bind the input, output, expected values, or metadata the judge needs. Publish the version.
4. Pin that version in your run. A one-shot [CLI run](/evaluations/run-your-agent) executes pinned model judges by default; `--no-judges` or `--no-output` skips them:

```sh theme={null}
npx -y @hue-run/sdk@latest eval --set "<eval set>" --command "<start command>" --scorer-version "<judge version id>"
```

Creating a judge does not run it. These steps publish an **LLM judge** for saved outputs. The **Judge** creation option configures a World judge for [case outcomes](/evaluations/case-from-trace).

Your agent runs on your machine or CI. Hue grades the uploaded evidence. Only evidence bound to the judge is sent to its model. Missing required evidence produces a skipped result before a model call.

## Inspect and regrade

Open the run in **Runs** to inspect the judge's metrics and evidence. An unsuccessful quality verdict is a scored result, distinct from a model error or skipped check.

To compare judge versions, create another scoring run over the saved outputs without running the agent again. Your coding agent can use [Hue MCP's scoring tools](/agents/mcp-tools#runs) to create scoring runs and [judge tools](/agents/mcp-tools#judges) to queue and inspect hosted judge jobs.

## Concurrency and funding

Hosted judges default to 50 concurrent checks per project, configurable up to 100 through Hue support. Existing pauses and custom limits still apply.

Hue pays for hosted judge calls through your organization's [monthly AI budget](/guides/redaction#monthly-ai-budget), which defaults to \$1,000 and is shared with other Hue-funded model calls. Your agent's own model calls use your credentials. [Hosted verifier compute](/evaluations/hosted-verifiers#concurrency-and-funding) has a separate allowance.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.