Skip to main content
A verifier checks concrete rules in your agent’s output, such as an exact value, required text, or document structure. A hosted verifier runs on Hue in an isolated sandbox. Local code verifiers run in your own process, as shown in Local evaluations. Deterministic verifiers make no model calls. Use a hosted judge when a check needs a model to apply a rubric.

​
Connect your verifier

  1. Ask Hue support to review your private JavaScript or Python verifier and its dependencies. Hue approves its source and runtime before it can run.
  2. Use the approved evaluator version in your eval set’s run command. The version pins the code used to grade the outputs.
  3. Run your agent through the CLI, on your machine or in CI:
Your agent uses your own model credentials. After the CLI uploads the required outputs and files, Hue queues the verifier. You do not need to keep a verifier process running on your machine.

​
Read the results

Open the run in Runs to inspect metrics, individual checks, and their evidence. A failed quality check is different from a verifier error or missing evidence. Checks can finish after the CLI exits. You can rescore saved outputs with another evaluator version without running the agent again. Each scoring run keeps its pinned version and results.

​
Concurrency and funding

Approved hosted verifiers are enabled by default, with 50 concurrent checks per project. Hue support can configure up to 100. Existing pauses and custom limits still apply. Hue funds a $1,000 monthly compute allowance per organization. Verifier compute is separate from the monthly AI budget used by hosted judges. Your agent’s own model calls use your credentials.