TypeScript API reference

This reference covers @hue-run/sdk 0.27.0 on Node.js 22 or 24 and Bun 1.4. For installation and a runnable first trace, use the TypeScript guide. See compatibility for supported runtimes and integration versions. Use Quickstart for the available package release.

Imports

Entry point	Use
`@hue-run/sdk`	`createHue`, `createHueSafe`, `HueClient`, `createHueTransport`, `HueTransport`, `hueExperimentalTelemetry`, `Attachment`, `contentPrefixes`, `HueConnectionError`, `HueExportError`, `HueTraceVerificationError` and the tracing types below.
`@hue-run/sdk/ai-sdk`	`hueTelemetry` and `HueTelemetryOptions` for Vercel AI SDK 7. Requires the optional `ai` and `@ai-sdk/otel` peers.
`@hue-run/sdk/evals`	Evaluation client, recorded media reads, local runners including `runSimulation` and `runLocalAgent`, scorer definitions, file-based case helpers, published Scenario resolvers, verdict helpers, attempt-preflight validators, evaluation errors and types. Uses `zod`, a regular dependency since 0.16.0.
`@hue-run/sdk/world`	`createWorldClient`, `WorldClient`, `WorldModelClient`, `bindWorldTools` and the canonical World and instance types.
`@hue-run/sdk/environment`	`createEnvironmentClient`, `EnvironmentClient`, `bindEnvironmentTools`, `HueEnvironmentError`, the world handoff helpers and the simulated-environment types.
`@hue-run/sdk/managed`	`createManagedTargetHandler` and the managed-target types.

The package is ESM; CommonJS applications on Node.js 22.12 or later load every entry point with require(). The SDK runs on the server. It does not call a model provider for you or create model spans for uninstrumented provider calls. Every exported type, option and member carries API documentation in the published declarations.

Create a tracing client

import { createHue } from "@hue-run/sdk";

const hue = createHue({
  apiKey: process.env.HUE_API_KEY!, // Read the key from server configuration.
  serviceName: "my-agent",
  captureContent: true, // Record supported input/output content.
  captureHttp: (process.env.HUE_CAPTURE_HTTP ?? "true") !== "false",
  captureHttpBodies: (process.env.HUE_CAPTURE_HTTP_BODIES ?? "true") !== "false",
});

createHue(options: HueOptions | ExistingHueProviders): HueClient and new HueClient(options) accept the same configuration. HueOptions is SharedHueOptions, the options every client and transport accepts, plus enabled, apiKey and serviceName.

HueOptions

Option	Type	Default and behavior
`apiKey`	`string`	Required when enabled; ignored for `enabled: false`. Use Read and write for tracing, known-trace receipts and evaluation APIs. Keep it in server-only configuration.
`serviceName`	`string`	Required when enabled; 1–256 characters with non-whitespace content. Sets `service.name` on providers created by Hue.
`enabled`	`boolean`	`true`; `false` creates a disabled client without attaching processors or requiring a key.
`captureContent`	`boolean`	Optional; defaults to `true` for an enabled client. `true`, the recommended setting, records supported content; `false` sends metadata only. See content policy. A disabled client defaults it to `false`.
`captureHttp`	`boolean`	`true`. Records supported outbound HTTP CLIENT spans. `false` disables HTTP capture.
`captureHttpBodies`	`boolean`	Follows `captureContent`. `false` retains HTTP identity without request or response content. `captureContent: false` always disables HTTP content. See app HTTP exchanges.
`baseUrl`	`string`	`https://app.hue.run`. Supply an origin, without an API path, query, fragment or credentials. HTTPS is required except for loopback HTTP.
`allowInsecureHttp`	`boolean`	`false`. `true` permits `http://` to hosts other than loopback, for example a docker-compose or in-cluster collector, and records a one-time `warning` issue because the key then travels unencrypted.
`serviceVersion`	`string`	Optional `service.version` on providers created by Hue.
`resourceAttributes`	`Attributes`	Optional additional resource attributes for an owned client, for example `deployment.environment.name` or `service.namespace`; `serviceName` and `serviceVersion` take precedence over same-named keys. Ignored with a `warning` issue in attach mode.
`maxQueueBytes`	`number`	`8388608`; 1024–67108864. Shared estimated trace/log queue bytes, including in-flight work. Not total process RSS.
`timeoutMillis`	`number`	`10000`; integer from `100` through `60000`. Configures transport requests and the project connection check; it is not an overall `flush()` deadline.
`liveSpans`	`boolean`	`true`. Sends placeholders for open Hue and AI spans. A span still open when its placeholder is exported, usually about 1.5 s after it starts, shows as running. Setup credentials never send placeholders. See live spans.
`redact`	`(value: string, path: string) => string`	Optional string transformation on Hue’s export path. See content policy.
`onExportIssue`	`(issue: ExportIssue) => void \| Promise<void>`	Optional callback for sanitized export diagnostics, called at most once per second.

Invalid configuration throws TypeError. Environment variables are read by your application and passed to the client; these constructors do not automatically load an environment file, and OTEL_EXPORTER_OTLP_* variables do not reach Hue’s exporters. Owned providers export the OpenTelemetry default resource (telemetry.sdk.language, telemetry.sdk.name, telemetry.sdk.version) beside service.name, service.version and resourceAttributes. Export requests carry a hue-sdk-typescript/<version> User-Agent.

ExistingHueProviders

Use createHueTransport(options: HueOptions): HueTransport when your application owns OpenTelemetry setup. Attach its spanProcessor and logRecordProcessor during provider construction, then pass:

createHue({ transport, tracerProvider, loggerProvider });

tracerProvider is a FlushableTracerProvider and loggerProvider a FlushableLoggerProvider: the OpenTelemetry API provider types plus forceFlush(): Promise<void>, which the OpenTelemetry SDK providers satisfy. Your application owns their resources, registration and shutdown; resourceAttributes is ignored with a warning issue in this mode. Hue never replaces the global tracer provider, logger provider or context manager. See the complete existing-provider recipe.

Trace application work

The signatures below use types exported by @hue-run/sdk and standard OpenTelemetry types.

hue.withSpan<T>(
  name: string,
  callback: (span: HueSpan) => T | Promise<T>,
  options?: SpanOptions,
): Promise<T>

hue.tool<T>(
  name: string,
  input: unknown,
  execute: () => T | Promise<T>,
  options?: ToolOptions,
): Promise<T>

hue.model<T>(
  model: string,
  callback: (span: HueSpan) => T | Promise<T>,
  options: ModelOptions,
): Promise<T>

hue.getContext(): Context
hue.inject(carrier: Record<string, string>, activeContext?: Context): void
hue.extract(carrier: Record<string, string | string[] | undefined>): Context
hue.recordError(span: Span, error: unknown): void
hue.recordMessages(
  messages: {
    input?: unknown;
    output?: unknown;
    systemInstructions?: unknown;
    operation?: string;
    provider?: string;
    model?: string;
  },
  explicitContext?: Context,
): void
hue.recordProviderToolCalls(response: unknown, options?: ProviderToolCallOptions): void
hue.recordFile(file: FileRecord, explicitContext?: Context): void

withSpan returns the callback’s result, ends the span after the callback settles, and records then rethrows an escaping application error. Consume streams inside the callback if their work belongs to that span. A returned streaming Response is not evidence that the stream has finished. tool runs execute inside a span named execute_tool {name} with gen_ai.tool.name, records input and a defined result as gen_ai.tool.call.arguments / gen_ai.tool.call.result according to your content policy, and returns the tool result. An undefined result omits result content. options.parentContext supplies an explicit parent for the tool span. ToolOptions extends Pick<SpanOptions, "parentContext"> with callId, the provider-issued tool-call id recorded as gen_ai.tool.call.id, and mcp, an McpServerInfo (the MCP initialize serverInfo, passed as client.getServerVersion() after connect) recorded as mcp.server.name / mcp.server.version so a generic tool name is attributed to the server that handled the call. When the server is a Hue surface, McpServerInfo also takes provider and surface (for example google.gmail and google.gmail/mcp), recorded as hue.mcp.provider / hue.mcp.surface. A blank, over-256-character or otherwise invalid label is omitted and counted as an instrumentation failure; the tool still runs. ToolOptions.endpoint takes a ToolEndpoint with the actual HTTP(S) URL used by the tool:

interface ToolEndpoint {
  url: string;
  source: "request" | "client_config" | "receipt";
  method?: string;
  captureQueryParameters?: readonly string[];
}

With content capture enabled, Hue records the endpoint through the configured redactor. It removes credentials, fragments and query parameters unless their non-secret names are allowlisted. method records the actual method when known, with no default. The option records routing evidence supplied by your application; it makes no request and never infers a URL from a tool or server label. model runs callback inside a GenAI client span for one direct provider call, named {operation} {model} unless options.name is given, with gen_ai.operation.name, gen_ai.request.model and gen_ai.provider.name. The argument order matches withSpan. Inside it, the handle’s setInput and setOutput record gen_ai.input.messages / gen_ai.output.messages, which should use the GenAI semantic-convention message shape; recordMessages inside the callback inherits the request metadata. When you send system instructions or tools separately from the messages, pass them as systemInstructions and tools in ModelOptions. Content helpers (setInput, setOutput, tool arguments and results, recordMessages and SpanOptions.input) accept unknown and encode plain JSON data (JsonValue) at runtime. A value that is not JSON, such as a Date or a class instance, is omitted with an instrumentation failure while the callback result is returned unchanged. recordMessages emits a correlated gen_ai.client.inference.operation.details log record with the messages, and optional systemInstructions as gen_ai.system_instructions, in its body. gen_ai.operation.name, gen_ai.provider.name and gen_ai.request.model are set as record attributes from the caller’s operation, provider and model or from the enclosing model() span, and gen_ai.conversation.id from the active session. With content capture enabled, it requires an active span or explicit span context. With capture disabled, it emits nothing. Use logs or span attributes for the same content event to avoid storing duplicate copies. recordError marks a span failed the way every helper does: error.type (the error’s name), an ERROR status without a description and an exception event carrying only exception.type. Exception messages and stack traces are never recorded, whatever captureContent is. inject writes W3C traceparent for the active span into a carrier, never the API key or baggage, and also propagates when the client is disabled or closed. extract reads W3C trace context from a carrier for use as parentContext. Hue registers no global propagator.

SpanOptions and context

Option	Default and behavior
`kind?: SpanKind`	`SpanKind.INTERNAL`.
`attributes?: { [key: string]: AttributeValue \| AttachmentJson \| undefined }`	Additional attributes; arbitrary names and values are caller-owned. A JSON object, an array OpenTelemetry cannot hold or an `Attachment` is recorded as JSON text, each attachment as its digest part. `AttachmentJson` is JSON that may hold an `Attachment` anywhere.
`input?: unknown`	Optional root input, recorded as `input.value`. Omitted when content capture is disabled.
`sessionId?: string`	Inherits from the surrounding Hue callback; maps to `gen_ai.conversation.id`.
`userId?: string`	Inherits from the surrounding Hue callback; maps to `user.id`. This is your application identity, not a Hue account.
`workspaceId?: string`	Inherits from the surrounding Hue callback; maps to `hue.workspace.id`. The workspace or tenant in your application that the work runs in.
`parentContext?: Context`	Explicit parent, for example from `hue.extract()`; otherwise the surrounding Hue context or active OpenTelemetry context.

Session, user and workspace identifiers must contain 1–4096 valid characters. hue.tracer is an OpenTelemetry tracer that preserves Hue’s local async context; spans other instrumentations create through it parent under withSpan and inherit identifiers. hue.getContext() supplies the helper’s current context to an integration that accepts an explicit parent. When your application has registered a context manager, withSpan, tool, model and hue.tracer.startActiveSpan also make their span the active OpenTelemetry span while the callback runs; Hue registers none itself.

ModelOptions

ModelOptions extends the sessionId, userId, workspaceId, input and parentContext members of SpanOptions.

Option	Behavior
`provider: string`	Required. Recorded as `gen_ai.provider.name`, for example `"openai"`.
`operation?: string`	Recorded as `gen_ai.operation.name`; defaults to `"chat"`.
`name?: string`	Span name; defaults to `"{operation} {model}"`.
`input?: unknown`	Recorded as `gen_ai.input.messages` when `captureContent` is true.
`systemInstructions?: unknown`	System instructions sent separately from the messages, for example `[{ type: "text", content: instructions }]`. Recorded as `gen_ai.system_instructions` when `captureContent` is true.
`tools?: unknown`	Tool definitions offered to the model, for example `[{ type: "function", name, description, parameters }]`. Recorded as `gen_ai.tool.definitions` when `captureContent` is true, after credential scrubbing.

Labels must be non-empty strings of at most 256 characters; other values fall back to "unknown" or "chat" and count as instrumentation failures.

HueSpan

The callback receives span (the underlying OpenTelemetry span), context, traceId, spanId, setInput(value: unknown), setOutput(value: unknown) and setUsage(usage: TokenUsage). Inside model(), setInput and setOutput record gen_ai.input.messages and gen_ai.output.messages; elsewhere they record input.value and output.value. TokenUsage is { inputTokens?: number; outputTokens?: number }, recorded as gen_ai.usage.input_tokens and gen_ai.usage.output_tokens; invalid values are omitted and counted as instrumentation failures. IDs are all zeros for a disabled client. Use span.setAttribute(...), span.addEvent(...) and the other standard OpenTelemetry APIs for additional metadata. Hue does not estimate absent token counts or cost. Captured JSON null, false, 0 and "" remain present values; they are distinct from an omitted field. When uploads are available, helper content can serialize up to 1 GB and 1,048,576 values; otherwise it is bounded by the receiver’s inline limit.

Provider-executed tools

Tools the provider runs itself, such as OpenAI hosted MCP, web search, file search and code interpreter, or Anthropic’s MCP connector and server tools, never pass through hue.tool(). Inside the same model() callback, hand the response to recordProviderToolCalls:

await hue.model(
  "gpt-5-mini",
  async (span) => {
    const response = await openai.responses.create(request);
    span.setOutput(response.output_text);
    hue.recordProviderToolCalls(response, { request });
    return response;
  },
  { provider: "openai" },
);

Each OpenAI Responses mcp_call, web_search_call, file_search_call or code_interpreter_call item, and each Anthropic mcp_tool_use or server_tool_use block with its result, becomes an execute_tool {name} child span with gen_ai.tool.type extension, gen_ai.tool.call.id and, for MCP calls, mcp.server.name (the provider’s label). Arguments and results follow captureContent; a failed call carries error.type (mcp_error, or the provider’s status or error code) and an ERROR status. An OpenAI mcp_list_tools item becomes a tools/list child span with that server’s gen_ai.tool.definitions. The spans have no duration of their own: the provider ran the tools inside the model request. Unreadable items are skipped and counted as instrumentation failures; nothing is thrown. ProviderToolCallOptions extends Pick<SpanOptions, "parentContext">:

Option	Behavior
`provider?: HostedToolProvider`	`"openai"` (Responses API) or `"anthropic"` (Messages API). Defaults to the enclosing `model()` call’s provider when it is one of these.
`request?: unknown`	The request the response answers. Only hosted MCP server URLs are read from it, never credentials, to record each server’s host as `server.address`.
`servers?: Record<string, HostedServerInfo>`	Identity of each hosted MCP server, keyed by the label the provider uses for it. `HostedServerInfo` has `name` (recorded as `mcp.server.name`, defaulting to the label), `version`, and the Hue `provider` and `surface`.

Composio and Pipedream

Since 0.24.0, instrumentComposio(hue, composio) and instrumentPipedream(hue, pipedream) patch the client instance you already hold and return it, so each composio.tools.execute(slug, params), Tool Router session.execute(slug, arguments) or pipedream.actions.run({ id, ... }) call records its own execute_tool {tool} span under the active span, even inside a wrapper tool. The span carries hue.integration.platform (composio or pipedream), hue.integration.version when the call pins a version, the platform’s REST request fields as gen_ai.tool.call.arguments and the SDK’s response as gen_ai.tool.call.result; content follows captureContent and the redactor. Credential fields such as Composio customAuthParams are never recorded, a disabled client is returned untouched, and the SDK takes no dependency on @composio/core or @pipedream/sdk.

import { instrumentComposio, instrumentPipedream } from "@hue-run/sdk";

const composio = instrumentComposio(hue, new Composio({ apiKey: process.env.COMPOSIO_API_KEY }));
const pipedream = instrumentPipedream(hue, new PipedreamClient({ projectId, clientId, clientSecret, projectEnvironment }));

For a dispatcher the SDK does not patch, pass ToolOptions.integration, a ToolIntegration with platform, optional toolkit and optional version, to hue.tool(); it is recorded as hue.integration.platform, hue.integration.toolkit and hue.integration.version.

Files

recordFile adds a hue.file event to the active span, or to the span of an explicit context passed second, for a file the work read, received or produced. Files are linked by content hash. Set upload: true to attach supplied bytes, or the file at path, to the span:

import { readFile } from "node:fs/promises";

await hue.withSpan("review contract", async () => {
  const pdf = await readFile("contract.pdf");
  hue.recordFile({ role: "input", mediaType: "application/pdf", data: pdf, name: "contract.pdf", upload: true });
  // ...
});

A FileRecord has role ("input" given to the agent, "attachment" from a tool or message, or "output" produced by the agent), mediaType, and one of data (a Uint8Array, or a string hashed as UTF-8), path or sha256, with optional byteSize, name and upload. data up to 25 MiB is hashed and measured locally; larger data is omitted and counted as an instrumentation failure, so pass sha256 and byteSize when you already have them. A supplied sha256 or byteSize must match data. path, in place of data (not both), names a regular file up to 1 GB: up to 25 MiB it is read as data is, and a larger file is hashed now and, with upload: true, streamed from disk when it uploads, only if it still has its recorded size and SHA-256. A path that is not a regular file, or a file over 1 GB, is omitted and counted. upload defaults to false. With upload: true, supplied data or path, captureContent: true and an ordinary project key, export adds a hue.blobs entry to the same span, keyed event:hue.file:<sha256>, and the bytes upload after the span is sent. An upload that is not made counts in uploadFallbacks, and the event remains. See Images, documents and audio. The event carries hue.file.sha256, hue.file.role, hue.file.media_type and, when known, hue.file.size. It is metadata, so it is recorded in both capture modes; hue.file.name is recorded only when captureContent is true. A name must be nonblank after trimming, at most 256 UTF-16 code units and well-formed Unicode, with no C0 control characters (U+0000–U+001F) or DEL (U+007F). An invalid name is omitted and counted as an instrumentation failure, and the event remains; before 0.17.0 tabs, newlines and DEL were accepted. An invalid record, or one without an active span, is omitted and counted as an instrumentation failure. new Attachment(options: AttachmentOptions) places a file in any value Hue records: a withSpan() input or output, model() messages, a tool’s arguments or result, recordMessages() messages or a span attribute. AttachmentOptions takes exactly one of data (an ArrayBufferView, an ArrayBuffer, or a base64 string or data: URL) and path, plus contentType and an optional name. data is limited to 25 MiB. path names a regular file up to 1 GB; over 25 MiB it is hashed when the attachment is created and streamed from disk when it uploads, only if it still has its recorded size and SHA-256. The attachment exposes readonly contentType, name (when valid), sha256 and size (both undefined when invalid). It is recorded as a GenAI blob part in digest form, and its bytes upload after its record is sent. An invalid attachment is recorded as null and counted as an instrumentation failure. See Attach a file anywhere.

Vercel AI SDK adapter

import { hueTelemetry } from "@hue-run/sdk/ai-sdk";

// Pass to an AI SDK 7 agent or generation call:
const telemetry = hueTelemetry(hue);

hueTelemetry(hue: HueClient, options?: HueTelemetryOptions): TelemetryOptions enables telemetry, sets recordInputs and recordOutputs from hue.captureContent, and configures the AI SDK OpenTelemetry integration with hue.tracer and usage reporting. It also records the end user and conversation from the call’s AI SDK runtime context: runtimeContext.userId as user.id, and sessionId, else threadId, as gen_ai.conversation.id, on every span of the call. It ignores a value that is not a string or finite number of 1–4096 valid characters, and an enclosing withSpan’s userId or sessionId takes precedence. AI SDK withholds runtime context from telemetry unless includeRuntimeContext names the key, so the returned options name these keys there; no other runtime-context value is recorded. To keep your own includeRuntimeContext, spread the returned one into it.

const result = await generateText({
  model,
  prompt,
  runtimeContext: { userId: user.id, sessionId: chat.id },
  telemetry: hueTelemetry(hue),
});

HueTelemetryOptions names other runtime-context keys. userKey?: string | false names the user’s key, "userId" by default. sessionKey?: string | readonly string[] | false names the conversation’s keys, of which the first with a value wins, ["sessionId", "threadId"] by default. false records none, and an empty key throws a TypeError. Supported peer ranges are ai@^7.0.0 and @ai-sdk/otel@^1.0.0; keep your installed AI SDK version and add an OTel peer in its supported range. Compatibility lists the tested AI SDK and OTel pairs; other releases in range are not individually verified. hueTelemetry reads the installed ai major version once per process and throws a TypeError below 7. For AI SDK 6, use hueExperimentalTelemetry from the core entry point instead:

import { hueExperimentalTelemetry } from "@hue-run/sdk";

const result = await generateText({
  model,
  prompt,
  experimental_telemetry: hueExperimentalTelemetry(hue),
});

hueExperimentalTelemetry(hue: HueClient): ExperimentalTelemetrySettings returns { isEnabled, recordInputs, recordOutputs, tracer }: isEnabled mirrors the client’s enabled flag, the record flags mirror captureContent, and tracer is Hue’s tracer, so AI SDK 6 spans parent under withSpan and inherit identifiers. It needs no @ai-sdk/otel peer. Do not upgrade an existing app’s AI SDK just to install Hue. For an app that already emits OpenTelemetry spans, assess the existing-provider path and its installed OpenTelemetry versions using the compatibility guide. Provider-executed tools, such as OpenAI hosted MCP (openai.tools.mcp), appear as execute_tool mcp.<name> spans with gen_ai.tool.type extension. The server is named only by serverLabel inside the recorded result, so before export the SDK copies it to mcp.server.name, and a result carrying an MCP error sets an ERROR status and error.type mcp_error. Both attributes survive metadata-only export. The label can only be read when AI SDK recorded the result: hueTelemetry(hue) with captureContent: false records none, whereas an application whose AI SDK integration records outputs and exports through Hue’s attached processors keeps the label. AI SDK per-call integrations replace the global integrations for that call. If your app already has telemetry, preserve its setup and attach Hue to the existing provider. See OpenTelemetry and the Example chatbot.

Content policy

See Data privacy for runnable email-masking examples and the correct attachment point for each integration.

captureContent: false disables Hue helper content and removes recognized GenAI, Vercel AI SDK, OpenInference, OpenLLMetry and Langfuse content, legacy GenAI content events, log bodies, status messages and exception text before Hue exports records, including records from other instrumentations on a shared provider. The exported contentPrefixes array lists the attribute keys, and their dotted children, that are removed; the compatibility matrix reproduces it. It does not classify every custom attribute name.
captureContent: true permits supplied content. Received content is stored by Hue; the SDK does not set a retention timer. Helpers still omit exception messages and stack traces.
With captureContent: false, a record whose tool definitions are removed keeps a summary of them: hue.tool.names lists each definition’s name (Chat Completions function.name, or the type of an unnamed built-in tool such as mcp) in order, and hue.tool.definitions.sha256 is the lowercase hex SHA-256 of the RFC 8785 canonical JSON of the credential-scrubbed definition list. The digest does not change when a credential rotates. Only definitions another integration recorded can be summarized: hueTelemetry(hue) with captureContent: false records none.
Hosted tools carry credentials in their definitions, such as the authorization and headers of an OpenAI hosted MCP tool. Before export, and before redact, Hue replaces the values of credential-like fields (authorization, authorization_token, headers, api_key, access_token, x-api-key, and keys ending in token, secret, password, apikey or credential, case-insensitively and ignoring - and _) with "[redacted]" in gen_ai.tool.definitions, ai.prompt.tools and llm.tools.*.tool.json_schema, and in the tools and mcp_servers entries of a raw provider request or response recorded as input.value, output.value or llm.invocation_parameters. Parameters named in a JSON Schema properties object keep their schemas, so a tool that takes a headers argument is still described. A definition nested more than 256 levels deep rejects its record.
A span’s own string attributes over Hue’s inline limit upload separately, up to 1,000,000,000 bytes, and keep a 16 KiB prefix. Inline media in the span attributes and log records that automatic capture reads uploads and leaves its digest in place; text media over 64 KiB keeps a 16 KiB UTF-8 prefix. Resources, links and span events never upload: inline media there up to 64 KiB stays as recorded and larger media becomes its digest. An Attachment (up to 25 MiB) uploads at its position, and recordFile uploads supplied data up to 25 MiB only with upload: true. Uploads run in the background after their record is sent. A value refused when its record is exported falls back to a cut, to the digest without a hue.blobs entry in recorded messages, or to the inline bytes elsewhere; one whose upload fails afterwards keeps its prefix or digest and its entry. Both count in uploadFallbacks. Nothing uploads with captureContent: false or a setup credential.
redact(value, path) transforms supported strings in attributes, resources, event/link attributes and log bodies before Hue exports them. It must return a string. A redactor failure rejects the affected record and becomes an export issue.
This policy applies to Hue’s export path. Another exporter and arbitrary span/scope names remain your application’s responsibility.

Flush, shutdown and diagnostics

hue.checkConnection(): Promise<ProjectConnection>
hue.flush(): Promise<ExportReport>
hue.shutdown(): Promise<ExportReport>

hue.transport.getReport(): ExportReport
hue.transport.getIssues(): ExportIssue[]
hue.transport.getFailureSequence(): number

checkConnection() returns a ProjectConnection, { id, name, organizationId, slug }, for the project associated with the key. It throws HueConnectionError on rejected authentication, network failure or an invalid response; status carries the HTTP status when Hue answered, and cause carries the underlying network, timeout or parsing error. Its message is fixed and safe to log. It sends no trace. flush() sends traces and logs, then waits for the uploads queued before it. Concurrent callers receive serialized drains that include their preceding emissions. Rejections, failed exports, queue drops and invalid acknowledgements throw HueExportError, whose issues and report contain sanitized diagnostics. A later non-overlapping flush reports new failures; counters remain cumulative. Overlapping calls also observe failures from shared in-flight work. ExportReport contains acceptedSpans, acceptedLogs, rejectedSpans, rejectedLogs, failedSpans, failedLogs, pendingSpans, pendingLogs, droppedSpans, droppedLogs, pendingBytes, instrumentationFailures, uploadedValues and uploadFallbacks. Accepted, rejected, failed, dropped, instrumentation and upload counts are cumulative; pending counts and bytes describe current queued/in-flight records. getIssues() returns the latest 128 ExportIssue values with sequence, signal (a Signal: "traces" or "logs"), kind, count, optional status and a sanitized message. Kinds are rejected, failed, dropped, invalid and warning. A warning with no rejection does not fail a flush. The transport’s spanProcessor and logRecordProcessor receive RecordValue records, the ReadableSpan or ReadableLogRecord the OpenTelemetry SDK hands to a processor. The exporter plumbing behind them is internal and not part of the published declarations; read getReport(), getIssues() and getFailureSequence() instead. shutdown() stops new Hue helpers and flushes. It closes providers that Hue created; borrowed providers remain usable. Repeated calls share the same shutdown promise. If your application supplied the providers, close those providers and then call transport.shutdown() during process shutdown. Reuse a shared client across server requests. Queues are in memory and bounded to 2,048 records per signal, including exports in flight. Export requests are sized from each record’s own encoding and split at 1 MiB. Retryable failures use the OpenTelemetry retry policy; partial rejections are not retried wholesale. An acknowledgement means collector receipt, not that every span of a distributed trace has arrived. Stop producing spans before shutdown.

Live spans

OpenTelemetry exports a span only when it ends, so a long streamed turn would otherwise stay invisible until it finishes. When a Hue or AI span is still open at the transport’s next 500 ms tick, the transport queues a placeholder: an ordinary OTLP span whose parent is the running span, with its name, kind, start time and current attributes and an end time of 0. Hue shows the span, and its trace, as running and replaces the placeholder when the finished span arrives. A placeholder whose span has ended by the time it is exported is not sent. A placeholder usually reaches Hue about 1.5 s after its span starts: up to 500 ms until the next tick, then about 1 s while the export queue batches it. It arrives sooner when a batch is already scheduled and later while an earlier export is still in flight, because the next 1 s batch is scheduled only after that export finishes. A span shorter than about 1.5 s therefore usually never shows as running and appears only when it ends.

Only spans from the client’s tracer (withSpan, tool, model, hue.tracer and the AI SDK adapters) and spans with a gen_ai., ai., llm. or traceloop. attribute at start, or a name starting with ai., are announced. HTTP, database and other framework spans are not.
Placeholder attributes follow captureContent and redact like the finished span. Tool definitions, system instructions and any value over 64 KiB are left out.
A span processor that scrubs, renames or drops spans in onEnd never sees placeholders: the transport builds each one from the running span’s current name and attributes. If a wrapper forwards onStart to Hue and the span is still open when its placeholder is exported, Hue receives the span’s original name and its attributes after redact, even when the wrapper later edits, renames or drops the finished span. Remove sensitive attribute values with redact or where you record them. redact does not change span names, so keep a sensitive value out of the name where you create the span, or turn live spans off with liveSpans: false.
Placeholders are advisory. They are queued only while the queue is under a quarter of its record and byte budgets, and skipped silently otherwise. While queued they count in pendingSpans and pendingBytes, but never as accepted, rejected, failed or dropped records. Losing only placeholders records a warning issue and does not make flush() throw.
Completed spans are exported in separate requests before placeholders, so their rejections stay visible. If the first trace acknowledgement lacks Hue-Pending-Spans: 1, the transport records one warning issue and stops sending placeholders for that client; the receiver gets at most one placeholder request.
Opt out with liveSpans: false.
With an existing provider, announcements start in spanProcessor.onStart. A wrapping processor that forwards onStart should forward onEnd for the same spans; a placeholder already sent for a span that never reaches Hue keeps it showing as running until Hue marks the trace stalled.

Trace verification

await hue.verifyTrace(traceId, {
  expectedSpanIds: [requestSpanId],
  requiredFields: ["model"],
  timeoutMillis: 10_000,
});

verifyTrace(traceId: string, options?: VerifyTraceOptions): Promise<TraceVerification>. VerifyTraceOptions has expectedSpanIds, requiredFields (an array of TraceReceiptField: "input", "output", "model", "usage" or "session") and timeoutMillis. TraceVerification is { verified: boolean; receipt: TraceReceipt | null; project?: ProjectConnection }; project is set when Hue reported the trace missing and names the project the key searched. A receipt contains traceId, spanCount, revision, fields (Record<TraceReceiptField, boolean>), matchedSpanIds, missingSpanIds, and traceUrl. verified requires a stored receipt, every supplied expected span, and every required field. Timeout returns verified: false with the latest partial receipt, or null if none arrived; a null receipt with project means no trace with that ID exists in that project, so check which key the process loaded. Other failures throw exported HueTraceVerificationError, with code and optional HTTP status, without response bodies or credentials. A disabled client rejects with HueConnectionError. Trace IDs are 32 lowercase hexadecimal characters; expected span IDs are 16. All IDs must be nonzero. Supply at most 100 unique expected span IDs and unique required field names. The timeout defaults to 10,000 ms and must be positive, finite, and at most 60,000 ms. The receipt response is limited to 64 KiB. Missing spans or required fields keep polling until the deadline. Only recognized trace-not-found responses, HTTP 429, and HTTP 503 are retried; connection failures, unsupported endpoints, redirects, and malformed responses raise a safe typed error. Verification reads stored normalized field presence across the trace, including supported correlated logs. It does not inspect values or prove that unlisted application operations were instrumented. Flush the owning providers before verification; this method never flushes implicitly.

Evaluation client

import { createEvaluationClient } from "@hue-run/sdk/evals";

const client = createEvaluationClient({ apiKey: process.env.HUE_API_KEY! });

createEvaluationClient(options: EvaluationClientOptions): EvaluationClient and new EvaluationClient(options) accept required apiKey, optional baseUrl (default https://app.hue.run), optional timeoutMillis (default 10000, range 100–60000) and optional maxAttempts (default 4, range 1–10): the bounded attempts for a read, an update or a keyed mutation after a transient failure, described under experiments, saved subjects and results; 1 sends every such request once. checkConnection() returns the key’s project. Evaluation registry writes require a Read and write key. Methods below return promises. PageOptions is { after?: string; limit?: number }, with a supplied limit from 1 through 100. Paged responses are Page<T>, { items, nextCursor }; pass a non-null cursor as the next after. IDs are UUIDs. Identity is { name, slug, description? }. listEvalSets, listEvaluators, listDatasets and listScorers accept RegistryPageOptions, PageOptions plus includeArchived?: boolean, which includes archived identities so callers can diagnose slug conflicts explicitly, and tags?: string[], which keeps identities carrying any of the named tags. Evaluation responses are limited to 4 MiB. listCases(versionId, page?) includes case content, so a valid row limit can still produce an oversized response when cases are large. Request a smaller page, such as { limit: 1 }, and follow nextCursor; the client does not automatically split or retry an oversized page. Request bodies are limited to 1 MiB.

Evaluation types

@hue-run/sdk/evals exports these types for method inputs and results.

Type	Meaning
`Dataset` / `DatasetVersion`	A dataset with its `versions` / one version with `revision`, `frozenAt` and `contentDigest` (`null` while it is a draft).
`DatasetCase`	A stored case: `id`, `externalKey`, `inputs`, optional `expected`, `metadata` and `datasetVersionId`, plus optional immutable provenance: `environmentVersionId` (the exact World selected for the case), `worldModelReference` (a `WorldModelReference`: `modelId`, `representedAt` and `generation` of a frozen World model selection whose World Hue prepares at execution; since `0.24.3` such a case runs as a World case), `sourceTraceId` / `sourceTraceRevision` and `artifactManifestId`.
`ExperimentCase`	A `DatasetCase` as an experiment sees it, with `hasExpected` (JSON `null` counts as present) and, on current servers, `inputFiles`, its pinned `CaseFile` entries.
`CaseFile` / `SubjectFile`	One pinned input file in Hue’s immutable manifest: `artifactId`, `role` (`source`, `attached_template`, `attached_reference`, `original`, or the evaluator-only `org_template` and `evaluator_reference`), `filename`, `contentType`, verified `byteSize` and hex `sha256` / the same entry on a saved subject, whose `role` may also be `output` for a file the target generated.
`LocalFile`	A verified copy of a case input or generated output on the runner’s disk: the `CaseFile` fields with `role` including `output`, the absolute `path` and optional `primary`. Generated files receive their `artifactId` after upload.
`OutputFile`	A file the target generated for one case: `filename` (sanitized to one path segment), `contentType`, optional `primary` (at most one per execution) and either a local `path` or in-memory `bytes`.
`EvalSet` / `EvalSetVersion` / `EvalSetCase`	The product-named forms of `Dataset`, `DatasetVersion` and `DatasetCase`: an eval set’s `versions` are `EvalSetVersion` values, which add `evalSetId`, and a case adds `evalSetVersionId`. The v1 fields remain present.
`CaseWrite`	Input for `addEvalSetCase` and `addCase`; optional `environmentVersionId` selects an immutable World version for the case, and optional `files` pins up to 32 input files.
`CaseFileReference` / `CaseFileWrite`	A pinned input file: `{ artifactId, role }` for an artifact the project holds / that reference, or `{ role, filename, contentType }` with `bytes` or a `path` that the client checks (one of `outputContentTypes`, a name of 1–255 UTF-16 code units without path separators or control characters, 1 byte to 25 MiB) and uploads as a verified artifact before the case. A file outside that policy throws `TypeError` or `RangeError` before any request; repeating the write reuses the uploaded artifact. A `path` file is hashed when the case is checked and read again for its upload, one file at a time; one that changed or can no longer be read throws `TypeError` and the case is not written.
`Scorer` / `ScorerVersion`	A scorer and its published `versions` / an immutable published definition with `contentDigest`.
`Evaluator` / `EvaluatorVersion`	The product-named forms of `Scorer`, with optional `versions`, and `ScorerVersion`, which adds `evaluatorId` when the server supplies it; older v1 responses may omit it.
`ScorerDefinition` / `MetricDefinition`	The pinned definition variants / a typed metric a scorer declares.
`JudgeConfig`	Hosted judge configuration: `model`, `provider`, `rubric`, `bindings`, `maxOutputTokens`, `timeoutMs` and optional `temperature`.
`Experiment`	A frozen dataset version, a configuration and pinned scorers, with its `evaluation` run and case counts by execution state.
`Run` / `RunCase`	An `Experiment` with `evalSetVersionId` and its `scoring` pass / an `ExperimentCase` with `evalSetVersionId`.
`ExperimentItem`	A case within an experiment and its latest `execution` (`null` before the first start).
`Execution` / `TerminalState`	One target attempt whose `state` is a `TerminalState` (`"succeeded"`, `"error"` or `"cancelled"`), `"started"` or `"uncertain"`.
`StartExecution` / `CompleteExecution` / `Completion`	Inputs and result of the low-level execution methods.
`EvaluationRun`	Scoring progress for an experiment or rescore: pinned `scorerVersions`, `itemCount` and result counts by state.
`EvaluationRunSummary`	One row of `listEvaluationRuns`: `id`, `name`, `experimentId` (`null` for a standalone scoring run) and `createdAt`. Read pins and score counts with `getEvaluationRun`.
`Scoring` / `ScoringSummary`	An `EvaluationRun` with its pinned `evaluatorVersions` and optional `runId` / an `EvaluationRunSummary` with `runId`. `runId` is `null` for standalone scoring. A run ID and a scoring ID identify different records.
`EvaluationItem`	A subject within an evaluation run, with `hasOutput` and `traceSnapshotId`.
`Subject`	An immutable saved outcome: `executionId`, `inputs`, optional `output` and `expected`, `metadata`, `executionState`, trace evidence fields and the source case, experiment and attempt. Current servers add `environmentVersionId` and `worldModelReference`, the frozen input and output `files` (`SubjectFile[]`) and the declared `primaryArtifactId`.
`ScoringSubject`	A `Subject` with its source `evalSetVersionId` and `runId`.
`Score` / `Result`	A scorer outcome (`scored`, `error` or `skipped`) / a score addressed to an `evaluationItemId` and `scorerVersionId`.
`Metric`	A reported metric value: `name`, `value` and optional `passed`; a failed metric stays `state: "scored"`.
`TypedError`	A sanitized `{ type, message? }`; the message is stored only when result content is persisted.
`ResultSummary` / `StoredResult`	A listed result with `state` / the full stored result with `metrics`, `explanation`, `evidence`, `error` and `sourceDigest`.
`ScoringResultInput` / `ScoringResultSummary` / `StoredScoringResult`	A `Score` addressed to an `evaluationItemId` and `evaluatorVersionId`, with `sourceDigest` for local code / a `ResultSummary` with `evaluatorVersionId` / a `StoredResult` with `scoringId` and `evaluatorVersionId`.
`ScoreContext` / `LocalScorer`	What a local scorer callback receives, including sealed `environment` evidence when the runner requires it / a pinned `local_code` definition with its bound callback.
`JudgeJob`	A hosted judge job with `state`, charge accounting and timestamps.
`JudgeBudget`	The project’s hosted judge configuration and admission controls; credential resolution does not establish provider acceptance.
`RegistryPageOptions`	`PageOptions` plus `includeArchived` and `tags` for `listEvalSets`, `listEvaluators`, `listDatasets` and `listScorers`.
`TaggedIdentity` / `IdentityUpdate`	An `Identity` plus optional `tags`, the tag names a new eval set or evaluator starts with / optional `name`, `description` and `tags` to change; omitted fields stay.
`Tag` / `TagColor`	A project tag: `id`, `name` (unique in the project without case) and `color` / `"default"`, `"gray"`, `"brown"`, `"orange"`, `"yellow"`, `"green"`, `"blue"`, `"purple"`, `"pink"` or `"red"`.
`UpdatedRun`	A run after `updateRun` or `updateExperiment`: `id`, `name`, its own `tags` and its eval set’s `evalSetTags`.
`EnvironmentEvidenceSnapshot` / `EnvironmentEvidence`	Sealed environment evidence resolved through one target execution: `runId`, `executionId`, `environmentVersionId`, `definitionDigest`, `seed`, terminal `status` (`completed`, `abandoned` or `expired`), `stepCount`, `stateDigest`, `initialState`, `finalState` and the coverage fields / the snapshot plus its complete `steps` ordered by ordinal.
`SimulationMcpCapability`	Short-lived execution-scoped MCP connection for one simulated instance: `url`, `token` and `expiresAt`. Never persist or expose the token.

Eval sets, evaluators, runs and scoring

Use these product-named methods in new code. They call the same v1 endpoints as the dataset, scorer, experiment and evaluation-run methods in the next sections and return the product fields, such as evalSetId, evalSetVersionId, evaluatorVersionId, runId and scoringId, alongside the v1 fields. The earlier method names are deprecated but remain callable for at least two more 0.MINOR releases.

Method	Result or purpose
`createEvalSet(input: TaggedIdentity)` / `getEvalSet(id)` / `listEvalSets(page?)`	An `EvalSet`, including its initial `versions` / a page of eval set summaries.
`updateEvalSet(id, input: IdentityUpdate & { slug? })`	Rename an eval set or replace its tags → the eval set without `versions`.
`createEvalSetVersion(evalSetId, { fromVersionId? }?)` / `getEvalSetVersion(versionId)`	A new editable `EvalSetVersion` / a version.
`listEvalSetCases(versionId, page?)` / `addEvalSetCase(versionId, input: CaseWrite)`	A page of `EvalSetCase` values / `{ item, version }`; retain the updated revision.
`freezeEvalSetVersion(versionId, expectedRevision)`	The frozen `EvalSetVersion`.
`createEvaluator(input: TaggedIdentity)` / `getEvaluator(id)` / `listEvaluators(page?)`	Register or retrieve `Evaluator` identities.
`updateEvaluator(id, input: IdentityUpdate)`	Rename an evaluator or replace its tags → `Evaluator`.
`publishEvaluatorVersion(evaluatorId, definition: ScorerDefinition)` / `getEvaluatorVersion(versionId)`	Publish or retrieve an immutable `EvaluatorVersion`.
`createRun(input)`	`{ idempotencyKey, name, evalSetVersionId, evaluatorVersionIds, config, tags? }` → `{ id, scoringId, evaluationRunId }`.
`updateRun(id, { name?, tags? })`	Rename a run or replace its own tags → `UpdatedRun`.
`getRun(id)` / `listRunItems(id, page?)` / `getRunCase(id, caseId)`	A `Run` / a page of `ExperimentItem` values / a `RunCase`.
`startRunExecution(runId, caseId, input: StartExecution)` / `getRunExecution(id)` / `completeRunExecution(id, input: CompleteExecution)`	Start or recover an attempt → `Execution` / read an attempt / acknowledge its terminal outcome → `Completion`.
`finishRun(id, idempotencyKey)`	`{ id, finishedAt }`.
`createScoring(input)`	`{ idempotencyKey, name, subjectIds, evaluatorVersionIds }` → `{ id }`.
`getScoring(id)` / `listScorings(page?)` / `listScoringItems(id, page?)` / `getScoringSubject(subjectId)`	A `Scoring` / a page of `ScoringSummary` values / a page of `EvaluationItem` values / a `ScoringSubject`.
`submitScoringResults(scoringId, { idempotencyKey, results: ScoringResultInput[] })`	`{ ids }`.
`listScoringResults(scoringId, page?)` / `getScoringResult(resultId)`	A page of `ScoringResultSummary` values / a `StoredScoringResult`.
`listTags()`	`{ items: Tag[] }`, the project’s tags in its order: as people arranged them in Hue, then newer tags.

CI evaluation invocations

An invocation freezes the selected cases, evaluator versions, configuration and gate policy for one CI check. These EvaluationClient methods recover the server’s saved decision. Every method returns Promise<EvalInvocation> and accepts an optional final AbortSignal. See Run in CI for the CLI and GitHub Action.

Method	Result or purpose
`createEvalInvocation(input: CreateEvalInvocationInput, signal?)`	Create or recover an invocation by its project-scoped `idempotencyKey`.
`getEvalInvocation(id, signal?)` / `getEvalInvocationByKey(idempotencyKey, signal?)`	Read an invocation by invocation or candidate run ID / recover it by request key. Neither method runs the agent.
`createEvalBaseline(id, input: CreateEvalBaselineInput, signal?)`	Create or recover an owned base run from the candidate’s frozen inputs.
`finalizeEvalInvocation(id, input: FinalizeEvalInvocationInput, signal?)`	Freeze the server’s receipt. Call after scoring settles; an early call saves an immutable `incomplete` verdict.
`cancelEvalInvocation(id, input: CancelEvalInvocationInput, signal?)`	Close execution and scoring, including an owned base run.

Type	Fields
`CreateEvalInvocationInput`	Required `idempotencyKey` (1–128 characters), ISO 8601 `deadlineAt` with a timezone, and exactly one of `evalSetId` or `evalSetVersionId`. Optional `name`, `evaluatorIds`, `evaluatorVersionIds`, JSON `config`, `policy`, `ci` and compatible completed `baselineRunId`.
`CreateEvalBaselineInput`	`idempotencyKey` and `ci` with `commitSha`.
`FinalizeEvalInvocationInput`	`idempotencyKey` and optional `executionIds`.
`CancelEvalInvocationInput`	`idempotencyKey` and `reason`: `interrupted`, `deadline` or `runner_error`.
`EvalCiContext`	`provider`: `github` or `local`. Optional `repository`, `commitSha`, `baseSha`, `pullRequestHeadSha`, `pullRequestNumber`, `workflowRunId`, `workflowRunAttempt` and `eventName`. `commitSha` identifies the checkout; a PR’s head may differ.
`EvalInvocation`	`id`, `runId`, `runUrl`, `deadlineAt`, `state` (`active`, `finalized`, `cancelled` or `expired`), frozen `manifest` and nullable `receipt`. The manifest contains its `digest`, eval set/version and original `scoringRunId`, `config`, `policy`, `cases`, `evaluators` and nullable `baselineRunId`. Cases preserve `contentDigest`, optional `inputIdentity` and `judgePinning`, evaluator version IDs and world pins.
`EvalGateReceipt`	`id`, `manifestDigest`, `runId`, `scoringRunId`, `verdict` (`pass`, `fail`, `incomplete`, `error` or `cancelled`), `finalizedAt`, `executionIds`, `results` and `reasons`. Each result identifies the case, execution, evaluator version and saved result, with its state and metrics. Optional `baselineRunId`, `baselineReceiptId` and `comparisons` are omitted when absent.

policy.thresholds contains { evaluatorVersionId, metric, operator: "gte" | "lte", value } checks on each applicable required result. Optional policy.baselineComparisons contains { evaluatorVersionId, metric, operator, tolerance } checks on paired case means; tolerance is finite and nonnegative. Each receipt comparison includes those rule fields plus candidateMean, baselineMean, passed and caseIds. The server alone decides the verdict. Each control input requires its own stable idempotencyKey. Retry with the original key and body, including the original deadline. Recover by key before resolving versions or choosing a new deadline. Read the saved verdict instead of deriving a new one from later scores.

Datasets and scorer definitions

Method	Result or purpose
`createDataset(input: TaggedIdentity)`	`Dataset`, including its initial `versions`.
`updateDataset(id, input: IdentityUpdate & { slug? })`	Rename a dataset or replace its tags → the dataset without `versions`.
`getDataset(id)` / `listDatasets(page?)`	A dataset / a page of dataset summaries.
`createDatasetVersion(datasetId, { fromVersionId? }?)`	A new editable `DatasetVersion`.
`getDatasetVersion(versionId)` / `listCases(versionId, page?)`	A version / a page of `DatasetCase` items.
`addCase(versionId, input: CaseWrite)`	`{ item, version }`; retain the updated revision.
`freezeDatasetVersion(versionId, expectedRevision)`	The frozen `DatasetVersion`.
`createScorer(input: TaggedIdentity)` / `getScorer(id)` / `listScorers(page?)`	Register or retrieve scorer identities.
`updateScorer(id, input: IdentityUpdate)`	Rename a scorer or replace its tags → `Scorer`.
`publishScorerVersion(scorerId, definition: ScorerDefinition)` / `getScorerVersion(versionId)`	Publish or retrieve an immutable `ScorerVersion`.

CaseWrite requires expectedRevision, externalKey and inputs; expected and metadata are optional. Omit expected to declare no reference; expected: null is a present reference. Freeze the dataset and publish scorer definitions before creating an experiment.

ScorerDefinition and metrics

Definition	Required fields
Built-in	Use `builtins.exactMatch()`, `builtins.includes(caseSensitive)` or `builtins.jsonSchema(schema)` to create the exact versioned declaration.
Local code	`kind: "local_code"`, `language: "typescript"` or `"python"`, `entrypoint`, `sourceDigest`, `metrics`. `defineLocalScorer` creates the TypeScript declaration and binds the callback.
Manual	`kind: "manual"`, `metrics`. Completion requires the platform’s human workflow.
Hosted judge	`kind: "llm_judge"`, `metrics`, `config`. Hosted dispatch is separate from a local run.
Hue-executed outcome	`kind: "world_outcome"`, `entry` (`hue.conversion_outcome.v2`, `hue.outcome_assertions.v2` or `hue.outcome_assertions.v3`) and the `metrics` the entry fixes, which are filled in when omitted. A `hue.outcome_assertions.v3` version also pins its judge in `config.judge`. Hue scores these after the instance seals; the local runner defers them.

Each MetricDefinition has a name and type: boolean, text, number (optional min/max), or category (required categories: string[]). An optional description (1–500 characters) is shown with the metric’s check in Hue’s run view. Returned metrics must match the pinned definitions exactly. An OutcomeJudgeConfig pins a hue.outcome_assertions.v3 judge: model (provider-prefixed), provider, template (the SHA-256 of the judge template), samples (1–9, majority decides), temperature (0–2), maxOutputTokens and timeoutMs. Hosted config is a JudgeConfig with required model, provider, rubric, bindings (an array of { name, path, required }), maxOutputTokens and timeoutMs; temperature is optional. A declaration identifies the requested evaluation policy; the server validates supported values and project execution settings when it is used.

Experiments, saved subjects and results

Method	Required inputs and result
`createExperiment(input)`	`{ idempotencyKey, name, datasetVersionId, scorerVersionIds, config, tags? }` → `{ id, evaluationRunId }`.
`updateExperiment(id, { name?, tags? })`	Rename an experiment or replace its own tags → `UpdatedRun`.
`getExperiment(id)` / `listExperimentItems(id, page?)` / `getExperimentCase(id, caseId)`	An `Experiment` / a page of `ExperimentItem` values / an `ExperimentCase`.
`startExecution(experimentId, caseId, input: StartExecution)`	Start or recover an attempt → `Execution`.
`getExecution(id)` / `completeExecution(id, input: CompleteExecution)`	Read an attempt / acknowledge its terminal outcome → `Completion`.
`getEnvironmentEvidence(executionId)` / `getEnvironmentSteps(executionId, { after?, limit? }?)`	The sealed `EnvironmentEvidenceSnapshot` linked to an execution / a `StepPage` of its sealed journal; `after` is a step ordinal (`-1` starts at the first step) and `limit` is 1–100.
`createSimulationMcpCapability({ runId, executionId })`	The execution-scoped generic MCP capability for one instance → `SimulationMcpCapability`. `runSimulation` calls it when no provider-profile preflight is requested.
`finishExperiment(id, idempotencyKey)`	`{ id, finishedAt }`.
`createEvaluationRun(input)`	`{ idempotencyKey, name, subjectIds, scorerVersionIds }` → `{ id }`.
`getEvaluationRun(id)` / `listEvaluationItems(id, page?)` / `getSubject(subjectId)`	An `EvaluationRun` / a page of `EvaluationItem` values / an immutable `Subject`.
`listEvaluationRuns(page?)`	A page of `EvaluationRunSummary` values: every evaluation run of the project, oldest first.
`submitResults(runId, { idempotencyKey, results: Result[] })`	`{ ids }`.
`listResults(runId, page?)` / `getResult(resultId)`	A page of `ResultSummary` values / the `StoredResult`.

StartExecution requires idempotencyKey; optional fields are traceExternalId, previousExecutionId and allowUncertainRetry. CompleteExecution requires idempotencyKey and a terminal state (succeeded, error or cancelled); optional fields are output, error (a TypedError), expectedTraceRevision, traceEvidence (required or omit) and omissionReason. Completion contains executionId, subjectId, nullable traceSnapshotId and evaluationItemId. Use these low-level methods only with a known recovery decision; see recovery. Each submitted Result requires evaluationItemId, scorerVersionId and the Score fields described below; a local-code result also carries its sourceDigest. A scored result contains metrics: [{ name, value, passed? }] and a nonempty explanation or an evidence value. Error results contain error: { type, message? }; skipped results contain an explanation. On the evaluation client, a read, an update that sets given fields (updateEvalSet, updateEvaluator, updateRun and their dataset, scorer and experiment forms), or a mutation the server deduplicates by the idempotencyKey in its body (the local worker’s register, claim, heartbeat, completion and capability routes included), is sent again after a connection failure, a timeout or a 408 or 5xx that carried no Retry-After, up to maxAttempts (default 4, 1–10) times with a jittered backoff. isTransientApiError(error) names those failures. Any other mutation is sent once: reuse a saved idempotency key with the same intended mutation after a lost response. HueApiError.status distinguishes an HTTP rejection from a connection/response failure, HueApiError.diagnostic carries Hue’s X-Hue-Diagnostic code when it sent one, and retryAfterSeconds the wait of a refusal Hue timed; errors do not echo server bodies.

Hosted judge jobs

getJudgeBudget() returns a JudgeBudget with the hosted judge configuration and admission controls, and the organization’s monthly AI budget as the optional organizationBudget. createJudgeJobs(runId, { idempotencyKey, jobs: [{ evaluationItemId, scorerVersionId }] }) returns job IDs. Inspect JudgeJob values with listJudgeJobs(runId, page?) or getJudgeJob(jobId), and request cancellation with cancelJudgeJob(jobId, reason). Submitting hosted jobs is an explicit platform operation and draws on that budget. SDK method availability does not establish that a hosted provider is configured or that the budget has room. Local runners leave hosted and manual scorer pins for their owning workflows.

Artifacts

runExperiment uses these methods to publish generated files; you rarely call them directly.

Method	Result or purpose
`reserveArtifact({ idempotencyKey, filename, contentType, byteSize, sha256 })`	Reserves an artifact by declared identity → `ArtifactReservation`; replaying the key returns the same reservation.
`requestArtifactUpload(id)`	Issues a short-lived storage capability for the reserved bytes → `ArtifactUpload`.
`uploadArtifactBytes(upload, bytes, contentType)`	Stages the bytes with a `PUT` to that capability. The Hue key is never sent to storage.
`completeArtifact(id)` / `getArtifact(id)`	Asks Hue to verify the staged bytes against the declared identity / reads the reservation → `ArtifactReservation`.
`downloadArtifact(id)`	The verified bytes of one ready artifact in this project as a `Uint8Array`, bounded to 25 MiB.

ArtifactReservation has id, filename, declaredContentType, declaredBytes, declaredSha256, state (reserved, verifying, ready, rejected, cancelled or abandoned; only ready artifacts are verified and downloadable), copyState (none, started or acknowledged), verifiedBytes and verifiedSha256 (null until ready) and failureCode. ArtifactUpload is { uploadUrl, method: "PUT", headers?, expiresAt }; omitted or null headers mean the file content type only.

Recorded media

client.media is a MediaClient that reads recorded files back by SHA-256 with a Read or Read and write key; a snapshot’s pinned copy needs Read and write. See Read recorded media back.

Member	Result or behavior
`fetch(target: MediaFetchTarget, options?: MediaFetchOptions)`	Downloads one file through `GET /api/v1/traces/{traceId}/blobs/{sha256}`, without the project key, and checks its SHA-256 → `MediaFile` with `bytes`, `sha256`, `size`, `contentType` and `fileName` (both absent for a snapshot’s copy).
`MediaFetchTarget`	`{ traceId, sha256, spanId? }`, where `traceId` is Hue’s ID or the 32-hex OpenTelemetry ID and `spanId` limits the read to that span’s record and its correlated log records (a file a log record holds inline is found only this way), or `{ snapshotId, sha256 }` for a snapshot’s pinned copy.
`MediaFetchOptions`	`maxBytes` (default 64 MiB, at most 1 GB; a larger file fails as `too_large` before any byte downloads) and `signal`.
`resolve(value, options: MediaResolveOptions)`	Writes the bytes of every digest part in `value` back into the part’s own field → `MediaResolution` with a new `value`, the count `resolved` and `failures`.
`MediaResolveOptions`	`traceId` and optional `spanId`, or `snapshotId`; `as`; `blobs`, the record’s `hue.blobs` entries in any form Hue reads, which type a part that declares none and mark `prefix_only` parts; `concurrency` (1–16, default 4), `maxBytes` and `signal`.
`MediaWriteOptions`	`as: "dataUrl"` (the default) writes a `data:` URL where the part’s field takes one and bare base64 where its convention requires it; `as: "bytes"` writes a `Uint8Array`.
`MediaFetchError`	Thrown by `fetch`, with `reason` (`MediaFetchErrorReason`: `not_found`, `missing`, `unavailable`, `forbidden`, `too_large`, `digest_mismatch` or `failed`), `status` and `retryAfterSeconds`. A `forbidden` read of a snapshot’s copy says that it needs a Read and write key; the constructor’s `snapshot` option selects that message. An unavailable answer, a lost connection or a timeout is asked once more, after a `Retry-After` of up to 10 seconds.
`MediaResolveFailure`	A part left unchanged: `pointer` (RFC 6901), `sha256` and `reason`, a `MediaFetchErrorReason`, `not_found` for no matching local file, or `prefix_only` for a part holding only the first 16 KiB of its file.
`rehydrateMedia(value, files: MediaSourceFile[], options?: MediaWriteOptions)`	Writes case files back into the digest parts of `value` by SHA-256 → `MediaResolution`. A `MediaSourceFile` is `{ sha256, contentType?, path }`, such as a runner’s `LocalFile`, or `{ sha256, contentType?, bytes }`.

Published scenarios and eval sets

A published Scenario is a case converted from a trace, with immutable pins to a frozen dataset version, an environment version and one scorer version. The evaluation client reads them with listCaseConversions(page?), a page of CaseConversionSummary values (drafts and published), and getCaseConversion(id), a CaseConversion. Both need a Read and write key.

Type	Meaning
`CaseConversionSummary`	`id`, `domain`, `status` (`"draft"` or `"published"`), `revision`, `createdAt` and the source `traceId`.
`CaseConversion`	`id`, `status`, `publication` (`null` while a draft) and the optional summary fields.
`CaseConversionPublication`	The pins created on publication: `caseId`, `datasetId`, `datasetVersionId`, `environmentId`, `environmentVersionId`, `scorerId` and `scorerVersionId`.

These @hue-run/sdk/evals functions resolve a selector to the pins an experiment or runSimulation needs.

Function	Behavior
`listScenarios(client, page?)` / `getScenario(client, id)`	Call `listCaseConversions` / `getCaseConversion`.
`parseScenarioSelector(selector, segments = ["scenarios"])`	Returns a `ScenarioSelector`: `{ kind: "id", id }` for a UUID or a Hue URL containing `/<segment>/<uuid>` (query parameters ignored), otherwise `{ kind: "name", name }` with the trimmed name. Throws `TypeError` for an empty selector or a URL without a matching segment.
`matchByName(candidates, name)`	Case-insensitive matching over `NamedCandidate` values (`{ name }`): exact matches first, then prefix, then substring. Returns a `NameMatch`, `{ matches, exact }`.
`resolveScenarioPins(client, selector)`	Resolves a published Scenario by ID, Hue URL or dataset name → `ScenarioPins`. Throws `Error` when none matches, several match or the selection is an unpublished draft.
`resolveEvalSetPins(client, selector, { scorerVersionIds? }?)`	Resolves an eval set by ID, Hue URL, exact slug or name to its latest saved version → `ScenarioPins`. Without a saved version it returns the latest draft with `saved: false`, so you can freeze it explicitly. An eval set pins no scorers; pass `scorerVersionIds`.

ScenarioClient is the subset of EvaluationClient these resolvers use: listCaseConversions, getCaseConversion, getDataset, getDatasetVersion and listDatasets. ScenarioPins contains scenarioId (null for an eval set), name, datasetId, datasetVersionId, scorerVersionIds (exactly one for a published Scenario), environmentVersionId (null when the selection pins none), saved (whether the version is frozen; experiments require a saved version) and revision, the version’s current revision for freezing a draft.

Run locally and rescore

runExperiment(options: RunExperimentOptions): Promise<RunnerReport>
rescore(options: RescoreOptions): Promise<RunnerReport>

Both functions are exported from @hue-run/sdk/evals.

Option	Required/default
`client`	Required `EvaluationClient`.
`checkpointDirectory`	Required dedicated local directory for this run. Preserve it for upload recovery; keep it out of Git.
`persistResultContent`	Required boolean, independent of telemetry capture.
`scorers`	Optional array of bound `LocalScorer` definitions/callbacks.
`concurrency`	Default `20`; positive safe integer.
`schemaTimeoutMillis`	Default `2000`; integer `100–60000`; applies to JSON Schema validation, not arbitrary callbacks.
`environmentEvidence`	Optional `"required"`: resolve sealed instance evidence for local scoring and historical rescoring. `runSimulation` sets it. `"when_pinned"` skips the read for cases that pin no World while still requiring it for cases that do.
`filesDirectory`	Default `<checkpointDirectory>/files`. Holds verified input copies and generated files.
`deferUnboundLocalScorers`	Optional. Reports pinned `local_code` versions this process has no binding for in `deferredScorerVersionIds`, leaving them to another executor, instead of refusing the run.

runExperiment additionally requires hue, experimentId, traceEvidence and target(inputs, context), where context is a RunExperimentTargetContext. The target may return a JSON value, a TargetResult from withFiles, undefined, or a promise of any of these. Evidence must be { mode: "required" } or { mode: "omit", reason: "..." }. Telemetry and evaluation clients must use the same project and origin. For required trace evidence, traceNotAccepted defaults to "stop": a rejected export stops the runner. Choose "fail_case" to complete that case as failed without its output or generated files and continue with other cases. onTelemetryNotAccepted(entry) reports each newly failed case before continuing. The RunnerReport.telemetryNotAccepted array contains TelemetryNotAccepted entries with caseId, caseKey, executionId and issues. Each TelemetryIssueCount records the signal (traces or logs), kind (rejected, failed, dropped or invalid), optional HTTP status and record count; it contains no captured content. Choose "pending" to complete a case whose telemetry Hue did not accept in time in its true state with its trace evidence pending: Hue attaches the trace once it holds every span the case ended, or records the evidence as omitted a day later. RunnerReport.tracePending lists those cases as TracePending entries with caseId, caseKey, executionId and, when the checkpoint recorded it, spanCount. rescore additionally requires runId and has no target callback. Create its evaluation run from existing subject IDs first. Both return a RunnerReport, { runId, subjectIds, resultIds, deferredScorerVersionIds }. With persistResultContent: false, completions and checkpoints omit raw output, target error messages, scorer evidence and arbitrary explanations. Declared metric values remain stored, including text metrics. Already stored dataset inputs/references are independent of this setting. Historical scoring skips unavailable output; it does not rerun a target to reconstruct it.

File-based cases

RunExperimentTargetContext is what the runExperiment target receives for one frozen case:

Field	Meaning
`config` / `item`	Frozen experiment configuration, validated as JSON / the frozen `ExperimentCase`, cloned before invocation.
`span`	The `hue.experiment.case` `HueSpan` this attempt runs inside.
`executionId`	Identity of this attempt. Derive an environment run’s idempotency key from it to keep a resumed upload bound to the same instance.
`files`	`LocalFile` copies of the case’s pinned input files meant for the agent, verified by byte count and SHA-256. Evaluator-only roles are withheld.
`outputDirectory`	A private scratch directory for this case.

To save generated documents, return withFiles(output, files): it takes a JSON output (or undefined when the target produced only files) and an OutputFile[], and returns a TargetResult with readonly output and files. Generated files are uploaded regardless of persistResultContent, and local scorers receive every input and generated file in ScoreContext.files as LocalFile entries, generated ones with role: "output". rescore downloads a subject’s frozen files for code evaluators.

Export	Value or behavior
`targetFileRoles`	Roles the target receives: `source`, `attached_template`, `attached_reference` and `original`.
`outputFileLimits`	`{ count: 32, bytes: 26214400 }`: at most 32 generated files per execution, each at most 25 MiB.
`outputContentTypes`	The 22 accepted generated content types, the same for Python managed outputs: PDF, Word (`.docx`), PowerPoint (`.pptx`) and Excel (`.xlsx`) documents, `application/json`, `text/plain`, `text/csv`, `image/png`, `image/jpeg`, `image/webp`, `image/gif`, `image/avif`, `audio/mpeg`, `audio/wav`, `audio/ogg`, `audio/webm`, `audio/mp4`, `audio/flac`, `audio/opus`, `audio/aac`, `video/mp4` and `video/webm`. Declare `audio/mp4` or `video/mp4` for an `.mp4` file, and `audio/webm` or `video/webm` for a `.webm` file.
`safeFilename(name)`	Reduces a declared file name to one printable path segment of at most 200 characters, or `"file"` when nothing remains.
`OutputFileError`	Thrown when declared files cannot be used: too many, more than one `primary`, a missing or duplicate filename, an unsupported type, or a file that is unreadable, empty or larger than 25 MiB. The runner records it as the target’s `TargetError`, not an uncertain execution.
`ArtifactSizeError`	Stops an artifact download that exceeds the caller’s expected byte count, exposed as `maxBytes`.
`CaseFileError` / `CaseFileErrorCode`	Refuses a pinned input file before execution, instance creation or target invocation. `artifactId` identifies the file; `code` is `case_file_mismatch` for a size or SHA-256 mismatch, or `case_file_name_refused` for an unsafe file name in a simulated instance.

Local scorer exports

Export	Behavior
`builtins.exactMatch()`	Publishable typed JSON equality definition.
`builtins.includes(caseSensitive = true)`	Publishable string-inclusion definition.
`builtins.jsonSchema(schema: JsonValue)`	Publishable JSON Schema draft 2020-12 definition.
`defineLocalScorer({ source, entrypoint, metrics, score })`	Returns a `LocalScorer`; source is `string` or `Uint8Array`.
`sourceDigest(source: string \| Uint8Array)`	SHA-256 declaration for supplied source bytes/text.
`scoreLocally(version, context, { scorers?, schemaTimeoutMillis? }?)`	Computes `Promise<Score>` without uploading it.

JSON Schema scoring uses ajv, a separate optional peer dependency (8.17 or later) loaded inside a worker only when builtins.jsonSchema scores a case; without it that scorer reports SchemaValidatorUnavailable. The tracing core depends only on @opentelemetry/* packages. ScoreContext contains inputs, hasOutput, hasExpected, optional output/expected, metadata, executionState and, when the runner requires environment evidence, environment (an EnvironmentEvidence). With environment evidence present, a local_code scorer runs even when no output is stored. When required evidence cannot be read, local_code results record an EnvironmentEvidenceUnavailable error; an instance marked environment_incomplete produces a skipped result with the fixed explanation Environment incomplete: provider behavior is not implemented. A Score is scored with declared metrics and explanation/evidence, error with a typed error, or skipped with an explanation. A failed quality metric is still scored, with passed: false. Custom callbacks run locally, without a general execution timeout or side-effect cancellation. The source digest is a caller declaration, not attestation of dependencies or closures. Hosted judge pins must be dispatched through judge jobs; scoreLocally does not execute them.

Recovery errors

UncertainExecutionError exposes caseId and optional executionId: no durable outcome is available, so the runner refuses to repeat potentially completed work.
OutcomeSerializationError exposes executionId: the target returned, but its outcome could not be saved as JSON. The runner does not invoke it again.
TargetCancelledError is thrown by runExperiment() and runSimulation() when cooperative caller cancellation stops target execution; the run stays open, and rerunning with the same checkpoint directory retries the interrupted case as a new attempt.
TargetOutcomeUncertainError exposes executionId: the target or its simulated instance may have committed but acknowledgement is unavailable, so a resume will not invoke the target again.
TargetTimeoutError is thrown to a target still running five seconds past its instance’s expiresAt, through the signal in its context; the case ends as an error of type TargetTimeout and the instance is sealed abandoned.
TraceExportUnacknowledgedError is thrown by a resumed case whose saved outcome still waits for a trace export Hue never acknowledged; it is that case’s own state on resume, and the cases beside it still run.
Required trace export failures surface through HueExportError or recovery checks. Each export issue names the traces of the records it concerned, so a failure marks only the case whose trace it names; one naming no trace is decided by the case’s trace receipt. A new empty exporter cannot prove receipt of an earlier trace.

Use one checkpoint directory per experiment or rescore run. Checkpoint files contain allowed result content and are private local files, not encrypted storage. After an upload failure, resume with the same run, directory and options so saved payloads and idempotency keys are reused. If a process crashes, confirm its recorded owner has stopped before removing the directory’s .lock file. Removing a lock never authorizes another target invocation. When a target’s outcome is uncertain, restarting the runner will not repeat it. Investigate side effects first. The low-level startExecution API requires an explicit previousExecutionId for a replacement attempt and allowUncertainRetry: true to replace a still-started attempt. The convenience runner does not automatically adopt an externally created attempt; use a new experiment for an intentional fresh target run. Required evidence never silently changes to omitted evidence after an export failure. For a runnable experiment and historical rescore, see the public Local evaluations.

Wait for results and verdicts

Scorers that Hue executes grade after the runner returns, so a caller that wants per-case verdicts waits for them. These functions are exported from @hue-run/sdk/evals.

Function	Behavior
`waitForResults(client, options: WaitForResultsOptions)`	Polls an evaluation run until every item has a terminal (`scored`, `error` or `skipped`) result for every listed scorer version, or the budget elapses → `VerdictResults`. A timeout or abort returns the partial state with `complete: false`.
`summarizeVerdicts(results, { experimentItems, scorerVersionIds?, scorerVersions?, subjects? })`	Groups results by experiment case, in `experimentItems` order → `VerdictSummary`. Pass `subjects` when items do not carry `execution.subjectId`; `scorerVersionIds` defaults to versions seen in `results`, while `scorerVersions` lets the summary apply evaluator definitions.
`compareVerdicts(current, baseline)`	Diffs two `VerdictSummary` values by case key → `VerdictComparison`.
`collectExperimentVerdicts(client, options: CollectExperimentVerdictsOptions)`	Reads an experiment’s pinned scorer versions, waits for its results and summarizes them per case → `ExperimentVerdicts`.
`metricPassed(metric)`	Whether one `Metric` passes: its explicit `passed`, otherwise a boolean `value`; any other value counts as passing.

VerdictClient is the subset of EvaluationClient that waitForResults uses: listEvaluationItems, listResults and getResult. collectExperimentVerdicts also needs getExperiment, listExperimentItems and getSubject. WaitForResultsOptions requires runId (for example report.runId) and scorerVersionIds. Optional fields are subjectIds (restrict the wait, for example to report.subjectIds; defaults to every item), timeoutMillis (overall budget, default 300000; 0 reads once), pollIntervalMillis (default 2000; integer 250–60000) and signal. CollectExperimentVerdictsOptions takes the same optional fields with experimentId in place of runId and scorerVersionIds. Runs of more than 5,000 items throw RangeError.

Type	Meaning
`VerdictResults`	`complete`, the waited-for `items` (`{ id, subjectId }`) and `results`, one `VerdictResult` per item and scorer version read so far.
`VerdictResult`	`id`, `itemId`, `subjectId`, `scorerVersionId`, `state`, `metrics` (empty unless scored), `explanation` and `error` (`null` when absent). Optional `notApplicable` and `requires` identify an evaluator that does not apply and the missing requirement.
`CaseVerdict`	One row per case: `caseId`, `externalKey`, `subjectId` (`null` before completion), `state` (`passed`, `failed`, `error`, `skipped` or `pending`), `passed`, `metrics` tagged with their `scorerVersionId`, the persisted `explanations` of results that did not pass and the `errors` types. Evaluators marked not applicable are neither passes nor failures. A case needs at least one applicable evaluator and must pass every applicable pinned evaluator; no applicable evaluator produces an error.
`VerdictSummary`	`cases` (`CaseVerdict[]`) and `totals`: counts of `cases`, `passed`, `failed`, `error`, `skipped` and `pending`, plus `notApplicable`, the number of evaluator results that do not apply.
`VerdictComparison`	`improvements`, `regressions`, `unchanged` and `cases`, one `{ externalKey, before, after, change }` per case key in either run; `before` and `after` are `missing` for a case absent from that run, and `change` is `improved`, `regressed` or `unchanged`.
`ExperimentVerdicts`	`experimentId`, `runId`, the pinned `scorerVersionIds`, the raw `results` and the per-case `summary`.

Run a local agent worker

runLocalAgent(options: RunLocalAgentOptions): Promise<void>

runLocalAgent starts an outbound-only worker for one fixed local TypeScript agent entry point. Hue chooses only the registered key and revision; no command or source is received from the cloud. The worker is exported from @hue-run/sdk/evals and uses the same provider-aware instance lifecycle as runSimulation. A matching Hue API deployment is required. Register the process first with registerLocalAgent(input: LocalAgentRegistration), which returns a RegisteredLocalAgent. The worker then claims queued runs as a LocalAgentClaim. Candidate callbacks receive a LocalAgentTargetContext: case identity and cloned task inputs only, not expected criteria, case metadata or environment pins. Like SimulationTargetContext, it carries an optional world and an optional mcp. The worker sends the registered agent.revision as the agent revision of every instance it creates.

RunLocalAgentOptions

Option	Required/default
`client`	Required `EvaluationClient` for the worker project.
`environmentClient`	Required `EnvironmentClient` for the same Hue origin and project.
`hue`	Required `HueClient` used for target and tool spans.
`checkpointDirectory`	Required private directory for worker identity and execution checkpoints.
`agent`	Required `LocalAgentRegistration`: `key`, `name`, `revision`, optional `capabilities` and `scorerDigests`.
`target(inputs, tools, context)`	Runs cases pinned to a hosted World (`environment:v1`). Invokes the existing local agent with isolated `EnvironmentTool`s and a `LocalAgentTargetContext`. Supply `target`, `directTarget` or both.
`directTarget(inputs, context)`	Runs ordinary cases without a World (`direct:v1`) with a `LocalAgentDirectContext`; returns a JSON value, a `TargetResult` from `withFiles`, `undefined` or a promise of any of these.
`scorers`	Optional `LocalScorer[]` matching published source digests.
`concurrency`	Default `20`; positive safe integer.
`pollIntervalMillis`	Default `2000`; integer `250–60000`.
`signal`	Optional `AbortSignal`; stops polling cooperatively and does not cancel an active callback.
`maxRuns`	Optional. Useful for one-shot jobs; omit to keep polling. Counts runs that settled either way.
`maxRunAttempts`	Default `5`. A claimed run whose attempt fails transiently keeps its claim and is resumed from its checkpoints by the same process; after this many failures it is given up as `attention` for a project member to requeue. A failure that would recur is given up at once.
`worldTtlSeconds`	Optional lifetime of each instance the worker creates.
`onPollError(error)`	Optional callback for a registration or claim that failed transiently past the client’s retries; the worker polls again with a growing wait.
`onRunFailed(failure)`	Optional callback with a `LocalRunFailure`: the claim, the error, the attempt number and the `outcome` (`retry` with `waitMillis`, `attention` with `resumable`, or `refused` when Hue no longer accepts this worker’s outcome).
`deprecationWarnings`	Default `true`. Emits a one-time `DeprecationWarning` when a case runs on an instance without a handoff.
`actualAgentManifest` / `requestedProviders` / `mcpSurface`	Optional V2 provider-profile preflight, validated together before the worker polls.
`onCompleted(report)`	Optional callback after the experiment and queue completion are acknowledged.

LocalAgentDirectContext carries config (cloned), item identity only (id and externalKey), executionId, trace ({ traceId, spanId }), files (verified LocalFile copies of the case’s agent-visible input files), outputDirectory (a private scratch directory; return generated files with withFiles) and optional signal. Expected outcomes and case metadata stay with grading. Hue offers a worker only the cases whose capabilities its registration declares. localAgentCapabilities is { environment: "environment:v1", direct: "direct:v1" }. registeredCapabilities({ agent, target?, directTarget? }) returns the registration’s capabilities: the declared agent.capabilities plus one per supplied callback. It throws TypeError when neither callback is supplied or a declared environment:v1 or direct:v1 capability has no matching callback.

Run a simulation

runSimulation(options: RunSimulationOptions): Promise<SimulationReport>

runSimulation runs an existing agent callback against one fresh hosted instance per case. It owns immutable resolution, execution linkage, finalization, scoring and resumable uploads, and it is exported from @hue-run/sdk/evals. The environment client comes from @hue-run/sdk/environment and must use the same Hue origin as the evaluation client. Agents that call simulated apps through their own clients receive a World handoff instead of local tools. Provider-profile preflight is a control-plane contract. Hue does not serve hosted Gmail or Slack provider calls through it, and the options, methods and credential types below do not establish provider parity. Agents reach simulated apps through a World’s URLs instead.

RunSimulationOptions

Option	Required/default
`client`	Required `EvaluationClient` for the target Hue project.
`environmentClient`	Required `EnvironmentClient` for the same Hue origin and project.
`hue`	Required `HueClient` used for target and tool spans.
`checkpointDirectory`	Required dedicated private directory for resumable checkpoints. Running a changed definition or agent revision while an unfinished simulation is checkpointed there fails; recover that simulation first.
`definition`	Required `SimulationDefinition`. The deprecated `scenario` option accepts the same value and is still accepted.
`persistResultContent`	Required boolean for saved target and scorer content.
`traceEvidence`	Required `{ mode: "required" }` or `{ mode: "omit", reason }`.
`target(inputs, context)`	Required. Invokes the existing local agent exactly once for this attempt with a `SimulationTargetContext`; returns a JSON value, `undefined` or a promise of either.
`localScorers`	Optional `LocalScorer[]` bound by their declared source digests.
`concurrency`	Default `20`; positive safe integer.
`schemaTimeoutMillis`	JSON Schema worker deadline in milliseconds.
`maxSteps`	Per-instance action ceiling, `1–500`.
`ttlSeconds`	Per-instance lease in seconds, `1–86400`.
`agentRevision`	Optional agent revision under test, 1–256 characters, sent when each instance is created.
`deprecationWarnings`	Default `true`. Emits a one-time `DeprecationWarning` when a case runs on an instance without a handoff, or when you pass provider-profile options.
`signal`	Optional `AbortSignal`; cancellation is cooperative.
`runName`	Optional display name for the fresh experiment; defaults to the source experiment or definition name.
`onProgress(event)`	Optional callback receiving bounded nonsecret `SimulationProgress` events.
`actualAgentManifest`	Optional `ActualAgentManifestInputV2`, or a function of `{ config, item, signal }` returning one, that opts into immutable provider-profile preflight. Requires `requestedProviders` and `mcpSurface`. Omitted evidence normalizes to explicit V2 missing evidence; it is never assumed equal to the baseline.
`requestedProviders`	Optional `RequestedAttemptProviderV2[]`: the exact provider instances and ordered surfaces asserted for strict preflight. Supplied together with `mcpSurface`.
`mcpSurface`	Optional `{ providerInstanceKey, surfaceKey: "google.gmail/mcp" \| "slack/mcp" }` selected from `requestedProviders` and projected to `context.mcp`.

SimulationReport extends RunnerReport with experimentId, the fresh experiment created for this invocation, and runUrl, the browser URL joining task, trace, instance evidence and scoring. Invalid bounds throw RangeError; mismatched origins and an unfinished checkpointed definition or agent revision throw Error.

SimulationDefinition

SimulationDefinition is a union of an app-authored reference, already published pins and a repository-authored definition. SimulationScenario is a deprecated alias for SimulationDefinition.

Kind	Fields
`{ kind: "experiment", experimentId }`	Selects an existing app-authored immutable experiment template. Each completed invocation clones its frozen dataset version, configuration and scorer-version pins into a fresh experiment.
`{ kind: "pins", datasetVersionId, scorerVersionIds, config?, name? }`	Runs already published immutable pins, such as the `ScenarioPins` of a published Scenario. The frozen dataset version’s cases pin their World versions; `config` defaults to `{}` and `name` to the dataset name, then `Simulation`. The checkpoint identity binds these pins.
`{ kind: "repository", name, slug, description?, environment, cases, scorers, config? }`	Publishes and resolves the definition. `slug` is the stable dataset slug used for immutable version resolution; `environment` is an `EnvironmentIdentity` plus its `definition: PublishableEnvironmentDefinition`; `config` is passed to every target callback. At least one case and one scorer are required, and case keys must be unique.

RepositorySimulationCase is { externalKey, inputs, expected?, metadata? }: a stable key unique within the dataset, the JSON inputs passed to the target, an optional reference output and optional caller-owned metadata. RepositorySimulationScorer is an Identity plus scorer, a ScorerDefinition to publish or a LocalScorer bound to its source digest. Stable slugs reuse matching immutable content digests; changed definitions publish new versions. Unsupported or server-only scorer kinds throw TypeError before any write.

SimulationTargetContext

Field	Meaning
`config` / `item`	Frozen experiment configuration / the frozen `ExperimentCase`, including its immutable environment-version pin.
`executionId` / `environmentRunId`	Target execution identity / stable instance identity for adapter control operations such as coverage reporting. Neither is a credential.
`tools`	`Record<string, EnvironmentTool>`: framework-neutral local callables backed by this attempt’s isolated instance. Empty for an instance with a handoff, whose calls go to the provider mirrors in `world`.
`world`	Optional `WorldHandoff`: the provider mirror URLs, world token, environment variables and MCP configuration. Present only when Hue’s simulation gateway serves the instance; see world handoff.
`mcp`	Optional `SimulationMcpCapability` (`url`, `token`, `expiresAt`) for providers that execute MCP remotely; scoped to one execution and instance. With a handoff it is the instance’s first MCP mirror with the world token; without one it is the deprecated `hue_sim_` capability.
`connectionBundle`	Optional `AttemptConnectionBundleV2` with credential-bearing provider connections for this callback only. Hue never checkpoints, logs or adds it to parity digests.
`signal`	Optional `AbortSignal`; pass it into the real agent or provider call.

SimulationProgress

Progress events carry no target output, credentials or other sensitive content.

`type`	Fields
`run_created`	`experimentId` and `runUrl`, emitted as soon as the inspectable experiment exists.
`world_created` / `target_started` / `world_sealed`	`experimentId`, `executionId`, `caseId` and `environmentRunId`.
`attempt_prepared`	The lifecycle fields plus `bindingId`, `status` (`"ready"` or `"environment_incomplete"`), stable nonsecret `findingCodes` and, for ready attempts, the credential-free `executionManifestDigest`.

Attempt preflight

An experiment whose config carries an immutable AttemptBaselineV2 can require strict provider-profile preflight before the callback runs. runSimulation performs it when requestedProviders and mcpSurface are supplied; the evaluation-client methods and validators below are exported for adapters that drive it directly.

Method	Result or purpose
`prepareAttempt(input: PrepareAttemptRequestV2)`	Prepares the immutable provider-profile binding for one execution → `PrepareAttemptResultV2`. The route identity is removed from the JSON body, and credential-bearing responses are validated against the request before being returned; a malformed success response throws `HueApiError` without exposing details.
`getAttemptBinding(bindingId)`	Coupled, secret-free V1 or V2 binding evidence → `AttemptBindingRead`. It never reacquires credentials.
`refreshAttemptConnection(previous: AttemptConnectionBundleV2, { idempotencyKey })`	Rotates an unexpired V2 connection while preserving its immutable binding evidence → `RefreshAttemptResultV2`. The credential generation must advance by one and the stable evidence must be unchanged.
`revokeAttemptConnection({ bindingId })`	Revokes an attempt binding → `RevokeAttemptResult`. A revoked connection must not be reused or refreshed.

Type	Meaning
`ActualAgentManifestV2` / `ActualAgentManifestInputV2`	Canonical caller-observed agent configuration after missing evidence is made explicit: six strict `components` (`agent`, `prompt`, `model`, `tools`, `approvals`, `orchestration`), each `{ digest: string \| null, evidence: "observed" \| "declared" \| "missing" }`, MCP `catalogs` in effective model-visible order (never sorted) and native `helperConfigurations` / the input form, in which omitted evidence normalizes to missing.
`ExpectedAgentManifestV2`	Immutable expected agent evidence pinned by an experiment: a required `digest` and `minimumEvidence` (`"observed"` or `"declared"`) for every strict component, catalog and helper configuration.
`SurfaceBindingV2`	Secret-free registration identity for one selected provider surface. MCP surfaces (`google.gmail/mcp`, `slack/mcp`) carry a `catalogDigest`; native surfaces (`google.gmail/rest`, `slack/web-api`) carry a `helperConfigurationDigest`.
`DependencyProviderV2` / `DependencyManifestV2`	One provider instance (`providerId` `"google.gmail"` or `"slack"`) with its synthetic principal, sorted `scopes` and immutable profile and surface pins / the secret-free dependency manifest pinned to one attempt, with 1–16 uniquely keyed providers.
`AttemptBaselineV2`	Immutable V2 parity and provider baseline stored with an experiment version: `expectedAgentManifestId`, `expectedAgentManifestDigest`, `expectedAgentManifest` and `dependencyManifest`.
`RequestedAttemptProviderV2`	Exact provider instance and surface selection requested for an attempt: `providerInstanceKey` and `surfaceKeys` in effective order.
`PrepareAttemptRequestV2` / `PrepareAttemptInputV2`	Caller input to `prepareAttempt` (`schemaVersion: 2`, `idempotencyKey`, `executionId`, `environmentRunId`, `expectedAgentManifestDigest`, optional `actualManifest` and `requestedProviders`) / the canonical input after defaulting caller evidence.
`PreflightFindingV2` / `PreflightReportV2`	One strict parity, profile, binding or coverage finding with a stable `code`, nullable `component`, `providerInstanceKey` and `surfaceKey`, and a fixed nonsecret `message` / the credential-free decision: `status`, `evidenceSource: "caller_supplied"` and `findings`, empty when ready.
`ParityEvidenceV2`	Stable parity evidence stored without endpoints or credentials, including `actualAgentManifestDigest`, `dependencyManifestDigest` and `executionManifestDigest`.
`AttemptIdentityV2`	Stable identities shared by every credential generation of an attempt: `bindingId`, `executionId` and `environmentRunId`.
`AttemptConnectionBundleV2`	Credential-bearing V2 provider connections delivered only to the callback: `expiresAt`, a monotonic `credentialGeneration`, `providers` whose surfaces add an HTTPS `endpoint` and a short-lived `bearer`, and `parity`. Never persist or log it.
`AttemptBindingRead`	Secret-free persisted binding state, discriminated by `schemaVersion` 1 or 2 without converting V1 evidence; includes `outcome`, `credentialGeneration`, `createdAt` and `revokedAt`.
`PrepareAttemptReadyV2` / `PrepareAttemptIncompleteV2` / `PrepareAttemptResultV2`	Successful preparation with a finding-free report and a fresh `bundle` / inconclusive preparation with `bindingId`, its findings and the durable `gap`, after which no target or scorer may run / their union.
`RefreshAttemptResultV2` / `RevokeAttemptResult`	A successful rotation, the same shape as `PrepareAttemptReadyV2` / `{ bindingId, revokedAt }`.

The runtime validators are zod schemas (z.ZodType) for the types of the same name: actualAgentManifestV2, expectedAgentManifestV2, surfaceBindingV2, dependencyProviderV2, dependencyManifestV2, attemptBaselineV2, prepareAttemptInputV2, preflightFindingV2, preflightReportV2, parityEvidenceV2, attemptIdentityV2, attemptConnectionBundleV2 and attemptBindingRead. They reject unknown keys, check digests against immutable contents and, for connection bundles, require the parity digests to match the bundle’s own evidence and attempt identity.

Function	Behavior
`agentManifestDigestV2(manifest)`	Computes the canonical `sha256:` digest shared by actual and expected V2 agent manifests.
`executionManifestDigestV2(actualManifest, dependencyManifest, identity)`	Binds actual agent evidence, secret-free dependencies and stable attempt identities into one digest.
`secretFreeBindingV2(bundle)`	Removes endpoints and bearers from a validated bundle, retaining its dependency identity as a `DependencyManifestV2`.
`projectMcpConnectionV2(bundle, providerInstanceKey)`	The `SimulationMcpCapability` projection of the provider’s MCP surface, or `null`. This legacy `context.mcp` projection is credential-bearing and is never another source of binding truth.

World client

createWorldClient(options: WorldClientOptions): WorldClient uses the canonical World and instance routes. The compatible EnvironmentClient below keeps the original routes. Both require a Hue deployment serving their routes and use the same connection options and handoff helpers.

Canonical API	Meaning
`createWorld(input: EnvironmentIdentity)`	Creates an authored World identity. The compatible `createEnvironment` method remains callable; registry writes are not retried.
`listWorlds(page?: EnvironmentPageOptions)` / `getWorld(id)`	Lists or reads authored Worlds. The compatible `listEnvironments` and `getEnvironment` methods remain callable.
`createInstance(input: CreateInstanceInput, options?: { signal?: AbortSignal })`	Creates an isolated instance; use `worldVersionId` where the compatible `createRun` accepts `environmentVersionId`. `getInstance` and `finishInstance` read and finish it.
`WorldInstance` / `WorldCoverage` / `WorldVersionSummary`	Canonical names for `EnvironmentRun`, `EnvironmentCoverage` and `EnvironmentVersionSummary`.
`bindWorldTools` / `BindWorldToolsOptions` / `WorldTool`	Canonical names for `bindEnvironmentTools`, `BindEnvironmentToolsOptions` and `EnvironmentTool`.
`HueWorldError` / `isTransientWorldError`	The same error class and classifier as `HueEnvironmentError` and `isTransientEnvironmentError`.
`WorldDefinition` / `WorldDefinitionV1`	The V1 authored starting-state definition.
`WorldDefinitionV2` / `WorldProviderInstance` / `WorldCredentialFailure`	A V2 definition with separately identified app accounts and optional recorded credential refusals. Each binding names a registered `providerId`, `providerInstanceKey`, `syntheticPrincipalId`, app-owned `configuration` and optional `dialects`; support is validated by the server.
`PublishableWorldDefinition` / `WorldVersion`	Accepted V1/V2 definitions, including compatible environment definitions / an immutable version containing that definition.

Reconstructed Worlds

new WorldModelClient(options: WorldClientOptions) reads the retained history of a WorldEntity, { namespace, id }, scoped to the project key. A WorldModel records that identity, its committed evidenceCutoff and its status. Account bindings are WorldModelAccount values, including a full accountId, connection generation and optional scope; multiple bindings can belong to the same app.

Method	Result
`createModel(input: CreateWorldModelInput)` / `getModel(modelId)`	`WorldModelResult`, `{ worldModel }`. Create input is `{ entity, name? }`.
`listModels(options?: WorldModelPageOptions)` / `listAccounts(modelId, options?: WorldModelPageOptions)`	`WorldModelPage` / `WorldModelAccountPage`, each `{ items, nextCursor? }`; options are `{ after?, limit? }`, with `limit` from 1 through 100.
`materialize(modelId, { at }, options?: { signal?: AbortSignal })`	`StartingWorldResult`, `{ world: StartingWorld }`, without a running instance. `StartingWorld` pins the represented time, evidence cutoff and immutable digest.
`startInstance(modelId, input: StartWorldInstanceInput, options?: { signal?: AbortSignal })`	`WorldInstanceResult`, `{ world, instance }`. Input requires `idempotencyKey` and `at`, with optional `ttlSeconds`; `instance` is a fresh `WorldHandoff`.
`reproduceInstance(instanceId, input: ReproduceWorldInstanceInput, options?: { signal?: AbortSignal })`	A fresh instance from the original immutable starting World. Input requires `idempotencyKey` and accepts `ttlSeconds`.
`eraseModel(modelId)`	`ErasedWorldModel`, `{ erased: true }`. Erased starting Worlds cannot be reproduced.

When Hue reports progress, WorldModel.processing contains WorldModelProcessing: its indexing state, retained and ready evidence cutoffs, and whether evidence is pending. StartingWorldResult.preparation and HueWorldError.preparation optionally contain WorldPreparationProgress, which also names the selected evidence cutoff. Error progress includes only validated fields from at most 16 KiB of response data; unrelated response content is discarded. If a response carries world_insufficient_evidence or world_processing_limit, the client stops retrying that request. Other transient diagnostics retain the retry behavior below. Materialization, instance creation and reproduction wait up to 120 seconds only after HTTP 503 with X-Hue-Diagnostic: world_preparing, separately from ordinary maxAttempts. Retries preserve the serialized body and idempotency identity. Pass { signal } to cancel the wait or an active request. Other failures retain the configured attempt limit; a preparing or incomplete World is not a usable starting state.

Compatible environment client

import { createEnvironmentClient } from "@hue-run/sdk/environment";

const environments = createEnvironmentClient({ apiKey: process.env.HUE_API_KEY! });

createEnvironmentClient(options: EnvironmentClientOptions): EnvironmentClient and new EnvironmentClient(options) create a typed client for authored Worlds, isolated instances and immutable journals.

Option	Type	Default and behavior
`apiKey`	`string`	Required Read and write project service key sent as a bearer token; server-side only.
`baseUrl`	`string`	`https://app.hue.run`; the same origin rules as the tracing client.
`timeoutMillis`	`number`	`10000`. Per-request deadline in milliseconds, with the same range as the evaluation client.
`maxAttempts`	`number`	`4`; integer `1–10`. Attempts for idempotent run mutations.

Run mutations retry on transport failure and on HTTP 408, 429, 500, 502, 503 and 504 with exponential backoff, reusing the same invocation or idempotency identity. When a 429 or 503 carries Retry-After, the client waits that long instead, at most 10 seconds. createEnvironment and publishVersion are non-idempotent registry writes and are not retried. Failures throw HueEnvironmentError, whose fixed message never includes response text or credentials; status carries the HTTP status when Hue answered and is absent for transport, timeout or parse failure, and retryAfterMs carries Hue’s bounded Retry-After from a 429 or 503. Responses are limited to 4 MiB and request bodies to 1 MiB; a published definition may contain up to 240 KB.

Worlds and versions

Method	Result or purpose
`createEnvironment(input: EnvironmentIdentity)`	`EnvironmentSummary`. Creates an environment identity; not retried.
`listEnvironments(page?: EnvironmentPageOptions)` / `getEnvironment(id)`	An `EnvironmentPage` of active identities / an `Environment` with its immutable version summaries.
`publishVersion(environmentId, definition: PublishableEnvironmentDefinition)`	`EnvironmentVersionSummary`. Publishes an immutable definition; not retried.
`getVersion(id)`	The full `EnvironmentVersion` with its generated action catalog.

EnvironmentIdentity is { name, slug, description? }; EnvironmentSummary adds id and archivedAt (null while active), and Environment adds versions. EnvironmentVersionSummary is { id, version, contentDigest, createdAt }, where contentDigest is the canonical digest of the stored, defaulted definition; EnvironmentVersion adds environmentId, the stored definition and the generated actions. EnvironmentPage is { items, nextCursor } and EnvironmentPageOptions is { after?, limit? } with a limit from 1 through 100.

EnvironmentDefinition

EnvironmentDefinition is the complete V1 World definition stored as an immutable version. EnvironmentDefinitionV1 is that same V1 document. publishVersion and EnvironmentVersion.definition accept a PublishableEnvironmentDefinition, the union of EnvironmentDefinitionV1 and EnvironmentDefinitionV2. Narrow definition.schemaVersion === 1 before reading V1-only fields. Use EnvironmentDefinitionV2 only when supplying providerInstances.

Field	Meaning
`schemaVersion: 1`	Definition schema discriminator.
`description?`	Human-readable World description.
`determinism?.clock`	Virtual clock: `startNs`, the initial virtual nanoseconds as a decimal string (default `"0"`), and `stepAdvanceNs`, the nanoseconds advanced after each recorded step (default `"1000000"`).
`state.collections`	Initial normalized state: collections keyed by collection, entity and field names, each field a `JsonValue`.
`actions`	The closed catalog of `EnvironmentActionDeclaration` values exposed by this World.
`provenance?`	`{ kind: "handwritten" }` (default), `{ kind: "synthetic", note? }` or `{ kind: "derived_from_trace", traceId, revision }`.
`metadata?`	Caller-owned immutable metadata.

Each EnvironmentActionDeclaration has a unique name, an optional agent-visible description, flat params, semantics and an optional observation whose only V1 projection is "identity". An EnvironmentActionParameter has name, type ("string", "number", "boolean" or "string[]"), required (default true), optional description and optional enum. semantics.entry is a SemanticEntry, a versioned deterministic semantic implemented by Hue’s environment kernel: hue.collection.get@1, hue.collection.list@1, hue.collection.search@1, hue.collection.create@1, hue.collection.update@1, hue.gmail.search_messages@1, hue.gmail.get_message@1, hue.gmail.get_thread@1, hue.gmail.create_draft@1, hue.gmail.update_draft@1, hue.slack.search@1, hue.slack.read_channel@1, hue.slack.read_thread@1, hue.slack.draft@1 or hue.slack.destination@1. semantics.config is an EnvironmentSemanticConfig: the primary collection plus optional draftsCollection, slackActorId, slackWorkspaceId, mailboxAddress, idParam, queryParam, fields, filter ({ field, param }), limit, set, setFromParams, idPrefix, guards ({ field, equals, error } entries, default []) and notFoundError (default "not_found"). Publishing generates one ActionDefinition per action: name, description? and inputSchema, an ActionSchema. That is the JSON Schema subset { type: "object", properties, required, additionalProperties: false } whose properties are scalar types with an optional enum or bounded string arrays with maxItems and item maxLength. Unknown action arguments are rejected. EnvironmentDefinitionV2 extends that V1 document with schemaVersion: 2 and providerInstances: GmailProviderInstance[]. Each GmailProviderInstance supplies providerInstanceKey, providerId: "google.gmail", syntheticPrincipalId and a GmailMailboxConfiguration. That configuration is either GmailMailboxConfigurationV1 (kind: "gmail_mailbox/v1", messagesCollection, draftsCollection, mailboxAddress) or GmailMailboxConfigurationV2 (kind: "gmail_mailbox/v2" with the additional labelsCollection). These are SDK configuration types; they do not establish that a hosted provider is available. Hue canonicalizes syntheticPrincipalId to a lowercase UUID. These bindings are immutable attempt-profile inputs; they do not establish hosted Gmail calls. Limits: 500 actions per instance, an 86,400-second lease, 240 KB per published definition, 64 collections, 2,000 entities, 8 MiB of evidence, 16,000-byte coverage-gap arguments, 4 MiB responses and 1 MiB request bodies. isTransientEnvironmentError(error) reports whether an environment error is a connection failure or a retryable HTTP status (408, 429, 500, 502, 503, 504). It classifies a failure; calling it does not retry an action.

Runs, actions and journals

Method	Result or purpose
`createRun(input: CreateRunInput)`	`EnvironmentRun`. Creates or recovers one fresh isolated instance using a stable idempotency key.
`getRun(runId)`	`RunState`, the authoritative current or sealed instance state. Older servers that omit coverage fields read as `validity: "not_assessed"` and `coverageGap: null`.
`act(runId, input: ActionInput)`	`ActionResult`. Invokes an action; repeating an invocation identity replays its recorded result.
`listSteps(runId, page?: StepPageOptions)`	`StepPage`. Pages the immutable journal by step ordinal.
`finishRun(runId, input: FinishRunInput)`	`SealedRun`. Seals an instance as `completed` or `abandoned` and freezes its evidence.
`recordCoverageGap(runId, input: CoverageGapInput)`	`CoverageGapResult`. Records a known coverage gap with durable identity; retries reuse the exact request.
`getEvidence(runId, options?: WorldEvidenceOptions)`	`WorldEvidence`. Reads a sealed instance’s evaluator-only evidence; see world handoff.

CreateRunInput requires idempotencyKey and, unless the execution’s case pins a World model reference, environmentVersionId; optional fields are executionId (the target execution linked to this instance, required without environmentVersionId), seed (32 lowercase hexadecimal characters), maxSteps (1–500), ttlSeconds (1–86400), traceparent (the case span’s version-00 W3C trace context, whose trace ID must be the trace the execution declared; the instance span is parented on it) and agentRevision (1–256 characters, part of the instance’s fingerprint). Invalid bounds throw RangeError or TypeError before any request. ActionInput is { invocationId, action, args? }; omitted args means an empty object. FinishRunInput is { idempotencyKey, status }, completed for accepted outcomes and abandoned for failures or cancellation. StepPageOptions is { after?, limit? }: after is the last seen ordinal (-1 starts at the first step) and limit is 1–100.

Type	Meaning
`EnvironmentRun`	Newly created isolated instance: `id`, `environmentVersionId`, `clockNs`, `stateDigest`, `maxSteps`, `expiresAt` and the closed generated `actions` catalog. An instance the simulation gateway serves also carries `worldId`, `token`, `lifecycle`, `completingUntil`, `baggage`, `traceparent`, `surfaces`, `env` and `mcpConfig`; read them with `worldHandoff`.
`ActionResult`	`runId`, zero-based `stepOrdinal`, the `observation`, normalized `effects`, `stateDigest` and `clockNs` after the action, `replayed` and `stepsRemaining`.
`Observation`	The recorded answer returned by the simulated instance: `status` `"ok"` or `"error"` plus additional bounded fields. An error observation is an instance answer, not a transport exception.
`Effect`	One normalized effect on instance state: `kind` (`created`, `updated` or `deleted`), `collection`, `entityId` and changed `fields`.
`RunState`	`id`, `environmentVersionId`, `executionId` (`null` for a standalone run), `seed`, `status` (`open`, `completed`, `abandoned` or `expired`), `stepCount`, `maxSteps`, `clockNs`, `expiresAt`, `createdAt`, `sealedAt`, `stateDigest`, `finalState` after sealing and the `EnvironmentCoverage` fields. A gateway instance adds `worldId`, `lifecycle`, `completingUntil`, `traceExternalId`, `surfaces` and advisory `flags` (`WorldFlag`: `no_calls` or `fingerprint_differs`); a status read never returns the token.
`Step` / `StepPage`	One immutable journal entry: `id`, `ordinal`, `invocationId`, `action`, validated `args`, `observation`, `effects`, `mutated`, `clockNs` and `stateDigest` / `{ items, nextCursor }` with a numeric cursor.
`SealedRun`	`id`, requested terminal `status`, final `stepCount`, `stateDigest` and `sealedAt`. For a gateway instance, `lifecycle` is `completing` or `sealed`, and `sealedAt` is `null` until the seal follows the `completingUntil` grace.
`CoverageGapInput` / `CoverageGap` / `CoverageGapResult`	A durable first-fault report for known missing provider behavior (`idempotencyKey`, `provider`, `operation`, `code`, `args` bounded to 16,000 JSON bytes and a nonsecret `description`) / the stored gap with `reportedAt` and `reportedBy` (`{ kind: "project_key" \| "user", id }`) / `{ runId, validity: "environment_incomplete", coverageGap }`.
`EnvironmentCoverage`	`validity` (`"not_assessed"` or `"environment_incomplete"`) and `coverageGap` (`null` when none was reported). Missing fields from older servers mean `not_assessed`, never verified parity.

Bind local tools

bindEnvironmentTools(options: BindEnvironmentToolsOptions): Record<string, EnvironmentTool>

bindEnvironmentTools binds a run’s generated catalog to plain local callables without changing the agent framework. BindEnvironmentToolsOptions requires hue, the client that records each action as tool telemetry, client, an environment client bound to the same Hue origin (only act is used), and run, the fresh EnvironmentRun whose closed catalog becomes callables. parentContext supplies an optional parent span context for the generated tool spans; invocationId(action) is an optional durable invocation-ID factory for caller-owned resume state, and without it each call uses a random UUID. Each EnvironmentTool has name, description?, inputSchema (the action’s ActionSchema) and execute(args?), which runs the action inside hue.tool(name, args, ...) and resolves with the instance’s Observation. When the catalog entry names its MCP server, the tool span also carries mcp.server.name and mcp.server.version, plus hue.mcp.provider and hue.mcp.surface when the entry includes them. Transport failures reject with HueEnvironmentError.

World handoff

Where Hue’s simulation gateway serves an instance, createRun also returns its handoff: a world token and one mirror URL per pinned provider surface, such as a Gmail MCP or REST mirror. Point the agent’s own provider client at the mirror and pass the world token where the provider credential would go; the project key stays with the runner. worldHandoff returns null for an instance created while a deployment’s gateway is off. runSimulation, runLocalAgent and hue eval do this for you. They create each instance with the execution, a stable idempotency key, the case span’s trace context and the agent revision, pass the handoff as context.world, and finish the instance before completing the execution. To drive an instance yourself:

import { agentEnvironment, worldHandoff, writeMcpConfig } from "@hue-run/sdk/environment";

const run = await environments.createRun({
  idempotencyKey: `execution:${executionId}`,
  environmentVersionId,
  executionId,
  traceparent: `00-${span.traceId}-${span.spanId}-01`,
  agentRevision: "my-agent@1.4.2",
});
let status: "completed" | "abandoned" = "abandoned";
try {
  const world = worldHandoff(run); // null when the world has no handoff
  if (!world) throw new Error("This deployment does not serve gateway worlds");
  const config = await writeMcpConfig(world);
  try {
    // spawnAgent is your application function, not an SDK method.
    await spawnAgent({ env: agentEnvironment(world), mcpConfigPath: config.path });
    status = "completed";
  } finally {
    await config.dispose();
  }
} finally {
  // Seal a failed or unrun attempt as abandoned, never as completed.
  await environments.finishRun(run.id, {
    idempotencyKey: `execution:${executionId}:${status}`,
    status,
  });
}

Function	Behavior
`worldHandoff(run)`	The `WorldHandoff` of a create or replay response, or `null` for an instance without one. Keeps only `HUE_WORLD_`, `BAGGAGE`, `TRACEPARENT`, and each surface’s `HUE_SIM__URL`, `HUE_SIM__ALIAS` or `HUE_SIM__TOKEN` from the server’s `env`; other names are dropped.
`worldNow(source?)`	The instance’s clock as a `Date`, read from a `WorldHandoff` or `HUE_WORLD_NOW` in an environment map (default `process.env`); `null` when absent or invalid.
`agentEnvironment(world, options?: AgentEnvironmentOptions)`	Environment variables for the agent’s child process: `parent` (default `process.env`) without Hue control-plane credentials or instance variables (`HUE_WORLD_`, `HUE_SIM__URL`, `HUE_SIM__ALIAS`, `HUE_SIM__TOKEN`, `HUE_MCP_CONFIG`, `HUE_MCP_URL`, `HUE_MCP_TOKEN` and `HUE_MCP_EXPIRES_AT`), then the instance’s `env`, which wins. Use the filtered handoff from `worldHandoff`. Also sets `HUE_MCP_URL`, `HUE_MCP_TOKEN` and `HUE_MCP_EXPIRES_AT` from the first MCP mirror unless `legacyMcpVariables: false`. Pass `includeHueCredentials: true` only for an agent that must call Hue’s own API.
`writeMcpConfig(world, { directory? }?)`	Writes `mcpConfig` to an owner-only `mcp.json` in a private directory → `McpConfigFile`, `{ path, dispose() }`. Call `dispose` after the run; it removes the file and directory.
`legacyMcpCapability(world)`	`LegacyMcpCapability`, `{ url, token, expiresAt }` for the first MCP mirror, the shape adapters written for `hue_sim_` read; `undefined` without an MCP surface.
`stripHueControlPlaneCredentials(parent)`	A copy of `parent` without Hue control-plane credentials or `undefined` values.
`isHueControlPlaneCredential(name, value)`	`true` for a name in `HUE_CONTROL_PLANE_VARIABLES` (`HUE_API_KEY`, `HUE_MCP_KEY`, `HUE_PROJECT_KEY`, `HUE_SERVICE_KEY`) or a value carrying a prefix from `HUE_CONTROL_PLANE_CREDENTIAL_PREFIXES` (`hue_sk_`, `hue_attempt_`, `hue_mcp_`, `hue_setup_`, `hue_inv_`, `hue_install_`, `hue_at_`, `hue_rt_`, `hue_oauth_`) anywhere in it, including wrapped, percent-encoded or JSON-Unicode-escaped values. A world token or `hue_sim_` capability is not one.

Type	Meaning
`WorldHandoff`	`id`, `token` (a `hue_world_` credential that lives as long as the instance), `expiresAt`, `lifecycle`, `completingUntil`, `traceparent` (`null` when none was sent), `baggage` (`hue-world=<id>`), `surfaces`, `env` and `mcpConfig`. `env` holds `HUE_WORLD_ID`, `HUE_WORLD_TOKEN`, `BAGGAGE`, `TRACEPARENT` and one `HUE_SIM_<SURFACE ID>_URL` per surface.
`WorldLifecycle` / `WorldFlag`	`pending`, `live`, `completing` or `sealed` / advisory status flags, `no_calls` or `fingerprint_differs`.
`WorldSurface`	`provider` (such as `google.gmail`), `surface` (such as `google.gmail/mcp`), `providerInstanceKey`, the mirror `url`, `alias` (`null` while unavailable), optional account-scoped `token` and optional `connection`. Prefer the surface token to the world token when present.
`WorldSurfaceConnection`	Connection metadata: optional provider URL `replaces`, required `root` and accepted `auth` placements, and optional `accountIds`, `versions`, `headers` and `clients`. Token placeholders refer to the surface token when present, otherwise the world token.
`WorldSurfaceRoot` / `WorldSurfaceAuth`	The mirror origin’s `url` and optional provider origin `replaces` / a token placement with `header`, `scheme` (`bearer`, `basic` or `raw`) and `format`.
`WorldAccountId`	A call identifier’s `name`, world-specific `value` and `description`; platform routes may add `providerInstanceKey`, `dialect` and `binding`.
`WorldSurfaceVersions` / `WorldRequiredHeader`	A version-selecting `header`, `served` version strings and `required` boolean / an additional header’s `name`, `value` and optional `description`.
`WorldClientHint`	Client `package`, `language` (`javascript` or `python`), credential option name `credential` and `options` entries with `name` and `value`. Optional `note` and `unavailable` (`alias_host`, `account_id` or `https_only`) explain configuration limits.
`WorldMcpConfig` / `WorldMcpServer` / `WorldMcpHeaders`	The common `mcpServers` shape, keyed by provider instance / one Streamable HTTP server, `{ type: "http", url, headers }` / `{ Authorization }`, the world token as a bearer.
`WorldEvidenceOptions` / `WorldEvidenceSection`	`{ section?, bodies? }`: one section or `all` (default), and whether to include ledger request and response bodies (default `true`) / `all`, `start`, `end`, `diff` or `ledger`.
`WorldEvidence`	The sealed instance’s evaluator-only evidence as Hue returns it: start and end state, the diff, the call ledger, coverage and the fingerprint.

Finishing a gateway instance answers lifecycle: "completing" with sealedAt: null; the seal follows a 5-second grace so in-flight calls land. A later finish answers 409, which the runners treat as the seal they can no longer change. getEvidence needs the project key; a world token can never read evidence, and an open instance answers 409 until it is sealed. Nothing in these helpers logs the token. Keep the handoff out of your own logs and checkpoints. Hue-native tools and the hue_sim_ capability are deprecated. When a case runs on an instance without a handoff, runSimulation and runLocalAgent emit a one-time DeprecationWarning with code HUE_NATIVE_SIMULATION_TOOLS, and provider-profile preflight emits HUE_PROVIDER_FACADE; for a gateway instance, provider-profile options are ignored with HUE_PROVIDER_FACADE_IGNORED. Pass deprecationWarnings: false to silence them. hue eval never emits these warnings.

Local setup session

@hue-run/sdk/setup is the session core behind the hue setup CLI: a pure, resumable state machine that detects a project, plans the deliberately narrow automatic integration matrix, installs the runtime, wires the application and verifies a metadata-only setup probe — emitting secret-free versioned events. It is a separate local inspection path, not a simulation launcher. Every JSONL event carries SETUP_EVENT_CONTRACT_VERSION.

Anonymous setup on app.hue.run is inactive, so do not use hue setup to start or claim a trial project. This section documents the exported API.

Export	Role
`runSetup`	Runs one resumable setup command and emits exactly one terminal event.
`SetupRunOptions`	Inputs for one invocation of the resumable setup runner.
`SetupRunResult`	Terminal outcome and latest resumable state from one setup invocation.
`SetupBackendOperations`	Backend operations consumed by the resumable setup runner.
`SetupProjectAdapter`	Detects the supported language and safe setup plan for a project root.
`SetupCheckpointAdapter`	Persists secret-free setup progress across interruptions.
`transitionSetup`	Pure transition that advances the setup state without I/O, clocks, randomness or environment access.
`createInitialSetupState`	Creates the deterministic initial state without reading the filesystem.
`SetupMachineState`	Persisted phase of the pure setup state machine.
`SetupMachineInput`	Input returned by an injected setup adapter.
`SetupTransition`	Pure transition result; its events are templates completed by the runner.
`SetupEffect`	Side effect requested by the pure state machine.
`detectSetupProject`	Reads bounded manifest and lockfile metadata without importing or executing project code.
`SetupProjectDetection`	Static project facts read without executing repository code.
`planSetupApplication`	Statically recognizes the narrow automatic matrix without executing project code.
`SetupApplicationPlan`	Closed automatic application matrix; other projects require an explicit agent-owned integration.
`SetupApplicationActionRequired`	Stable reason an automatic application integration is not safe.
`SetupPlan`	A bounded local setup plan.
`installSetupRuntime`	Installs the exact runtime with the detected owner of the project’s manifest and lockfile.
`wireSetupApplication`	Adds only two owned wiring blocks around an existing app object; business logic is untouched.
`SetupFileChange`	A secret-free managed integration file created or replaced by setup.
`SetupCommand`	Fixed argv execution boundary for package managers and application entrypoints.
`SetupCommandRunner`	Executes a fixed argv without a shell and returns only its status.
`runSetupCommand`	Default command runner: fixed argv, no shell, bounded output and deadline.
`SetupBackendAdapter`	Real implementation of the frozen Setup HTTP protocol v1.
`SetupBackendAdapterOptions`	Construction options for that adapter.
`SetupBackendError`	Sanitized setup failure with a stable local or protocol code.
`SetupBackendClaim`	Non-secret account-claim state; bearer handoffs never appear in public adapter results.
`SetupBackendReceipt`	Deprecated alias of `SetupProbeEvidence`; still exported.
`SetupBackendTrial`	Deprecated alias of `SetupInstallationStatus`; still exported.
`SetupPreflightStatus`	Read-only public setup availability and fixed anonymous bounds.
`SetupCredentialResult`	Installation status plus one idempotently recovered telemetry credential.
`SetupProbeEvidence`	Exact receipt evidence for one real metadata-only setup probe.
`SetupApplicationEvidence`	Exact receipt evidence from a request through the repository’s existing application.
`SetupClaimHandoff`	Non-secret descriptor for the current one-time browser handoff.
`SetupInstallationStatus`	Strictly validated Setup HTTP protocol v1 installation status.
`FileSetupInstallationStore`	Owner-only, project/origin-scoped storage for installation proof and telemetry credentials.
`SetupInstallationRecord`	Secret local installation state; it must never be emitted or copied into diagnostics.
`SetupStoredCredential`	Telemetry credential saved only in an ignored owner-only installation file.
`SetupStoredProbe`	Exact metadata probe identifiers retained for resumable receipt verification.
`SetupStoredClaimHandoff`	Durable client identity for an idempotent one-time browser handoff request.
`SetupStoredApplicationAttempt`	Durable no-replay marker written before starting an existing application request.
`SetupStoredApplicationEvidence`	Exact identifiers emitted by a request through the repository’s existing application.
`SetupEvent`	Version 1 setup JSONL event union.
`SetupEventName`	Names in the version 1 setup event contract.
`SetupOutputMode`	Supported setup transcript formats.
`selectSetupOutputMode`	Chooses a safe default: ANSI only on an ordinary interactive terminal.
`renderHumanEvent`	Renders one lightweight append-only terminal entry; never moves the cursor or clears the screen.
`renderPlainEvent`	Renders one ANSI-free append-only transcript entry.
`renderJsonlEvent`	Renders one newline-free JSON object for JSONL output.
`RunStartedEvent`	An installer setup-session command invocation began; this is not a Hue Run.
`StepStartedEvent`	A named setup step began.
`StepCompletedEvent`	A named setup step completed.
`ProjectDetectedEvent`	Project detection completed.
`PlanReadyEvent`	The deterministic setup plan is ready.
`FileChangedEvent`	A future mutating adapter changed a project file.
`PrivacyNoticeEvent`	Published, non-blocking privacy and security disclosure shown before telemetry.
`ActionRequiredEvent`	Progress needs an explicit local or human action.
`DiagnosticEvent`	A secret-free diagnostic safe for terminals and transcripts.
`TrialCreatedEvent`	A backend adapter created an anonymous trial.
`ClaimRequiredEvent`	The anonymous project can be claimed by a person.
`ClaimCompletedEvent`	A backend adapter confirmed that the project was claimed.
`ReceiptVerifiedEvent`	A backend adapter verified instrumentation-only receipt evidence.
`RunCompletedEvent`	Exactly one successful terminal installer event ends a JSONL invocation.
`RunFailedEvent`	Exactly one failed terminal installer event ends a JSONL invocation.

Managed target adapter

createManagedTargetHandler(options: ManagedTargetOptions) from @hue-run/sdk/managed returns (request: Request) => Promise<Response>. Required options are machineCredential, target and flushTelemetry; optional options are tracer, baseUrl, maxExecutionMillis (default 90000, at most 90000) and finalizationMillis (default 30000, at most 30000). Pass tracer: hue.tracer with a Hue-owned client; without it the handler falls back to the global tracer, its span is not recorded and every invocation returns uncertain. The target receives a ManagedTargetContext: executionId, attempt, input, config, verified inputFiles (ManagedInputFile entries with artifactId, filename, contentType, byteSize, sha256 and role, plus their data bytes), an AbortSignal signal and the traceId of the ai.managed_target span. It returns a ManagedTargetResult: optional state, output JSON, a safe error summary, files (ManagedOutputFile entries with filename, contentType, data and optional primary) and provider-reported usage. ManagedInvocation describes the protocol-version-1 request body Hue POSTs. The helper claims before execution and saves outcomes before returning. It never automatically reruns the agent. See managed runs for the full setup and recovery flow.

Serving lifecycle

createHueSafe(options) accepts the same options as createHue, catches initialization failures and returns a disabled client with an instrumentation failure; the caller’s onExportIssue stays attached and receives a fixed initialization-failure issue, never the original error’s message or properties. Pass an empty string for an unset environment key; the safe constructor handles the invalid configuration. enabled: false needs neither a key nor a captureContent choice. flushSafe(options?: SafeLifecycleOptions) and shutdownSafe(options?: SafeLifecycleOptions) return Promise<SafeLifecycleResult>, { ok: boolean; timedOut: boolean; report: ExportReport }, without rejecting. SafeLifecycleOptions.timeoutMillis is the caller deadline, 1–60000 ms, default 1000. A deadline does not cancel a borrowed provider. The report also exposes pendingBytes, droppedSpans, droppedLogs and instrumentationFailures. Dropped records are subsets of failed records; do not add those counts together. Safe ok reflects cumulative failures for the client lifetime. Capture, metadata, setup and cleanup failures in Hue helpers omit telemetry and preserve the original callback result or error. Async onExportIssue rejections are contained and callbacks are rate-limited. See TypeScript production setup. In Vercel Functions, Hue registers background delivery with waitUntil without calling borrowed providers’ forceFlush() or consuming flush failure reports. An explicitly awaited flush keeps its acknowledgement contract. See Vercel Functions and Workflow.

​Imports

​Create a tracing client

​HueOptions

​ExistingHueProviders

​Trace application work

​SpanOptions and context

​ModelOptions

​HueSpan

​Provider-executed tools

​Composio and Pipedream

​Files

​Vercel AI SDK adapter

​Content policy

​Flush, shutdown and diagnostics

​Live spans

​Trace verification

​Evaluation client

​Evaluation types

​Eval sets, evaluators, runs and scoring

​CI evaluation invocations

​Tags

​Datasets and scorer definitions

​ScorerDefinition and metrics

​Experiments, saved subjects and results

​Hosted judge jobs

​Artifacts

​Recorded media

​Published scenarios and eval sets

​Run locally and rescore

​File-based cases

​Local scorer exports

​Recovery errors

​Wait for results and verdicts

​Run a local agent worker

​RunLocalAgentOptions

​Run a simulation

​RunSimulationOptions

​SimulationDefinition

​SimulationTargetContext

​SimulationProgress

​Attempt preflight

​World client

​Reconstructed Worlds

​Compatible environment client

​Worlds and versions

​EnvironmentDefinition

​Runs, actions and journals

​Bind local tools

​World handoff

​Local setup session

​Managed target adapter

​Serving lifecycle

Imports

Create a tracing client

HueOptions

ExistingHueProviders

Trace application work

SpanOptions and context

ModelOptions

HueSpan

Provider-executed tools

Composio and Pipedream

Files

Vercel AI SDK adapter

Content policy

Flush, shutdown and diagnostics

Live spans

Trace verification

Evaluation client

Evaluation types

Eval sets, evaluators, runs and scoring

CI evaluation invocations

Tags

Datasets and scorer definitions

ScorerDefinition and metrics

Experiments, saved subjects and results

Hosted judge jobs

Artifacts

Recorded media

Published scenarios and eval sets

Run locally and rescore

File-based cases

Local scorer exports

Recovery errors

Wait for results and verdicts

Run a local agent worker

RunLocalAgentOptions

Run a simulation

RunSimulationOptions

SimulationDefinition

SimulationTargetContext

SimulationProgress

Attempt preflight

World client

Reconstructed Worlds

Compatible environment client

Worlds and versions

EnvironmentDefinition

Runs, actions and journals

Bind local tools

World handoff

Local setup session

Managed target adapter

Serving lifecycle