Media

​

How files reach Hue

​

What the trace view shows

​

Automatic capture

• Message parts in the OpenTelemetry GenAI (blob, uri and file), Vercel AI SDK, OpenAI Chat Completions and Responses, Anthropic, Gemini, Vertex AI and Amazon Bedrock Converse formats, including OpenAI assistant audio and Responses computer screenshots.
• MCP tool results’ image and audio blocks, and embedded resource blocks that hold bytes (blob), named by the last segment of the resource’s URI. A text resource stays text.
• Pydantic AI binary parts, image-url, audio-url, video-url and document-url parts, and a tool’s BinaryContent return.
• LangChain image, audio, video and file content blocks with base64 data, a url or a provider file_id, in LangChain v1’s standard form or the earlier source_type form.
• OpenInference’s flattened image contents and OpenLLMetry’s JSON-encoded multipart prompts.
• A whole data: URL that declares its media type, anywhere in recorded input, output, tool arguments, tool results or metadata, except as the uri of a GenAI uri part. Hue treats a uri part as a reference and doesn’t read or upload a data: URL there; put inline bytes in a blob part instead.

• Messages and values: gen_ai.input.messages, gen_ai.output.messages, ai.prompt.messages, ai.prompt, input.value and output.value (what withSpan / span() records), traceloop.entity.input and traceloop.entity.output, and Langfuse’s langfuse.observation.input, langfuse.observation.output, langfuse.trace.input and langfuse.trace.output.
• Tool arguments and results: gen_ai.tool.call.arguments, gen_ai.tool.call.result, ai.toolCall.args, ai.toolCall.result, tool.parameters, tool_arguments and tool_response.
• Metadata: metadata, langfuse.observation.metadata and langfuse.trace.metadata (whole, or one .<key> per attribute), and ai.telemetry.metadata.<key>.
• Indexed messages: OpenLLMetry’s gen_ai.prompt.<n>.content and gen_ai.completion.<n>.content when the value is the JSON of a parts list, and OpenInference’s flattened llm.input_messages.<n>.message.contents.<m>.message_content.image.image.url and its llm.output_messages counterpart.

• A provider part loses its byte field and carries sha256 and size, keyed at the part, for example gen_ai.input.messages#/0/parts/1. An OpenAI assistant message’s audio loses audio.data, and the message carries the digest.
• Raw 16-bit PCM keeps the format its type declares: audio/pcm, audio/x-pcm or audio/L16 with a rate is listed as, for example, audio/pcm;rate=24000;channels=1, and OpenAI pcm16 audio as audio/pcm;rate=24000;channels=1.
• A whole data: URL string becomes a GenAI blob part, {"type":"blob","mime_type":"image/png","modality":"image","sha256":"<64 hex>","size":68}, keyed at the string. A value that is entirely one, such as the string input or output Langfuse’s SDK records as it is, becomes that part’s JSON text, keyed <attribute>#. A text data URL up to 64 KiB stays inline.
• A file inside a JSON-encoded tool string takes the digest form inside the string. No JSON pointer can address inside a string, so Hue matches it by digest.
• Text media over 64 KiB of decoded bytes keeps a UTF-8-safe 16 KiB prefix in its content field, keyed at that field, for example gen_ai.input.messages#/0/parts/1/content. A text file that a GenAI blob, Vertex media, or AI SDK file or image part holds as bare base64 is decoded first: over 64 KiB of decoded bytes, it uploads and takes the digest form at the part, like binary media.

​

Attach a file

import { readFile } from "node:fs/promises";
import { Attachment } from "@hue-run/sdk";

const png = await readFile("scan.png");
await hue.withSpan(
  "review contract",
  (span) => {
    span.setInput({
      question: "Which clauses changed?",
      contract: new Attachment({ path: "contract.pdf", contentType: "application/pdf" }),
    });
  },
  { attributes: { metadata: { scan: new Attachment({ data: png, contentType: "image/png" }) } } },
);

from pathlib import Path

from hue_sdk import Attachment

png = Path("scan.png").read_bytes()
with hue.span(
    "review contract",
    attributes={"metadata": {"scan": Attachment(png, content_type="image/png")}},
) as span:
    span.set_input(
        {
            "question": "Which clauses changed?",
            "contract": Attachment(path="contract.pdf", content_type="application/pdf"),
        }
    )

​

Record a file

await hue.withSpan("voice turn", async () => {
  hue.recordFile({ role: "input", mediaType: "audio/wav", path: "turns/0042-user.wav", name: "user.wav", upload: true });
  hue.recordFile({ role: "output", mediaType: "audio/wav", path: "turns/0042-reply.wav", name: "reply.wav", upload: true });
});

with hue.span("voice turn") as span:
    span.record_file(role="input", media_type="audio/wav", path="turns/0042-user.wav", name="user.wav", upload=True)
    span.record_file(role="output", media_type="audio/wav", path="turns/0042-reply.wav", name="reply.wav", upload=True)

​

Upload behavior

• A file refused when its record is exported, because the receiver has no upload route, uploads were ended by shutdown or a budget ran out, keeps its digest without a hue.blobs entry in gen_ai.input.messages, gen_ai.output.messages and ai.prompt.messages; in any other attribute it keeps its bytes inline, under the 1 MiB value limit.
• A file whose upload fails after its record was exported keeps its digest and its entry, and shows Not uploaded until the same bytes are uploaded under the trace.
• A span’s own string attribute over the inline limit uploads whole, up to 1,000,000,000 bytes, and keeps a 16 KiB prefix inline.
• A file that cannot be uploaded never fails the export. It counts in uploadFallbacks in hue.transport.getReport() (TypeScript) or hue.export_status.upload_fallbacks (Python); successful uploads count in uploadedValues / uploaded_values.

​

Remote references

​

Attach files without the SDK

1. Reserve the upload with POST https://app.hue.run/api/v1/otlp/blobs and a JSON body of traceId (the span’s 32-hex trace ID), sha256 (the lowercase hex SHA-256 of the bytes), byteSize, contentType (the media type) and key (the file’s hue.blobs key, below). If Hue already holds the bytes for this trace, it answers {"status": "exists", "ref": …}; go to step 4. Otherwise it answers {"status": "upload", "url": …, "method": "PUT", "headers": …, "expiresAt": …, "ref": …}.
2. Send the bytes with PUT to url within 15 minutes, with exactly the returned headers and no Authorization header. A 412 answer means the bytes are already stored.
3. Complete the upload with POST https://app.hue.run/api/v1/otlp/blobs/complete and a JSON body of traceId and sha256. Hue checks the stored bytes against the reservation.
4. Name the file on a span of that trace and list it in the span’s hue.blobs attribute. The span can reach Hue before or after the upload, so you can export it first and upload the file afterwards. Retry a 429, a 5xx or a network failure a few times, never sooner than its Retry-After. Hue shows the file once its bytes land.

• A hue.file event, for a file the work read, received or produced: hue.file.sha256, hue.file.size, hue.file.media_type, hue.file.role (input, attachment or output) and an optional hue.file.name. Its key is event:hue.file:<sha256>, and the file appears with the span’s other files.
• A message part in digest form, for a file inside recorded messages: the part carries sha256 and size in place of its content, for example {"type": "blob", "modality": "image", "mime_type": "image/png", "sha256": "<64 hex>", "size": 48213}. Its key is the attribute and the part’s JSON pointer, such as gen_ai.input.messages#/0/parts/1, and the file appears in place in the message.

import hashlib
import json
import os
import urllib.error
import urllib.request

from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor

HUE = "https://app.hue.run"
api_key = os.environ["HUE_API_KEY"]  # Raises when unset; keep the key in the server environment.


def post(path, body):
    request = urllib.request.Request(
        HUE + path,
        data=json.dumps(body).encode(),
        method="POST",
        headers={"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"},
    )
    with urllib.request.urlopen(request, timeout=60) as response:
        return json.load(response)


def attach_file(span, data, media_type, name, role="input"):
    """Uploads a file under the span's trace, records its hue.file event and returns its hue.blobs entry."""
    trace_id = format(span.get_span_context().trace_id, "032x")
    sha256 = hashlib.sha256(data).hexdigest()
    reserved = post("/api/v1/otlp/blobs", {
        "traceId": trace_id,
        "sha256": sha256,
        "byteSize": len(data),
        "contentType": media_type,
        "key": f"event:hue.file:{sha256}",
    })
    if reserved["status"] == "upload":
        put = urllib.request.Request(reserved["url"], data=data, method="PUT", headers=reserved["headers"])
        try:
            urllib.request.urlopen(put, timeout=300).close()
        except urllib.error.HTTPError as error:
            if error.code != 412:  # 412: the bytes are already stored.
                raise
        post("/api/v1/otlp/blobs/complete", {"traceId": trace_id, "sha256": sha256})
    span.add_event("hue.file", {
        "hue.file.sha256": sha256,
        "hue.file.size": len(data),
        "hue.file.media_type": media_type,
        "hue.file.role": role,
        "hue.file.name": name,
    })
    return json.dumps({**reserved["ref"], "name": name}, separators=(",", ":"))


provider = TracerProvider(resource=Resource.create({"service.name": "file-demo"}))
provider.add_span_processor(
    BatchSpanProcessor(
        OTLPSpanExporter(
            endpoint=f"{HUE}/api/v1/otlp/v1/traces",
            headers={"Authorization": f"Bearer {api_key}"},
        )
    )
)
tracer = provider.get_tracer("example.instrumentation")

with tracer.start_as_current_span("agent.request") as span:
    with open("chart.png", "rb") as file:
        entry = attach_file(span, file.read(), "image/png", "chart.png")
    span.set_attribute("hue.blobs", [entry])  # Every file the span names, in one list.
    print("Trace ID:", format(span.get_span_context().trace_id, "032x"))
provider.shutdown()  # Flushes the batch processor and the exporter.

​

Attach a voice turn

import io
import wave


def wav_from_pcm16(pcm, rate=24000, channels=1):
    """Wraps 16-bit little-endian PCM, such as OpenAI's pcm16 (24 kHz mono), in a WAV header."""
    buffer = io.BytesIO()
    with wave.open(buffer, "wb") as wav:
        wav.setnchannels(channels)
        wav.setsampwidth(2)
        wav.setframerate(rate)
        wav.writeframes(pcm)
    return buffer.getvalue()


with open("user.pcm", "rb") as file:
    user_pcm = file.read()
with open("reply.pcm", "rb") as file:
    reply_pcm = file.read()

with tracer.start_as_current_span("voice.turn") as span:
    span.set_attribute("hue.blobs", [
        attach_file(span, wav_from_pcm16(user_pcm), "audio/wav", "user.wav", role="input"),
        attach_file(span, wav_from_pcm16(reply_pcm), "audio/wav", "reply.wav", role="output"),
    ])
provider.shutdown()

​

Upload errors

​

Read files back

import { createEvaluationClient } from "@hue-run/sdk/evals";

const client = createEvaluationClient({ apiKey: process.env.HUE_API_KEY! });

// One file by digest; pass the span that recorded it when you know it.
const file = await client.media.fetch({ traceId, spanId, sha256 });
console.log(file.contentType, file.fileName, file.size, file.bytes.length);

// Every digest part of a recorded value, written back in its own field.
const recorded = JSON.parse(attributes["gen_ai.input.messages"]);
const { value: messages, failures } = await client.media.resolve(recorded, { traceId, spanId });

import json
import os

from hue_sdk.evals import EvaluationClient

client = EvaluationClient(api_key=os.environ["HUE_API_KEY"])

# One file by digest; pass the span that recorded it when you know it.
file = client.media.fetch(trace_id=trace_id, span_id=span_id, sha256=sha256)
print(file.content_type, file.file_name, file.size, len(file.data))

# Every digest part of a recorded value, written back in its own field.
recorded = json.loads(attributes["gen_ai.input.messages"])
resolved = client.media.resolve(recorded, trace_id=trace_id, span_id=span_id)
messages, failures = resolved.value, resolved.failures

​

Download over HTTP

• traceId is Hue’s trace ID or the 32-hex OpenTelemetry trace ID.
• sha256 is the file’s lowercase hex SHA-256, as MCP get_span lists it under files.
• The optional spanId query parameter, a 16-hex OpenTelemetry span ID, reads only that span and its correlated log records, which is faster in a long trace. A file that a log record holds inline, or that a span holds in a format Hue began reading after the trace was recorded, is found only with spanId.

{
  "sha256": "<64 hex>",
  "size": 48213,
  "contentType": "image/png",
  "fileName": "chart.png",
  "url": "https://…",
  "method": "GET",
  "expiresAt": "2026-10-12T18:05:00.000Z"
}

​

Verify uploaded media

1. Flush with a budget that covers the uploads, then check uploadFallbacks / upload_fallbacks is unchanged.
2. Read the span with MCP get_span: each file in files should have state: "stored". missing means the bytes never arrived. With include_content: true, each file also carries a five-minute download_url; treat it as a credential and don’t paste it into chats or reports.
3. Download one file with client.media.fetch or the HTTP endpoint and compare its SHA-256 and size with the bytes you sent.

​

Files in cases

​

Coming from Langfuse or Braintrust