Measure effort without inventing data

A measurement needs provenance

Model identifiers, reasoning settings, token usage, elapsed time, and monetary cost help explain an outcome only when their origin is known. Runtime-reported usage and externally supplied telemetry are different evidence sources. Missing usage is unknown, not zero.

The experiment record combines independently verified success with available execution measurements. A hook can report an event or interruption, but a label such as confusion or intent misalignment needs an operational definition before it becomes a comparable metric.

evx telemetry ingest --file event.json
evx telemetry list

Inspect provider events before sharing them

evx hooks adapt reads a captured provider JSON or JSONL file and returns the normalized events with explicit coverage. It makes no network request. Generate one run ID for a capture and retain it for retries; formats without stable event identifiers require it.

RUN_ID=$(node -p 'require("node:crypto").randomUUID()')
evx hooks adapt --format codex-jsonl --file codex.jsonl --run-id "$RUN_ID"

Every adapter leaves task outcome unverified and reasoning level unknown. Provider completion is not an independent task verifier. The Claude result adapter requires exactly one result to avoid summing ambiguous cumulative costs. These are format adapters; they do not establish continuous runtime coverage, and the Claude integrations have not been qualified with a live Claude session.

Prompts, tool arguments, response text, raw errors, working directories, and transcript paths are discarded. The adapter never opens a referenced transcript automatically. Only the selected measurements and bounded provenance fields enter the event record.

After inspecting the local output, an authenticated host can explicitly upload the same capture. Keep the run ID unchanged for a retry. --file - also accepts standard input.

evx hooks ingest --upload --format codex-jsonl --file codex.jsonl --run-id "$RUN_ID"
evx telemetry list

Separate measured cost from an estimate

A token-derived estimate depends on the exact model, provider pricing, and any cache or batch discounts. An invoice includes costs that token counts may omit. Store the estimate's assumptions rather than presenting it as an observed bill.

Local agent execution uses your runtime and provider account. Early-access cloud storage does not run a model on your behalf. Website auditing still consumes network, function, and database resources even when no model is involved.

Bound the experiment

Set execution timeouts and the repetition count before running. Platform admission limits bound public audit throughput. Operational controls can pause new work when resource usage requires intervention; a pause should be visible as an explicit error rather than a silently missing result.

Share the smallest record that supports the analysis. Review traces for credentials and private task data before uploading. Aggregate costs without discarding the identifiers needed to investigate an outlier.

Early-access storage limits

Each account can store 50 projects and 200 experiment records, with a total storage limit of 50 MB and a 2 MB limit per result upload. The usage API reports the account's consumption.

Telemetry events are retained for 30 days. Anonymous public report links expire after 7 days. A daily cleanup removes expired data within a further 24 hours. Save work to an account when you need persistent ownership, and keep local experiment artifacts as your original record.