Start with a question

Who this is for

Evals AX is for people building the environments agents work in: tool authors, teams maintaining codebases, and product builders opening their interfaces to agents. A coding agent can use the CLI directly during development.

The framework runs experiments on your machine with your agent runtime. The platform keeps projects and experiment records alongside your website audits. Early access does not execute models in the cloud.

Inspect a website

Sign in, accept the current terms, then enter a public URL in Vibecheck. For CLI access, authenticate the host first. Reports contain observations from public HTML and companion files. Local inspection and public report reads do not require an account.

bunx --package @evals-ax/evx@https://evals.ax/downloads/evx-0.7.1.tgz evx vibecheck https://example.com --wait

npx --yes https://evals.ax/downloads/evx-0.7.1.tgz vibecheck https://example.com --wait

pnpm dlx https://evals.ax/downloads/evx-0.7.1.tgz vibecheck https://example.com --wait

yarn dlx -p @evals-ax/evx@https://evals.ax/downloads/evx-0.7.1.tgz evx vibecheck https://example.com --wait

The current auditor has 36 checks. Reports distinguish partial standards checks from unvalidated hypotheses and optional conventions. A report without flags is not proof that an interface is effective. Legacy scores remain in JSON for compatibility; they do not predict agent success.

Test a proposed change

Choose a task with a result you can independently verify. Prepare a baseline environment and a candidate that changes the interface. Record the hypothesis before the run. The experiment guide explains the manifest, local execution, and result format.

evx experiment init ./my-experiment
evx experiment validate ./my-experiment/manifest.json
evx experiment run ./my-experiment/manifest.json --output ./results

The starter is a fixture for learning the contract. Replace its task and verifier with your own before treating the result as evidence about a real agent.

Keep your work

Create a free account to keep report history and private project records. Use CLI sign-in to connect a host machine. Public website report links can be shared; project and experiment records require account access.

Already have reports saved in this browser? The platform asks explicitly before attaching them to the signed-in account.