For AI agents: the complete documentation index is available at https://viteval.dev/llms.txt, the full documentation bundle is available at https://viteval.dev/llms-full.txt, and this page is available as Markdown at https://viteval.dev/features.md.
V1 release candidate available. V0 support ends at the v1 launch.Migrate to v1 →
Star on GitHub

Why Viteval

Test prompts and agents
against reference cases.

Run your prompts and agents against real cases. Score the answers and catch regressions before you ship.

Explore featuresSee examples
Captured order extraction evaluation: three cases pass with ExactMatch scores of 1

Familiar tooling

Use your Vitest configuration

Keep native configuration, watch mode, retries, and CI exit codes. Import your application functions into an evaluation, then score their answers.

Write an eval

Your criteria

Score exact answers and free-form responses

Use deterministic checks for exact answers and structured output. Add embeddings or LLM judges for free-form responses, or define a scorer for your own criteria.

Choose a scorer

Local by default

Inspect results locally

Start with JSON datasets and results in your project. Open the local dashboard to inspect inputs, reference answers, outputs, and individual scores.

Open the dashboard

Run on your infrastructure

Run evaluations in CI

Run the same evaluations on your laptop, in CI, or on a runner you control. Return a nonzero exit code when a case fails its scoring threshold.

Set up CI Choose providers

Put your agent to the test.

Open source. No hosted account required.

Write your first evalSee examples