For AI agents: the complete documentation index is available at https://viteval.dev/llms.txt, the full documentation bundle is available at https://viteval.dev/llms-full.txt, and this page is available as Markdown at https://viteval.dev/api.md.
V1 release candidate available. V0 support ends at the v1 launch.Migrate to v1 →
Star on GitHub

API reference

Configuration
Configure Viteval in `vitest.config.ts` using Vitest's `test` options. Initialize models and storage providers in a worker setup file.
createScorer()
`createScorer()` gives a scoring function a name. Use the returned scorer in an evaluation's `scorers` array.
defineDataset()
`defineDataset()` wraps a case generator in a reusable dataset. Pass the dataset to `evaluate()` or save it with `viteval data`.
evaluate()
`evaluate()` creates a Vitest suite with a test for each case. It runs your task, scores the output, and checks the threshold.
Scorers
Viteval includes deterministic scorers, embedding similarity scorers, and LLM judges. Call each factory to create a scorer. LLM and embedding scorers use the models initialized in Vitest setup. See [Configuration](/api/configuration).