Viteval

VitevalNext Generation Eval Framework

Define, run, and debug LLM evaluations with a familiar API

Get Started
View Examples
Viteval

Powered by Vitest

Define and run evaluations the same way you run tests, with a familiar and intuitive interface

Fully Featured

Run evals, create datasets, and more with everything you need out of the box

CI/CD Ready

Integrate seamlessly into your CI pipeline and run evaluations alongside your tests

See it in action ​

ts
import { evaluate, scorers } from 'viteval';

evaluate('Uppercase', {
  data: [
    { input: 'hello', expected: 'HELLO' },
    { input: 'world', expected: 'WORLD' },
  ],
  task: ({ input }) => input.toUpperCase(),
  scorers: [scorers.exactMatch()],
});

Viteval is free and open source,
made possible by wonderful sponsors.

Special Sponsors

Joggr