Examples
These examples cover deterministic checks, generated datasets, and model-based scoring. The first two run without an API key.
Extract order IDs
Check a deterministic extractor against casing differences and a missing ID. Use these same cases when replacing the task with a model call.
Reusable dataset
Generate valid IDs in both cases to check the normalization rule over more values. This adds coverage of the ID format, not the variety of real customer messages.
Multiple scorers
Install ai and @ai-sdk/openai, and provide OPENAI_API_KEY in the environment.
Register the SDK provider and judge/embedding defaults in Viteval configuration:
Then add the eval:
The threshold applies to the mean of the three scores. This example performs real generation, embedding, and judge requests. Judge scores vary by model and are not proof that an answer follows your policy. For exact extraction checks, use the model-backed evaluation guide.
Runnable examples
The repository examples include project configuration and environment loading:
- Basic Evaluation: deterministic evaluations without API keys.
- Vercel AI SDK: model-backed evaluations with the AI SDK.
- VoltAgent: evaluations for agent workflows.