> For AI agents: the complete documentation index is available at https://viteval.dev/llms.txt, the full documentation bundle is available at https://viteval.dev/llms-full.txt.

# `evaluate()`

`evaluate()` registers an evaluation with a check for each case. It runs your task,
scores the output, and checks the threshold.

## Import

```ts
import { evaluate } from 'viteval';
```

## Signature

```ts
function evaluate<DATA extends Data, TASK_OUTPUT = InferDataOutput<DATA>>(
  name: string,
  {
    data,
    aggregation = 'mean',
    task,
    scorers,
    threshold = 1.0,
    timeout,
    concurrency,
    retries,
    results,
    skip,
  }: Eval<DATA, TASK_OUTPUT>
): void;
```

## Parameters


Parameters

namestringrequired

The suite name shown in Vitest and evaluation reports.

optionsEval<DATA, TASK_OUTPUT>required

Cases, task, scorers, and passing criteria.

Properties

dataData<DATA_ITEM>required

A case array, asynchronous generator, or reusable dataset. Empty
datasets fail the evaluation.

Case fields

inputINPUTrequired

Input passed to the task.

expectedEXPECTEDrequired

Reference value passed to scorers, not the task.

namestringoptional

Display name for the case.

...extraRecord<string, unknown>optional

Additional context or metadata passed to the task and scorers.

taskTask<INPUT, TASK_OUTPUT, EXTRA>required

Synchronous or asynchronous function receiving input and extra case
fields. Expected values are not passed to it. The returned output type
may differ from the expected-value type.

scorersScorer<TASK_OUTPUT, EXPECTED, EXTRA>[]required

At least one scorer. Each receives output, expected value, input, and
additional case fields. See
[createScorer()](https://viteval.dev/api/scoring/create-scorer.md).

thresholdnumberoptional

Default: `1`

A finite minimum for each case's aggregate score. The case passes when
its aggregate is greater than or equal to this value. Sum thresholds may
exceed 1.

aggregation'mean' | 'median' | 'sum'optional

Default: `'mean'`

Combines each case's scorer values. Null contributes zero. Task or
scorer exceptions and timeouts fail the case regardless of threshold.

timeoutnumberoptional

Positive per-case timeout in milliseconds, including task and scoring.

concurrencynumberoptional

Positive case concurrency limit, bounded by the worker limit.

retriesnumberoptional

Additional attempts after a failed case; reruns tasks and scorers.

resultsResultProviderName[]optional

Result destinations. Omitted inherits config; an explicit array replaces
it; an empty array disables persistence.

skipbooleanoptional

Register the evaluation without executing its cases.

descriptionstringoptional

Descriptive text that does not affect scoring or case discovery.

## Related types

```ts
type Data<DATA_ITEM extends DataItem> =
  DATA_ITEM[] | DataGenerator<DATA_ITEM> | Dataset<DataGenerator<DATA_ITEM>>;

type Task<INPUT, OUTPUT, EXTRA extends Extra> = (
  args: TaskArgs<INPUT, EXTRA>
) => Promise<OUTPUT> | OUTPUT;

type TaskArgs<INPUT, EXTRA extends Extra> = EXTRA & { input: INPUT };
```

Each case is a `DataItem`. Additional fields are preserved in `EXTRA`:

```ts
type DataItem<
  INPUT = unknown,
  OUTPUT = unknown,
  EXTRA extends Extra = Extra,
> = EXTRA & {
  name?: string;
  input: INPUT;
  expected: OUTPUT;
};
```

`Extra` holds additional case fields:

```ts
type Extra = Record<string, unknown>;
```

## Return value

Returns `void`. Evaluation outcomes are recorded during execution, not returned
as completed scores or runner objects by this call.

## Worker-local dependencies

`data` can select a registered integration's dataset handle. `task` and `scorers`
also accept factories that resolve dependencies from the worker-local context.
Generation destinations do not select the evaluation's read source:

```ts
evaluate('Reviewed support answers', {
  data: ({ langfuse }) => langfuse.dataset('reviewed-support'),
  task: ({ input }) => answerSupportQuestion(input),
  scorers: [scorers.factual()],
  results: ['json'],
});
```

Import and register the Langfuse integration in config before using its handle.
Storage integrations initialize lazily, fail the run on persistence errors, and
close after persistence. A successful write is not rolled back if another
destination fails.

## Example

```ts
import { evaluate, scorers } from 'viteval';

evaluate('Order extraction', {
  data: [
    { input: 'Please cancel ord-2087.', expected: 'ORD-2087' },
    { input: 'How do I update my address?', expected: '' },
  ],
  task: ({ input }) => input.match(/\bORD-\d+\b/i)?.[0].toUpperCase() ?? '',
  scorers: [scorers.exactMatch()],
  threshold: 1,
  timeout: 60_000,
  retries: 2,
});
```

## References

- [Evaluations and tasks](https://viteval.dev/concepts/evaluations.md) for the execution model
- [Configure models](https://viteval.dev/guide/models.md) for model-backed tasks and judges
