`wrapScorer()`

Transform the output or expected value before passing them to an existing scorer. The wrapper preserves the scorer's name and forwards input and extra case fields.

Import

import { wrapScorer } from 'viteval';

Signature

function wrapScorer<
  OUTPUT,
  EXPECTED,
  INNER_OUTPUT = OUTPUT,
  INNER_EXPECTED = EXPECTED,
  EXTRA extends Extra = Extra,
>(
  scorer: Scorer<INNER_OUTPUT, INNER_EXPECTED, EXTRA>,
  options: WrapScorerOptions<OUTPUT, EXPECTED, INNER_OUTPUT, INNER_EXPECTED>
): Scorer<OUTPUT, EXPECTED, EXTRA>;

Parameters

The scorer receiving the transformed values.

Synchronous transformations. Omitted transformations pass the original value through.

Transform the task's output before scoring.

Transform the dataset's expected value before scoring.

Example

import { evaluate, scorers, wrapScorer } from 'viteval';

evaluate('Structured answer', {
  data: [{ input: 'What is 2 + 2?', expected: { answer: '4' } }],
  task: () => ({ text: '4' }),
  scorers: [
    wrapScorer(scorers.exactMatch(), {
      output: (value: { text: string }) => value.text,
      expected: (value: { answer: string }) => value.answer,
    }),
  ],
});

Return value and errors

Returns a scorer with the outer OUTPUT and EXPECTED types. The inner scorer's score and metadata are returned unchanged. Transformations are not awaited. Exceptions propagate to the evaluator and fail the case.

See createScorer() for the scorer callback contract.

#wrapScorer()

#Import

#Signature

#Parameters