Define judge models — judges • evalthat

Judge models allow for automated grading of model output (with grade_model()) by passing responses to other LLM judges.

To learn more about how evalthat makes use of judge models, see vignette("Model grading", package = "evalthat").

Usage

judges(...)

Arguments

...: Named (optionally) ellmer::Chat() objects, e.g. the output of ellmer::chat_openai() or ellmer::chat_claude().

Examples

library(ellmer)

j <- judges(
  gpt4o = chat_openai(model = "gpt-4o"),
  claude = chat_claude()
)
#> Error in openai_key(): Can't find env var `OPENAI_API_KEY`.

j
#> Error: object 'j' not found

# doesn't necessarily need to be named:
judges(chat_openai(model = "gpt-4o"), chat_claude())
#> Error in openai_key(): Can't find env var `OPENAI_API_KEY`.