How to compare LLM evaluation frameworks without getting sold to

TL;DR

• Eight criteria separate evaluation frameworks in practice: evaluator types, dataset management, trajectory depth, human review, CI gating, production sampling, cost reporting, and licence and hosting.
• Model-graded scoring is useful but biased. Zheng et al. (2023) report GPT-4 judges agreeing with humans over 80 percent of the time; Wang et al. (2023) show the same judges can be flipped by reordering candidates.
• Agent benchmarks justify a trajectory criterion. On tau-bench, GPT-4o solved under half the tasks; on Terminal-Bench 2.0, frontier agents score under 65 percent.
• Nine tools are read against the criteria from their own documentation. This is a docs check, not a hands-on test, and the table says where a capability could not be confirmed.

Evaluation is the part of building with language models that everyone agrees matters and most teams still do by eye. Comparing frameworks is hard because vendor pages converge on the same words: observability, quality, guardrails. This guide sets out the criteria that separate them, cites the research behind each criterion, and reads nine current frameworks against them using their own documentation.

Three jobs one word covers

“Evaluation” is used for three distinct jobs, and a framework can be good at one and poor at the others.

1. Offline evaluation runs a prompt, model or application against a fixed dataset before anything ships.
2. Online evaluation scores production traffic after the fact, with no reference answer, from the request, the response and whatever the trace recorded.
3. Agent evaluation treats a multi-step trajectory, with tool calls and intermediate decisions, as the unit under test.

Decide which of these you need before you open a vendor page.

The eight criteria

What the research says about judges

Most frameworks lean on LLM-as-a-judge, where a strong model scores another model’s output against a rubric. The reference paper is Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena by Zheng et al. (2023). The authors report that “strong LLM judges like GPT-4 can match both controlled and crowdsourced human preferences well, achieving over 80% agreement, the same level of agreement between humans.” The same paper names the failure modes: “position, verbosity, and self-enhancement biases.”

Those biases are not theoretical. Large Language Models are not Fair Evaluators by Wang et al. (2023) found that “the quality ranking of candidate responses can be easily hacked by simply altering their order of appearance,” and that with ChatGPT judging, Vicuna-13B “could beat ChatGPT on 66 over 80 tested queries.” Its fixes, swapping candidate positions and human review for uncertain cases, are the workflow a framework should make easy. A broader audit, Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge by Ye et al. (2024), identifies “12 key potential biases” and concludes that “significant biases persist in certain specific tasks.”

The consequence is two criteria, not one: model-graded evaluators because nothing else scales, and a human review path whose labels flow back to the judge, because the judge is measurably wrong in ways that depend on the task. Our editorial on why “hallucination” is the wrong frame makes a related point: a score is only as good as the failure taxonomy behind it.

Why trajectories need their own criterion

Agent benchmarks since 2024 show that single-answer scoring hides the failures that matter. tau-bench (Yao et al., 2024) simulates conversations between a user and an agent with domain-specific tools, then compares the final database state against an annotated goal state. It introduces pass^k, which asks whether the agent succeeds on all k trials of the same task. The paper reports that GPT-4o succeeds on “less than 50% of the tasks,” with “pass^8 less than 25% in retail.”

The follow-up, tau2-Bench (Barres et al., 2025), adds a telecom domain in which both the agent and the simulated user act on a shared environment, and reports “significant performance drops when agents shift from no-user to dual-control.” Terminal-Bench 2.0 (Merrill et al., 2026) covers 89 command-line tasks, each with “a unique environment, human-written solution, and comprehensive tests for verification,” on which “Frontier models and agents score less than 65%.”

Three things follow. The trace must capture every tool call with arguments and results. An evaluator must be able to read the trajectory, not just the final message. And reliability across repeated runs is a metric in its own right, so running the same case many times should be cheap.

A framework that scores only the last message is grading the destination and ignoring the route.

Nine frameworks against the criteria

Every entry below comes from the tool’s own documentation or repository as read in September 2026. Where a page did not state a capability, the cell says “not confirmed” rather than “no.” This is a documentation check, not a hands-on test, and it will age.

Hosted platforms with enterprise deployment

Maxim AI documents AI judges “with curated prompts (e.g., Clarity, Agent Trajectory),” statistical, voice and programmatic evaluators. Its online evaluation page describes filters, a sampling rate “to control what percentage of logs are evaluated,” and rate limits, noting that “sampling and rate limiting apply to trace-level evaluation only.” Human annotation covers internal and external raters. The self-hosting overview describes Zero Touch, where “Your data stays completely within your infrastructure,” and a hybrid Data Plane. No source licence is published.

LangSmith documents human, code, LLM-as-judge and pairwise evaluators. Its agent guide sets out final response, single step, and a trajectory evaluator that “calculates a score based on how many of the expected steps were completed correctly.” Online evaluators take filters, a sampling rate and weekly spend limits. Self-hosting is “an add-on to the Enterprise plan.”

Open-source platforms

Langfuse states its licence plainly: “This repository is MIT licensed, except for the ee folders.” Its LLM-as-a-judge evaluators run on “individual observations within your traces,” including tool calls, and annotation queues exist for “reviewing larger batches of traces, sessions and observations.” Self-hosting is documented for Docker Compose and Helm.

Arize Phoenix is under the Elastic License 2.0, which says “You may not provide the software to third parties as a hosted or managed service.” Arize’s licence page calls Phoenix “fully open-source”; ELv2 is not an OSI-approved licence, so the label depends on your definition. The documentation source describes annotations from “Human feedback,” “LLM-as-a-judge evaluations” and “Code-based evaluations,” and a pytest plugin in which a failed assertion “fails the pytest item, fails the run, and fails CI.”

Libraries that live in your test suite

DeepEval is Apache 2.0 and ships agentic metrics named Task Completion, Tool Correctness and Argument Correctness alongside G-Eval. Its evaluation guide says you can “include the command as a step in a .yaml file in your CI/CD workflows.” Human feedback and online evals are documented for the Confident AI platform, not the library.

Ragas is Apache-2.0 and began in retrieval metrics; our RAG versus long-context comparison explains why those failures need their own scores. Its agent metrics include Tool Call Accuracy, Tool Call F1 and Agent Goal Accuracy. Human review and production monitoring are not documented.

promptfoo is MIT and, per its README, stays so after joining OpenAI. Its tracing page adds an OTLP receiver with trajectory:tool-sequence and trajectory:goal-success assertions; the GitHub Action posts a comparison on the pull request. The self-hosting page is candid: “Self-hosting is not recommended for production use cases.”

Model-evaluation frameworks

OpenAI Evals is MIT with match, includes and model-graded templates plus a registry. Its README says custom code implementations are not currently accepted, though model-graded YAML evals are. It benchmarks models, not applications.

Inspect, from the UK AI Security Institute, is MIT and built around datasets, solvers and scorers, with a ReAct agent, MCP tools, sandboxes and an Agent Bridge for external agents. Its eval logs record each sample’s “input, output, target, and score” plus tool calls. The Human Agent “enables human baselining of agentic tasks” using “the identical dataset, sandbox, and scorer configuration that models use.”

How to run the comparison yourself

Take one workflow you actually run, ideally one that has broken before, and build the same evaluation in two or three candidates. Record how long each took, which criteria you had to work around, and who could operate it without help. Run the same case ten times and look at the spread; tau-bench’s pass^k result is a warning that one green run means little. If your model calls already pass through a gateway, its logs are the cheapest source of production samples; our field notes on running an AI gateway in front of every model call cover what those logs contain.

Limits of this comparison

This piece did not run any of the nine tools. Every cell in the table is a reading of documentation, which lags, overstates and omits. “Not confirmed” means the pages read did not state the capability; the tool may still have it. Pricing was excluded because tiers change faster than an article can, and judge accuracy because it depends on the rubric and model, not the framework. Two things would change the conclusions: a reproducible test of judge calibration workflows across these tools, and licence changes, which open-source projects in this space have made before.