Skip to content
Road to Intelligence

Concept · Chapter 13: Agents

Agent Evaluation

Must knowUnderstand7 minDifficulty

Agent evaluation checks final outcomes, constraint compliance and repeated-run reliability under a specified budget.

The problem

Fluent transcripts and selected successful runs can hide failed tasks.

The solution

Evaluate from a controlled initial state against explicit success conditions and retain the action trace.

The consequence

Scores depend on the task, harness, tools, model and available compute.

You should understand first

  1. Text as Data
  2. Probability and Distributions
  3. Conditional Probability and Bayes' Theorem
  4. Probability of Sequences
  5. Language Modeling
  6. Vectors
  7. Dot Product
  8. Embeddings
  9. Attention
  10. Softmax
  11. Self-Attention
  12. Causal Masking
  13. Entropy
  14. Loss Functions
  15. Cross-Entropy Loss
  16. Autoregressive Next-Token Prediction
  17. Pretraining at Scale
  18. GPT-1 → GPT-2 → GPT-3
  19. In-Context Learning
  20. The Turing Test
  21. Symbolic AI
  22. Logic and Rules
  23. Expert Systems
  24. Knowledge Representation
  25. The Knowledge-Acquisition Bottleneck
  26. From Rules to Learning
  27. Supervised, Unsupervised and Self-Supervised Learning
  28. Expected Value and Variance
  29. Reinforcement Learning
  30. MDPs, Policies and Value
  31. Decoding: Greedy, Temperature, Top-k, Top-p
  32. One-Hot Encoding
  33. Tokenization
  34. Chat Templates
  35. System Prompts and Instructions
  36. Structured Outputs and Constrained Decoding
  37. Tool Calling
  38. LLM Agents
  39. Agent Reliability and Budgets
  40. Features, Labels and Tasks
  41. Derivatives and Gradients
  42. Gradient Descent
  43. Linear Regression
  44. Logistic Regression
  45. Evaluation Metrics for Classifiers
  46. Agent Evaluation

The final state is the target

A coding task can be checked with tests and diff review. A scheduling task needs the correct saved record. Retain traces for diagnosis, but do not substitute apparent effort for a verified result.

τ-bench evaluates tool-and-user interaction against a target database state and examines consistency across repeated trials.

Two questions about repeated attempts

For independent runs of a fixed task with success probability pp, at least one success in kk attempts has probability 1−(1−p)k1-(1-p)^k. Success on all kk has probability pkp^k. These correspond to the questions behind pass@k and pass^k; empirical benchmarks may use finite-sample estimators.

At p=0.8p=0.8, five attempts give a 99.968% chance of at least one success, but only 32.768% of all five succeeding. A showcase choosing the best run and a user depending on repeated operations experience different systems.

Report failures, permission violations, costs and latency as well as success. Compare architectures under similar budgets and check for benchmark contamination or access to hidden evaluation artifacts.

What to remember

  • Agent evaluation checks final outcomes, constraint compliance and repeated-run reliability under a specified budget.
  • Evaluate from a controlled initial state against explicit success conditions and retain the action trace.
  • Scores depend on the task, harness, tools, model and available compute.

Key papers

Essential

SWE-bench: Can Language Models Resolve Real-World GitHub Issues?

Carlos E. Jimenez et al. · 2023

Makes real repository issue resolution an executable evaluation problem.

How to read it: Check how tasks and tests are constructed before interpreting a score.

~35 min readarXiv:2310.06770✓ verified 2026-10-05

Watch