Agent-Bench
A benchmark suite specifically for agentic AI capabilities across coding, OS interaction, database queries, and web browsing. Measures the kinds of failure modes (planning, error recovery, tool use) that generic LLM benchmarks miss.
Related terms
- AI Agent
An autonomous AI system that can perceive its environment, make decisions, and take actions to achieve specific goals. Agents can use tools, interact with APIs, and coordinate with other agents.
- Eval Harness
The full pipeline that runs a set of evaluation prompts through one or more models, scores the outputs (via reference match, LLM-as-judge, or human review), and reports results. Common open-source harnesses: lm-eval-harness, HELM, OpenAI evals. Enterprise programs usually build a custom harness against a golden-dataset backend.
Related on this site
Where this fits
Agent-Bench is part of the Evaluation & Governance vocabulary used in the Generative AI Maturity Framework. See the full glossary for the complete set of 149 defined terms, or take the free maturity assessment to see where your organisation stands.