Agentic AI

Why this dimension matters

Agentic AI changes the failure surface. Standard LLM evals do not catch runaway loops, cascading tool errors, or prompt-injection escalation through tool outputs. Organizations that ship agents without an agentic-specific eval harness and runtime guardrails experience production incidents at 5-10× the rate of orgs with them.

Signals we look for

• Standardised tool-use protocol adopted (e.g. Model Context Protocol)
• Agent evaluation harness distinct from LLM evals
• Runtime guardrails specific to autonomous execution
• Selective use of reasoning models for planning-heavy subtasks

What this dimension looks like at each maturity level

1. 1Aware

   No agentic systems in production. Some experimentation with prompt chains.

2. 2Exploring

   First agent prototypes using off-the-shelf frameworks. No formal evaluation.

3. 3Operational

   At least one agent in production with runtime guardrails and basic task-success metrics.

4. 4Integrated

   Standardized tool-use protocol (Model Context Protocol) adopted across multiple agents. Agent eval harness distinct from LLM evals.

5. 5Autonomous

   Planning-capable reasoning models drive dynamic tool selection. Continuous evaluation auto-improves prompts and workflows.

6. 6Transformative

   Fleet of collaborating agents including software- and browser-operating agents. Multi-agent orchestration is a first-class engineering discipline.

Common blockers

• Building agents on ad-hoc tool integrations rather than adopting MCP or an equivalent protocol.
• Using LLM evals for agents — missing trajectory quality, tool-call correctness, and cost-to-solution.
• No per-agent cost budget or step limit — first incident is a runaway loop.

How to move up a level

• Adopt Model Context Protocol (or equivalent) for all agent tool integrations.
• Build an agent eval harness that measures task-success rate, trajectory quality, and cost-to-solution.
• Impose per-agent budgets: max steps, max tool calls, max cost per task.
• Add human-in-the-loop checkpoints for irreversible actions (send email, transfer funds, delete data).

Related standards & regulation

• Model Context Protocol — Specification