Compound AI System
An AI system that solves tasks by orchestrating multiple components — LLMs, retrievers, tools, validators, classical code — rather than relying on a single monolithic model call. Coined by the Berkeley AI Research group in 2024 and now the dominant pattern for production GenAI. Contrast with a "single-shot LLM call".
Last reviewed:
Related terms
- Agentic AI
AI systems that can act autonomously to accomplish goals, make decisions, and take actions with minimal human intervention. These agents can plan, reason, use tools, and adapt their approach based on feedback.
- RAG (Retrieval-Augmented Generation)
A technique that enhances LLM responses by retrieving relevant information from external knowledge bases before generation. RAG reduces hallucinations and enables access to current or proprietary information.
- Tool Use
The capability of AI models to invoke external tools, APIs, and systems to accomplish tasks. Modern implementations rely on structured outputs (JSON schemas or function signatures) so the model's tool calls can be parsed and executed reliably. Extends AI beyond text generation into real-world actions.
- LLM Gateway
Middleware sitting between applications and LLM providers, offering unified authentication, cost tracking, rate limiting, retries, and observability. Examples: Portkey, Kong AI, LiteLLM. Reduces per-app integration burden and centralizes policy enforcement.
Related on this site
Where this fits
Compound AI System is part of the AI Fundamentals vocabulary used in the Generative AI Maturity Framework. See the full glossary for the complete set of 159 defined terms, or take the free maturity assessment to see where your organisation stands.