AI Fundamentals

Speculative Decoding

An inference-time optimization that uses a fast draft model to propose tokens which the target model then verifies in parallel, reducing latency without changing output quality. Ships in vLLM, TensorRT-LLM, and most frontier provider stacks.

Related terms

Small Language Model (SLM)
A compact language model (typically ≤ 10B parameters) optimized for on-device inference, low-latency use cases, or cost-sensitive high-volume workloads. Examples: Phi-4, Gemma 3, Llama 3.2 1B/3B. Increasingly capable — some SLMs match 2024-era mid-tier LLMs on narrow tasks.

Framework dimensions

Free tools

LLM Model Comparison

Whitepapers

LLM Cost Optimization Patterns

Next steps

Take the assessment— See where you stand

Where this fits

Speculative Decoding is part of the AI Fundamentals vocabulary used in the Generative AI Maturity Framework. See the full glossary for the complete set of 149 defined terms, or take the free maturity assessment to see where your organisation stands.