AI Techniques

Test-Time Compute

The strategy of spending more inference compute — through longer reasoning, self-consistency sampling, tree search, or verifier passes — to raise output quality on a fixed model. Alternative to training-time scaling. Reasoning models (o-series, Claude extended thinking, Gemini Deep Think) are the canonical example.

Last reviewed: 2026-09-02

Related terms

Reasoning Model
A class of LLMs (OpenAI o-series, Claude with extended thinking, DeepSeek-R1) trained to spend significant inference-time compute on internal reasoning before producing the final answer. Substantially better on math, coding, and multi-step problems at the cost of higher latency and token consumption.
Chain-of-Thought (CoT)
A prompting or training pattern where the model produces intermediate reasoning steps before its final answer, typically improving accuracy on multi-step tasks. Modern reasoning models (o1, o3, Claude with extended thinking) internalize CoT during training.
LLM-as-Judge
An evaluation pattern where one LLM (typically a strong model) grades outputs from another. Cheaper and faster than human eval, but subject to positional bias, sycophancy toward the model being evaluated, and mode collapse — mitigate with rubric prompts and reference answers.
Speculative Decoding
An inference-time optimization that uses a fast draft model to propose tokens which the target model then verifies in parallel, reducing latency without changing output quality. Ships in vLLM, TensorRT-LLM, and most frontier provider stacks.

Framework dimensions

Free tools

Whitepapers

LLM Cost Optimization Patterns

Comparisons

RAG vs Fine-tuning vs Prompt Engineering: Decision Framework

Next steps

Take the assessment— See where you stand

Where this fits

Test-Time Compute is part of the AI Techniques vocabulary used in the Generative AI Maturity Framework. See the full glossary for the complete set of 159 defined terms, or take the free maturity assessment to see where your organisation stands.