GRPO
Group Relative Policy Optimization — a reinforcement-learning alignment technique used in DeepSeek-R1 and other reasoning-model training. Compares groups of sampled outputs against each other to compute advantages, avoiding the value-model overhead of PPO.
Related terms
- RLHF
Reinforcement Learning from Human Feedback — a training method where human preferences over pairs of outputs are used to train a reward model, which in turn shapes the base LLM via RL (typically PPO or DPO). Central to aligning modern chat assistants.
- DPO
Direct Preference Optimization — an RLHF-like fine-tuning approach that skips training an explicit reward model, instead directly optimizing the policy against pairwise preferences. Simpler and often more stable than PPO-based RLHF; widely adopted in open-weights fine-tuning.
- Reasoning Model
A class of LLMs (OpenAI o-series, Claude with extended thinking, DeepSeek-R1) trained to spend significant inference-time compute on internal reasoning before producing the final answer. Substantially better on math, coding, and multi-step problems at the cost of higher latency and token consumption.
Related on this site
Where this fits
GRPO is part of the AI Fundamentals vocabulary used in the Generative AI Maturity Framework. See the full glossary for the complete set of 149 defined terms, or take the free maturity assessment to see where your organisation stands.