Quantization
Reducing the numeric precision of model weights (from FP16/BF16 to INT8, INT4, or lower) to shrink memory and speed inference, usually with a small quality loss. Common formats: GGUF, AWQ, GPTQ. Enables running large open-weights models on consumer hardware.
Related terms
- LoRA
Low-Rank Adaptation — a parameter-efficient fine-tuning technique that freezes the base model and trains small rank-decomposition matrices instead. Reduces fine-tuning cost by 10-100× and enables swapping many task-specific adapters over a single base model.
- Fine-tuning
Continuing to train a pretrained model on a task-specific dataset to change its behavior, style, or domain knowledge. Techniques range from full fine-tuning (all weights) to parameter-efficient methods like LoRA and QLoRA that update only small adapter layers.
Related on this site
Where this fits
Quantization is part of the AI Fundamentals vocabulary used in the Generative AI Maturity Framework. See the full glossary for the complete set of 149 defined terms, or take the free maturity assessment to see where your organisation stands.