GGUF
GPT-Generated Unified Format — a quantized-model file format used by llama.cpp and its ecosystem. Standard for shipping quantized open-weights models for local / on-device inference. Successor to GGML.
Related terms
- Quantization
Reducing the numeric precision of model weights (from FP16/BF16 to INT8, INT4, or lower) to shrink memory and speed inference, usually with a small quality loss. Common formats: GGUF, AWQ, GPTQ. Enables running large open-weights models on consumer hardware.
- Small Language Model (SLM)
A compact language model (typically ≤ 10B parameters) optimized for on-device inference, low-latency use cases, or cost-sensitive high-volume workloads. Examples: Phi-4, Gemma 3, Llama 3.2 1B/3B. Increasingly capable — some SLMs match 2024-era mid-tier LLMs on narrow tasks.
Related on this site
Where this fits
GGUF is part of the Data & Infrastructure vocabulary used in the Generative AI Maturity Framework. See the full glossary for the complete set of 149 defined terms, or take the free maturity assessment to see where your organisation stands.