llama.cpp
A C/C++ open-source runtime for running LLM inference on commodity hardware (CPU + Metal / CUDA). Widely used for local Llama, Qwen, DeepSeek, and other open-weights model serving without dedicated inference infrastructure.
Related terms
- GGUF
GPT-Generated Unified Format — a quantized-model file format used by llama.cpp and its ecosystem. Standard for shipping quantized open-weights models for local / on-device inference. Successor to GGML.
- Quantization
Reducing the numeric precision of model weights (from FP16/BF16 to INT8, INT4, or lower) to shrink memory and speed inference, usually with a small quality loss. Common formats: GGUF, AWQ, GPTQ. Enables running large open-weights models on consumer hardware.
Related on this site
Where this fits
llama.cpp is part of the Data & Infrastructure vocabulary used in the Generative AI Maturity Framework. See the full glossary for the complete set of 149 defined terms, or take the free maturity assessment to see where your organisation stands.