Synthetic Data
Data generated by AI models used to train or fine-tune other models. Enables scaling in low-resource domains but risks model collapse if used exclusively across generations. Best practice: mix with fresh real data at defined ratios and audit for distribution drift.
Related terms
- Distillation
Training a smaller "student" model to reproduce the outputs (or intermediate representations) of a larger "teacher" model. Common technique for compressing frontier-model capability into smaller, cheaper models suitable for high-volume or on-device use.
- Model Collapse
The degradation of model quality that results from training successive generations on synthetic data produced by earlier models rather than fresh human-generated data. A structural concern for the LLM ecosystem as AI-generated content saturates the open web.
- Fine-tuning
Continuing to train a pretrained model on a task-specific dataset to change its behavior, style, or domain knowledge. Techniques range from full fine-tuning (all weights) to parameter-efficient methods like LoRA and QLoRA that update only small adapter layers.
Related on this site
Where this fits
Synthetic Data is part of the AI Fundamentals vocabulary used in the Generative AI Maturity Framework. See the full glossary for the complete set of 149 defined terms, or take the free maturity assessment to see where your organisation stands.