Prompt Injection
An attack where adversarial instructions embedded in untrusted content (user input, retrieved documents, tool responses) hijack the model into ignoring its original instructions. Considered the primary security risk for LLM applications; defenses include separation of channels and output validation.
Related terms
- Guardrails
Runtime controls that constrain what an LLM system can produce or do — input validators, output filters, topic classifiers, tool allowlists, cost budgets, and human-approval checkpoints. Distinct from model-level safety training; guardrails run at inference time.
- Red-Teaming
Structured adversarial testing of an AI system, either manually or via automated attack pipelines, to find prompt injection, jailbreak, harmful-output, and misuse vulnerabilities before release. Increasingly required by AI regulation for high-risk systems.
Related on this site
Where this fits
Prompt Injection is part of the Evaluation & Governance vocabulary used in the Generative AI Maturity Framework. See the full glossary for the complete set of 149 defined terms, or take the free maturity assessment to see where your organisation stands.