Data & Infrastructure

Response Caching

Client-side or gateway-side caching of complete LLM responses by prompt hash. Different from prompt caching (server-side KV cache reuse). Useful for high-repetition workloads where identical prompts appear.

Related terms

Prompt Caching
A provider-side optimization that reuses the KV cache for repeated prompt prefixes across requests, typically reducing input token cost by ~10× for reused content. Anthropic, OpenAI, and Google offer it. Order-of-magnitude cost driver for RAG and agent workloads.

Next steps

Take the assessment— See where you stand

Where this fits

Response Caching is part of the Data & Infrastructure vocabulary used in the Generative AI Maturity Framework. See the full glossary for the complete set of 149 defined terms, or take the free maturity assessment to see where your organisation stands.

Related terms

Related on this site

Framework dimensions

Free tools

Whitepapers

Next steps

Where this fits