AI Techniques

Semantic Router

A pre-LLM classification layer that inspects an incoming request and routes it to the right downstream path — a specific prompt, model tier, retrieval pipeline, or tool. Often built on cheap embedding similarity. Reduces cost and improves quality by matching requests to the right capability instead of sending everything to the most capable model.

Last reviewed: 2026-09-02

Related terms

LLM Gateway
Middleware sitting between applications and LLM providers, offering unified authentication, cost tracking, rate limiting, retries, and observability. Examples: Portkey, Kong AI, LiteLLM. Reduces per-app integration burden and centralizes policy enforcement.
Embedding
A numeric vector representation of text (or images, audio, etc.) produced by a model such that semantically similar inputs map to nearby vectors. Used to power search, clustering, and retrieval in modern GenAI stacks.
Small Language Model (SLM)
A compact language model (typically ≤ 10B parameters) optimized for on-device inference, low-latency use cases, or cost-sensitive high-volume workloads. Examples: Phi-4, Gemma 3, Llama 3.2 1B/3B. Increasingly capable — some SLMs match 2024-era mid-tier LLMs on narrow tasks.
Prompt Engineering
The practice of designing and optimizing input prompts to get desired outputs from AI models. Effective prompts can significantly improve model performance without additional training.

Framework dimensions

Free tools

Whitepapers

LLM Cost Optimization Patterns

Comparisons

RAG vs Fine-tuning vs Prompt Engineering: Decision Framework

Next steps

Take the assessment— See where you stand

Where this fits

Semantic Router is part of the AI Techniques vocabulary used in the Generative AI Maturity Framework. See the full glossary for the complete set of 159 defined terms, or take the free maturity assessment to see where your organisation stands.