AI Techniques

Realtime API

A bidirectional streaming API — WebSocket or WebRTC — that lets a client stream audio, video, and text to an LLM and receive a streamed multi-modal response with sub-second latency. OpenAI Realtime, Gemini Live, and Anthropic's streaming voice endpoints are the 2026 exemplars. Enables voice agents that feel conversational instead of turn-based.

Last reviewed: 2026-09-02

Related terms

Speech-to-Text (STT)
Automatic transcription of audio into text. Modern systems (Whisper, Deepgram, AssemblyAI, ElevenLabs) run near-real-time with speaker diarization and cross-language support. STT is the front of most voice-AI pipelines feeding an LLM downstream.
Text-to-Speech (TTS)
Synthesis of natural-sounding speech from text. Modern systems (ElevenLabs, OpenAI TTS, Google Gemini TTS) support voice cloning, emotion, and streaming. Combined with STT + LLM, TTS closes the loop for voice-AI applications.
Multi-Agent Systems
Systems where multiple AI agents work together, each with specialized capabilities, to accomplish complex tasks. Agents may cooperate, negotiate, or compete depending on the system design.
Tool Calling
The general pattern where an LLM emits structured output (typically JSON matching a schema) that a runtime interprets as a call to an external function, API, or service. Also called "function calling". MCP standardizes discovery and invocation across providers.

Framework dimensions

Free tools

Whitepapers

LLM Cost Optimization Patterns

Comparisons

RAG vs Fine-tuning vs Prompt Engineering: Decision Framework

Next steps

Take the assessment— See where you stand

Where this fits

Realtime API is part of the AI Techniques vocabulary used in the Generative AI Maturity Framework. See the full glossary for the complete set of 159 defined terms, or take the free maturity assessment to see where your organisation stands.