A realtime voice AI model is a speech-to-speech model that listens to streamed audio and speaks back over a persistent connection, so a person can hold a spoken conversation with an AI system without a separate transcription, text model and text-to-speech chain. Five providers currently document a general-purpose realtime speech-to-speech API: Alibaba Cloud (Qwen Omni realtime), Amazon (Nova Sonic), Google (Gemini Live API), OpenAI (Realtime API) and xAI (Grok Voice Agent API). This page sets out the same fields for each: model IDs, connection methods, audio formats, languages, voices, session limits, latency statements, tools and published prices. It is written for developers, product owners and architects choosing a voice stack. Providers are listed alphabetically; order is never a ranking, and the page makes no recommendation.
Last checked: 2026-11-15.
This is an independent reference page from NHI Mgmt Group based on public provider documentation; it is not sponsored or endorsed by any provider.
Key takeaways
- Five providers document a realtime speech-to-speech API. Alibaba Cloud, Amazon, Google, OpenAI and xAI each offer a bidirectional streaming interface with built-in turn-taking and interruption handling.
- Connections are WebSocket or WebRTC, and sessions are capped. Documented caps include 8 minutes per connection (Amazon Nova 2 Sonic), about 10 minutes per connection with 15-minute audio sessions (Google) and 60 minutes per session (OpenAI); each provider documents a renewal or resumption pattern.
- Latency is rarely given as a number. No provider publishes a millisecond value; xAI states "sub-second" latency and Google describes its 2.5 native audio model as offering sub-second native audio streaming, while the others say only "low latency", so measure in your own network path.
- Pricing units differ. OpenAI and Google price audio per million tokens (Google also gives per-minute equivalents for one model), xAI prices per minute, and Alibaba Cloud and Amazon point to their console or pricing pages.
- Client-side keys need ephemeral tokens. Google and xAI both tell developers to use short-lived ephemeral tokens rather than standard API keys in browser or mobile clients.
What is a realtime voice model?
A classic voice assistant chains three systems: automatic speech recognition (ASR) turns speech into text, a language model writes a text reply, and text-to-speech (TTS) reads it out. Each hop adds delay and drops information such as tone of voice. A realtime voice model accepts audio directly and can produce audio directly, streamed in small chunks over a connection that stays open for the whole conversation.
The providers on this page share four building blocks:
- Persistent bidirectional stream: audio flows from the user while the model is still responding. Amazon describes structured JSON events that control session lifecycle, audio streaming, text responses and tool interactions; OpenAI, xAI and Alibaba Cloud use a similar event model (for example a
session.updateevent). - Turn detection: voice activity detection (VAD) decides when the user has finished speaking. OpenAI's examples use
semantic_vad; Alibaba Cloud documentssemantic_vad,server_vadand manual commit; xAI's examples useserver_vad; Amazon describes intelligent turn-taking. - Barge-in: the user can interrupt and the model stops and listens. Amazon and Google both list interruption handling as a core capability.
- Tool use: the model can call functions mid-conversation, for example to look up an order, then keep talking.
For how these models fit into wider multimodal systems, see the Multimodal Models Explained.
Which realtime voice models are available?
As of 2026-11-15. Model IDs are as shown in each provider's documentation.
| Provider | Product | Model IDs documented | Status as shown | Source |
|---|---|---|---|---|
| Alibaba Cloud | Qwen Omni realtime (Model Studio) | qwen3.8-omni-flash-realtime (current); qwen3.5-omni-plus-realtime (previous) | Not stated by Alibaba Cloud on the realtime page | Model Studio realtime documentation |
| Amazon | Amazon Nova Sonic (Version 1) and Nova 2 Sonic (Amazon Bedrock) | Not stated on the Nova speech guides | Both documented; Nova 2 Sonic has its own guide | Amazon Nova user guides |
| Gemini Live API | gemini-3.8-live (default Live API model), gemini-3.8-live-extended-thinking, gemini-3.1-flash-live-preview (legacy preview), gemini-2.5-flash-native-audio-preview-12-2025 | 3.8 Live marked New; 3.1 Flash Live marked Preview | Gemini models page | |
| OpenAI | Realtime API | gpt-realtime-2.1, gpt-realtime-2.1-mini | Listed on models and pricing pages | OpenAI API docs |
| xAI | Grok Voice Agent API | grok-voice-latest | Listed in the voice guide | xAI voice documentation |
Google also lists gemini-3.5-live-translate-preview for speech-to-speech translation. Several providers publish separate streaming transcription or TTS models (for example qwen3-asr-flash-realtime and gemini-3.5-transcribe-live); those are speech-to-text or text-to-speech only and are out of scope for the comparison tables.
How do connections, audio formats and session limits compare?
As of 2026-11-15. "Not stated" means the provider documentation checked does not give the value.
| Provider | Transport | Input audio | Output audio | Session or connection limit | Continuation |
|---|---|---|---|---|---|
| Alibaba Cloud | WebSocket and WebRTC, regional endpoints (Singapore, China Beijing) | PCM 16 kHz, 16-bit little-endian, mono (1, 2 or 4 channels on WebSocket) | PCM 24 kHz | Not stated by Alibaba Cloud (concurrency in rate-limit docs) | Not stated by Alibaba Cloud |
| Amazon | Bidirectional streaming API on Amazon Bedrock | Not stated on the Nova speech guides | Not stated on the Nova speech guides | 8 minutes per connection (Nova 2 Sonic) | Connection renewal and session continuation pattern in code samples |
| Stateful WebSocket (WSS) | Raw 16-bit PCM, 16 kHz, little-endian | Raw 16-bit PCM, 24 kHz, little-endian | About 10 minutes per connection; 15 minutes audio-only and 2 minutes audio plus video without compression | Session resumption tokens valid for 2 hours; context window compression can extend sessions | |
| OpenAI | WebRTC (browser), WebSocket (server); SIP for telephony | PCM at 24 kHz mono (audio/pcm); audio/pcmu in examples | PCM 24 kHz; audio/pcmu in examples | 60 minutes per session | New session required after the limit |
| xAI | WebSocket (wss://api.x.ai/v1/realtime) | Not stated by xAI in the voice guide | Base64 audio in response.output_audio.delta events | Not stated by xAI | Not stated by xAI |
Session caps matter for long calls such as contact-centre conversations. The documented patterns differ: Google offers resumption handles and server-side compression of older turns, Amazon documents reconnecting and carrying state into a new connection, and OpenAI states a fixed session maximum. Whichever provider you use, design the application to carry conversation state across reconnects. The Context Windows Explained covers why long sessions eventually need trimming or compaction.
Which languages and voices does each model support?
As of 2026-11-15.
| Provider | Languages (as documented) | Voices | Custom voice |
|---|---|---|---|
| Alibaba Cloud | Speech recognition in 113 languages and dialects; speech generation in 36 languages and dialects | Default Tina; others such as Ethan; full list with previews on the voice list page | Voice cloning documented |
| Amazon | Nova 2 Sonic: English (US, UK, India, Australia), French, Italian, German, Spanish, Portuguese, Hindi; automatic language detection and switching | Masculine-sounding and feminine-sounding expressive voices; polyglot voices that keep the same voice across languages | Not stated on the Nova speech guides |
| Conversation in 70 languages; Live Translation in 70+ languages | Not stated on the Live API overview | Not stated on the Live API overview | |
| OpenAI | Not stated on the realtime conversation guide | 10: alloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, cedar (marin and cedar suggested for best quality) | Not stated on the realtime conversation guide |
| xAI | Not stated for speech-to-speech (the 38+ figure in xAI docs applies to speech-to-text) | 28 built-in voices, default eve; all voices can speak every supported language | Not stated in the voice guide |
Language support claims are provider statements, not measured accuracy. Test with recordings in the accents, dialects and background noise your users actually have.
What latency do realtime voice models offer?
Latency is the field most often asked about and least often published. As of 2026-11-15:
- Alibaba Cloud: no millisecond figure stated on the realtime page.
- Amazon: describes Nova Sonic as low-latency, multi-turn streaming; no millisecond figure stated.
- Google: describes
gemini-2.5-flash-native-audio-preview-12-2025as offering sub-second native audio streaming, andgemini-3.8-liveas the default model "for most low-latency voice agent experiences"; no millisecond figure stated. - OpenAI: no millisecond figure stated on the realtime guides checked.
- xAI: states "sub-second" latency for the Voice Agent API.
Perceived latency depends on more than the model: network distance to the provider region, transport (WebRTC is designed for browser media, WebSocket for server-to-server), VAD settings such as silence duration (Alibaba Cloud's example uses silence_duration_ms 900), tool calls that pause speech, and reasoning effort where the model supports it. Measure time from end of user speech to first audio byte in your own deployment, at the 50th and 95th percentile.
How much do realtime voice models cost?
Prices in USD, as of 2026-11-15, from each provider's official pricing page or documentation.
| Provider and model | Audio input | Audio output | Text input / output | Unit |
|---|---|---|---|---|
Alibaba Cloud qwen3.8-omni-flash-realtime | Not stated on the realtime page; Alibaba Cloud refers to the Model Studio console | Not stated | ||
| Amazon Nova Sonic / Nova 2 Sonic | See the Amazon Nova pricing page; Amazon notes that text token rates apply to uses such as speech-to-text transcription, tool calls and adding conversation history | Per token | ||
Google gemini-3.1-flash-live-preview | $3.00 per 1M tokens or $0.005 per minute | $12.00 per 1M tokens or $0.018 per minute | $0.75 / $4.50 per 1M tokens | Per token or per minute (paid tier) |
Google gemini-2.5-flash-native-audio-preview-12-2025 | $3.00 per 1M tokens | $12.00 per 1M tokens | $0.50 / $2.00 per 1M tokens | Per token (paid tier) |
OpenAI gpt-realtime-2.1 | $32.00 per 1M tokens ($0.40 cached) | $64.00 per 1M tokens | $4.00 / $24.00 per 1M tokens | Per token |
OpenAI gpt-realtime-2.1-mini | $10.00 per 1M tokens ($0.30 cached) | $20.00 per 1M tokens | $0.60 / $2.40 per 1M tokens | Per token |
xAI grok-voice-latest | $0.08 per minute (realtime) | Per minute | ||
Google's pricing page did not list a price for gemini-3.8-live at the time of checking. OpenAI also prices image input for the realtime models ($5.00 per 1M tokens for gpt-realtime-2.1, $0.80 for the mini). To compare per-token and per-minute prices you need your own audio token rate per minute and your ratio of listening to speaking time. The Model Pricing Explained explains token billing and caching.
What tools and features does each API document?
As of 2026-11-15. Only features named in the documentation checked are listed.
| Provider | Function calling | Other tools | Image or video input | Other documented features |
|---|---|---|---|---|
| Alibaba Cloud | Yes | Remote MCP tools, web search | JPG/JPEG frames, about one per second, up to 1080p | Semantic VAD that filters backchannel speech and noise; voice cloning |
| Amazon | Yes, with agentic workflows | RAG grounding on enterprise data | Not stated (Nova 2 Sonic mixes audio and text input) | Asynchronous tool handling (keeps speaking while tools run); prosody-adaptive responses; noise robustness |
| Yes | Google Search | JPEG images, at most one frame per second | Proactive audio, affective dialog, transcription of input and output, Live Translation | |
| OpenAI | Yes | MCP servers and connectors (Tools and MCP guide) | Image input on gpt-realtime-2.1; video not supported | 128,000-token context window and 32,000 max output for gpt-realtime-2.1; configurable reasoning effort |
| xAI | Yes | Web search (web_search tool in examples) | Not stated in the voice guide | Server VAD turn detection; 28 voices |
How to choose a realtime voice model: factual criteria
This is a checklist of questions, not a ranking. Each answer comes from the tables above and your own testing.
- Languages and accents: confirm every language you need is on the provider's documented list, then test recognition and speech quality with real samples.
- Call length: compare your typical and longest call with the documented session or connection cap and the continuation method.
- Transport and telephony: browser apps favour WebRTC (documented by Alibaba Cloud and OpenAI); phone systems need SIP or a telephony bridge (OpenAI documents SIP).
- Region and data location: Alibaba Cloud documents Singapore and Beijing endpoints with separate keys per region; Amazon runs through Amazon Bedrock regions; check each provider's region list against your data residency needs.
- Tools: list the systems the agent must call and confirm the provider supports the tool type (functions, MCP, search, RAG).
- Cost model: model a typical call in the provider's unit (tokens or minutes), including tool calls and conversation history.
- Measured latency: run a pilot from your users' region and record end-of-speech to first-audio times.
Where security fits
Realtime voice puts an AI agent on a live audio channel with tools, so the usual agent controls apply plus a few voice-specific ones:
- Do not ship API keys to clients: Google recommends ephemeral tokens for client-to-server Live API connections, and xAI says to use ephemeral tokens instead of exposing the key in client-side apps. Mint them server-side, scoped and short-lived. See the Dynamic Secrets and Ephemeral Credentials Guide.
- Treat tools as privileged: a voice agent that can refund, transfer or change account details needs the same authorisation checks as any other client, applied server-side.
- Voice is not identity: callers can use cloned voices. See the Voice AI Agent Security and Fraud Guide and the Deepfake and AI Impersonation Guide.
- Recordings and transcripts are personal data: decide retention, consent notices and access before launch.
Practitioner checklist
- Shortlist providers by documented language support and region, then test with your own audio samples.
- Record each provider's session or connection limit and build reconnect and state carry-over into the client from day one.
- Match audio formats end to end (for example 16 kHz input and 24 kHz output PCM) to avoid resampling delay and quality loss.
- Measure end-of-speech to first-audio latency at p50 and p95 from your users' regions before committing.
- Model cost per typical call in the provider's own unit, including tool calls, history and image input.
- Use server-minted ephemeral tokens for browser and mobile clients; never embed long-lived API keys.
- Enforce authorisation for every tool the voice agent can call, on the server, not in the prompt.
- Define retention and consent for call audio and transcripts, and re-check provider docs before launch because model IDs and prices change.
Standards and references
- Alibaba Cloud: Model Studio realtime (Qwen Omni realtime) (accessed 2026-11-15)
- Amazon: Nova 2 Sonic speech-to-speech and Amazon Nova Sonic user guide (accessed 2026-11-15)
- Amazon: Nova pricing (accessed 2026-11-15)
- Google: Gemini Live API overview and Live API session management (accessed 2026-11-15)
- Google: Gemini models and Gemini API pricing (accessed 2026-11-15)
- OpenAI: Realtime API and Realtime conversations (accessed 2026-11-15)
- OpenAI: gpt-realtime-2.1 model page and API pricing (accessed 2026-11-15)
- xAI: Voice (speech to speech and Voice Agent API) (accessed 2026-11-15)
Related NHI Mgmt Group resources: Multimodal Models Explained · Voice AI Agent Security and Fraud Guide · Model Pricing Explained · AI Agent Memory Explained