Join our Newsletter — 33% off our NHI Course
Home› Guides› LLM Speed and Latency Compared: Speed Tiers, Documented…
Guide AI Platforms

LLM Speed and Latency Compared: Speed Tiers, Documented Claims and How to Measure

PDF for newsletter subscribers · free to read online

← All guides
By NHI Mgmt Group, edited by Lalit Choda Updated 15 November 2026 15 min read
On this page

Output tokens per second and time to first token decide how responsive an LLM application feels, yet most providers publish speed as a paid service tier and a multiplier rather than as a fixed number per model. This page compares what six API providers document about speed and latency: which faster or prioritised tiers exist, what each tier claims, what it costs, how it is switched on, how it falls back under load and how to confirm which tier served a request. It also explains the two core metrics and how to measure them on your own workload, because a provider's multiplier only tells you the change relative to its own standard tier. It is written for developers, platform engineers and buyers sizing latency-sensitive applications such as chat interfaces, voice agents and coding assistants. Providers are listed alphabetically (Anthropic, DeepSeek, Google, Mistral AI, OpenAI, xAI) and order is never a ranking.

Last checked: 2026-11-15.

This is an independent reference page from NHI Mgmt Group based on public provider documentation; it is not sponsored or endorsed by any provider. Speed tiers, prices and supported models change often, so confirm details on each provider's current pages, linked in the sources list.

Key takeaways

  • Few providers publish a tokens-per-second figure. Anthropic and OpenAI state "up to 2.5x" faster output for their fast modes relative to standard; Google, Mistral AI and xAI describe priority tiers in terms of queueing and latency class; DeepSeek's pricing page makes no speed statement.
  • Two different mechanisms are sold. A faster inference configuration (Anthropic fast mode, OpenAI Fast and Ultrafast modes) raises generation speed, while priority queueing (Google Priority, Mistral AI Priority Tier, xAI Priority Processing, Anthropic's legacy Priority Tier) serves requests ahead of standard traffic, which matters most under load.
  • Speed costs a premium. Documented premiums range from 1.75x (Mistral AI Priority Tier) and 75 to 100% above standard (Google Priority) to 2x the standard rate (Anthropic fast mode on Opus 5.5, OpenAI Fast mode on GPT-5.6 Sol).
  • Every tier can fall back. Google, Mistral AI, OpenAI and xAI downgrade to standard processing when priority or fast capacity is unavailable, and Anthropic fast mode returns a 429 that callers can retry at standard speed; each provider reports the tier actually used in the response.
  • Time to first token and output speed are separate. Anthropic states its fast mode improves output tokens per second, not time to first token; xAI states priority typically lowers both; reasoning effort and prompt length often dominate either figure.

What do speed and latency mean for an LLM API?

LLM latency has two parts that respond to different levers. Provider documentation uses the following terms.

MetricWhat it measuresWhat usually drives it
Time to first token (TTFT)Seconds from sending the request to receiving the first streamed tokenQueueing, prompt length (prefill), prompt cache hits, and for reasoning models the thinking that happens before visible output
Output tokens per second (OTPS)Generation rate once output has started; Anthropic's fast mode page uses this termModel size, inference configuration, server load
Inter-token latency (ITL)Gap between successive streamed tokens; xAI's priority page uses this termThe inverse view of OTPS; spikes indicate contention
End-to-end timeTotal time to the final tokenTTFT plus output length divided by OTPS, plus any tool call round trips

For a chat interface TTFT shapes perceived responsiveness; for long code generation OTPS dominates; for agents that call tools many times, per-call overhead and transport also matter, which is why OpenAI recommends WebSockets for its Ultrafast mode in agentic apps with many quick tool calls. Streaming does not change total time but shows output earlier; Anthropic notes the fast mode gain is most visible with streaming.

The six providers at a glance

As of 2026-11-15. Sources: each provider's service tier, fast mode or pricing documentation, linked in Standards and references.

ProviderSpeed or priority options documentedSpeed claim stated by providerRequest parameterStatus stated
AnthropicFast mode; Priority Tier (legacy commitments only)Fast mode: up to 2.5x higher output tokens per second than standard; no latency figure for Priority Tierspeed: "fast"; service_tier: "auto" or "standard_only"Fast mode available on Fable 5.2, Mythos 5.2 and Opus 5.5; Priority Tier marked legacy
DeepSeekOff-peak and peak pricing periodsNot stated by DeepSeek on the pricing page checkedNot stated by DeepSeekOff-peak rates half of peak rates
Google (Gemini API)Priority, Standard, Flex and Batch tiersPriority latency class "Seconds"; Standard "Seconds to minutes"; Flex 1 to 15 minute target; Batch up to 24 hoursservice_tier: "priority" (default standard)Priority labelled Preview
Mistral AIPriority Tier, Standard Tier, BatchPriority latency "seconds" versus "seconds to minutes" for Standard; Batch over 24 hoursservice_tier: "auto" or "standard_only"Requires a Priority Tier entitlement set up by an account executive
OpenAIFast mode (formerly priority processing), Ultrafast mode, Scale Tier, Flex, BatchFast mode: up to 2.5x faster than Standard; Ultrafast: described as the fastest tier, no figure givenservice_tier: "fast" or "priority"; "ultrafast"Ultrafast broadly available for GPT-6 Astra and GPT-6.1 Sol
xAIPriority Processing, default tier, Batch APITypically lower time to first token and faster inter-token latency, especially under high demand; no figure givenservice_tier: "priority" (or "default")Not labelled beta or deprecated; page updated 2026-06-15

What each provider documents about speed

Anthropic

Fast mode runs the same model with a faster inference configuration and, per Anthropic, does not change intelligence or capabilities. It gives up to 2.5x higher output tokens per second and does not improve time to first token. It is set per request with speed: "fast" on the Messages API and is available on Claude Fable 5.2, Claude Mythos 5.2 and Claude Opus 5.5; Claude Opus 5 is in research preview behind the fast-mode-2026-02-01 header, and fast mode on Claude Opus 4.8 is deprecated with a removal date of 30 November 2026. On Microsoft Foundry, fast mode uses separate model IDs such as claude-opus-5-5-fast. The response field usage.speed reports fast or standard. Fast mode is not available with the Batch API or with a Priority Tier commitment, and switching speeds causes a prompt cache miss. Anthropic's separate Priority Tier, which targets 99.5% uptime and is selected with service_tier: "auto", is marked legacy: commitments are no longer sold and existing ones run to contract end. See Anthropic Models.

DeepSeek

DeepSeek's pricing page lists deepseek-flash (DeepSeek-V4.1-Flash-1017) and deepseek-v4-pro (DeepSeek-V4-Pro-0813), both with a 1M context length and up to 384K output tokens, and both defaulting to thinking mode with a non-thinking option. The page makes no statement about speed, latency or a priority tier. It sets off-peak rates at half the peak rates, with peak hours of 01:00 to 04:00 and 06:00 to 10:00 UTC, Monday to Friday, excluding Chinese public holidays, and lists concurrency limits of 2,500 for Flash and 500 for Pro, pointing to a separate Rate Limit and Isolation page. Thinking mode on or off is the main documented lever that changes how long a response takes. See DeepSeek Models.

Google

The Gemini API documents four inference tiers. Priority, labelled Preview, is ranked ahead of Standard and Flex traffic, is non-sheddable under load, and is designed for second-level response times; it is set with service_tier: "priority". Standard is the default with latency of seconds to minutes; Flex is best-effort with a 1 to 15 minute target at a 50% discount; Batch is asynchronous with up to 24 hours at a 50% discount. Priority requests over the Priority limit during congestion are downgraded to Standard rather than failing, and the x-gemini-service-tier response header shows the tier used. Supported models listed include Gemini 4 Argon, Gemini 3.8 Flash, Gemini 3.6 Flash, Gemini 3.5 Flash-Lite and Gemini 2.5 Pro, among others. See Google Gemini Models.

Mistral AI

Priority Tier gives eligible requests priority queueing ahead of Standard Tier traffic during high load and is aimed at real-time and business-critical workloads. It needs an active entitlement and custom model-specific rate limits configured through a Mistral AI account executive. Mistral AI lists Priority latency as seconds against seconds to minutes for Standard, a 99.5% uptime SLA for Priority, and no stated SLA for Standard or Batch. Requests use service_tier: "auto", which uses Priority when eligible and falls back to Standard otherwise; usage.service_tier reports priority or standard. Availability can be global or regional depending on account setup. See Mistral AI Models.

OpenAI

Priority processing was renamed Fast mode on 30 July 2026. OpenAI states Fast mode delivers up to 2.5x faster speeds and more consistent latency with pay-as-you-go pricing, with the 2.5x increase specified for gpt-5.6-sol. It is set per request with service_tier: "fast" or "priority", or per project in Project Service Tier settings. OpenAI states Fast mode for GPT-6 Astra has no latency SLA. Ultrafast mode, set with service_tier: "ultrafast", is described as the fastest service tier in the API and is available for GPT-6 Astra and GPT-6.1 Sol; the guide gives no speed figure and recommends WebSockets for agentic apps. Scale Tier is a separate, separately billed option. See OpenAI Models.

xAI

Priority Processing gives a request higher scheduling priority, which xAI says typically results in lower time to first token and faster inter-token latency, especially during periods of high demand. It is set with service_tier: "priority" on Chat Completions and Responses text inference and needs no capacity reservation or advance provisioning. Priority is granted only when priority capacity is available; otherwise the request is served at the default tier, and every response includes a service_tier field. Customers are billed the priority rate only when the response confirms priority. xAI suggests the Batch API for background jobs, evaluations and bulk processing. The models page lists grok-4.7 with a 500K context and configurable reasoning. See xAI Models.

How much do faster tiers cost?

As of 2026-11-15, USD, from each provider's own pages. Standard per-token prices are on the LLM Pricing Tracker.

ProviderTierPrice or premium statedNotes
AnthropicFast modeOpus 5.5: $8 input and $40 output per 1M tokens; Fable 5.2 and Mythos 5.2: $16 and $80; Opus 5 and Opus 4.8: $10 and $50Multiplier applies across the full context window; prompt caching and data residency multipliers apply on top
DeepSeekOff-peak periodOff-peak rates are half of peak rates (for example Flash output $0.6 off-peak, $1.2 peak per 1M)A time-of-day price, not a speed tier
GooglePriority75 to 100% more than Standard, per tokenDowngraded requests bill at the Standard rate; Flex and Batch are 50% below Standard
Mistral AIPriority Tier1.75x Standard Tier list pricing on input, output and cached tokensMultiplier applied after any prompt caching discount
OpenAIFast mode; UltrafastFast: GPT-5.6 Sol at twice Standard, $8 input and $40 output per 1M (short context); Ultrafast: priced in a separate pricing tableDowngraded Fast requests bill at standard rates; GPT-5.6 Sol promotional pricing at least through 21 November 2026
xAIPriority ProcessingPremium per-token rate published on the pricing pageCache discount applies before the priority multiplier; priority rate billed only when confirmed

Rate limits and fallback behaviour

A faster tier is only useful if capacity exists when traffic arrives. Each provider documents a different relationship with standard limits, which matters for capacity planning and for the LLM API Rate Limits and Usage Tiers Compared page.

ProviderLimit for the faster tierWhen capacity runs outHow to confirm the tier used
AnthropicFast mode: dedicated limit, default in the Start, Build and Scale usage tiers; anthropic-fast-* headers429 with retry-after; SDK retries twice by default; caller may retry without speedusage.speed; usage.service_tier for Priority Tier
DeepSeekConcurrency 2,500 (Flash) and 500 (Pro)Detailed on the Rate Limit and Isolation pageNot stated by DeepSeek on the pricing page checked
GooglePriority: own limit, default 0.3x the standard limit, also counted toward interactive limitsDowngraded to Standard instead of 503 or 429x-gemini-service-tier header
Mistral AICustom, model-specific limits agreed with Mistral AIFalls back to Standard Tierusage.service_tier
OpenAIFast and Standard share a limit; Ultrafast has separate TPM limits (for example GPT-6.1 Sol: 1M Build, 4M Launch, 40M Grow)Fast requests ramped too quickly may be downgraded to standard and show service_tier: "default"; rule of thumb above 1M input TPM is no more than 50% growth every 15 minutesservice_tier in the response
xAINot stated on the Priority Processing pageServed at the default tierservice_tier in the response

Benchmarks

None of the six providers publishes a per-model tokens-per-second or time-to-first-token table on the pages checked; the stated figures are the relative multipliers and latency classes above. Independent services such as Artificial Analysis publish measured latency and speed for many models, and their figures depend on prompt length, reasoning effort setting, the provider endpoint used and the date of measurement. Because those conditions differ from production traffic, and because a reasoning model's time to first answer token includes its thinking time, treat independent figures as a starting point and record the measurement date and settings alongside any number you quote. Reasoning effort trade-offs are covered on Reasoning LLMs Compared.

How to measure speed yourself

  1. Fix the workload: use representative prompts at the input lengths you expect, a fixed maximum output length and fixed reasoning or thinking settings, because each changes TTFT more than the tier does.
  2. Stream and timestamp: record request start, first content token and final token on the client; compute TTFT, OTPS (output tokens divided by generation time) and end-to-end time.
  3. Record the tier actually served: log usage.speed, usage.service_tier, service_tier or x-gemini-service-tier with every sample so downgraded requests are not counted as fast ones.
  4. Sample over time: run at peak and off-peak hours across several days; priority tiers are documented to matter most under high load, so a quiet-hour test can show no gain.
  5. Report percentiles: use p50, p95 and p99 rather than averages, and note prompt cache hit rates, because a cache miss after a speed switch (documented by Anthropic) inflates TTFT.
  6. Compare cost per useful second: multiply the tier premium by tokens used and set it against the latency gain at p95 for your use case.

How to choose a speed tier: factual criteria

These are criteria to test, not recommendations.

CriterionFacts to check
Which metric limits youLong outputs gain from higher OTPS (Anthropic and OpenAI fast modes); short interactive turns under load gain from priority queueing (Google, Mistral AI, xAI)
Model coverageWhether the model you use is on the tier's supported list; lists differ by tier and change often
Commercial modelPay-as-you-go per request (Anthropic fast mode, Google Priority, OpenAI Fast, xAI Priority) or an entitlement or commitment (Mistral AI Priority Tier, Anthropic legacy Priority Tier, OpenAI Scale Tier)
Fallback behaviourSilent downgrade to standard or an explicit 429; whether downgraded requests are billed at the standard rate
Interaction with other featuresBatch, prompt caching, data residency and zero data retention eligibility per tier
Latency-insensitive workFlex, Batch or off-peak pricing (DeepSeek) for jobs that can wait

A small team with a single chat feature can usually start with the standard tier, measure p95 TTFT and enable a faster tier only for the user-facing route. A larger organisation should route by workload: fast or priority for interactive paths, Flex or Batch for evaluations and back-office jobs, and alerts on downgrade rates. Agents that call tools in loops multiply per-call latency, so see AI Agent Rate Limiting and Quota Design and LLM Tool Use, Function Calling and Structured Output Support Compared.

Where identity fits

Faster tiers are billed per key, project or organisation, and OpenAI ramp limits are shared across all projects in an organisation. A leaked API key used on a premium tier costs more per token than one on standard, so scope keys per project, alert on unexpected tier usage and follow the API Key Management Guide. For provider-specific steps see How to audit OpenAI and Anthropic organisation API keys.

Practitioner checklist

  • Decide whether TTFT, OTPS or end-to-end time is the metric your product needs, and set a p95 target for it.
  • Benchmark your own prompts at standard speed first, with streaming, fixed reasoning settings and timestamps on the client.
  • Confirm your model is on the faster tier's supported list on the provider's current page before building on it.
  • Log the tier actually served on every response and alert when downgrade rates rise.
  • Handle fallback explicitly: retry at standard speed on fast mode 429s and ramp traffic gradually where the provider documents ramp limits.
  • Route latency-insensitive jobs to Flex, Batch or off-peak pricing instead of the default tier.
  • Restrict who can enable premium tiers at project level and watch spend by key.
  • Re-run the benchmark after model, tier or prompt changes, and record the date with every figure you publish internally.

Standards and references

Related NHI Mgmt Group resources: Frontier AI Models Compared, LLM Pricing Tracker, LLM API Rate Limits and Usage Tiers Compared, Small and Low-Cost LLMs Compared.

PDF for subscribers

Take this guide with you

Every section, table and checklist in one PDF, with links to the sources and related guides. Subscribe to our weekly NHI & AI Identity Journal and we’ll email you a download link.

Free weekly newsletter

Subscribe to the NHI & AI Identity Journal

The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.

Bonus 33% off our NHI Course when you subscribe.

NHIMG Editorial Note
Researched and written by the NHI Mgmt Group editorial team using AI tools, under the editorial direction of Lalit Choda. Facts are drawn from the sources listed on this page and checked by our automated quality process. Last updated 15 November 2026. Spotted an error? Use our contact page.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM and Agentic AI security. nhimg.org

PDF Downloads for Newsletter Subscribers

Get the guide

Enter your details and we’ll email you a download link.

The full guide is free to read online. The PDF is a thank you for subscribing.

Fill in all fields marked * and tick the PDF box to continue.