Quick Answer
An LLM inference provider hosts open-weight models like Llama, DeepSeek, and Qwen behind a pay-per-token API, handling GPUs, scaling, and serving for you. The same Llama 3.3 70B model ranges from $0.10 to $1.04 per million input tokens depending on who serves it, so provider choice is a pricing decision. Top picks as of September 2026: Groq and Cerebras for speed, DeepInfra for price, Together and Fireworks for breadth, Baseten for custom models.
Somewhere right now, two teams are running the exact same Llama 3.3 70B model in production. One pays about $0.12 per million tokens on a quantized FP8 tier. The other pays about $1.05 for full-precision serving, close to nine times more, per Saturn Cloud’s mid-2026 provider landscape report. Neither team chose a different model. They chose a different provider, and with it, different fine print. This guide reads that fine print for you.
That’s the entire thesis of this guide: with open models, the provider, not the model, drives most of the cost. What follows compares 16 LLM API providers on the axis that actually decides your invoice, cost per completed task, not just cost per token. And yes, those are different numbers, sometimes wildly so.
Which comparison do you actually need?
Four adjacent buying decisions get tangled constantly, so thirty seconds of routing before we spend your afternoon. Comparing what the frontier labs charge for their own models, GPT-5.x versus Claude versus Gemini? That’s our LLM API pricing comparison. Renting the GPUs and running models yourself? Cloud GPU pricing comparison. Need the traffic layer that routes and governs calls across all of it? That’s an LLM gateway.
This guide covers the middle path most production teams take with open models: serverless inference, where a provider hosts Llama, DeepSeek, Qwen, or Mistral and you pay per token. No capacity planning, no GPU procurement, and a market competitive enough to produce that 9x spread.
2026 State of AI Spend Report
Why Finance Caps What It Can’t See
Learn how finance leaders are managing AI spend, and how they’re approaching FY2027
The CloudZero read: inference is the AI bill that never stops growing
Before the rankings, the money context, because it explains why this decision deserves more care than most teams give it.
2026 is the year inference spending overtook training: $23.3 billion versus $19 billion in AI-optimized cloud infrastructure, reaching 55% of that spend this year and 59% next, per Gartner. Training is a project that ends. Inference is a run rate that scales with every user, every feature, and every agent you ship.
Even OpenAI feels it: its inference costs reached an estimated $8.4 billion in 2025 and are projected at $14.1 billion for 2026, per Sacra, and leaked documents reported by The Register put its Azure inference spend at $8.7 billion in just the first three quarters of 2025.
And it’s already constraining strategy: about one in five respondents say AI operating costs are limiting their organization’s AI use, and AI high performers report cost constraints on their coding agents roughly three times as often as everyone else, per McKinsey’s State of AI 2026.
Which means the 9x provider spread above isn’t a curiosity. For an inference-heavy product, it’s the difference between a feature with healthy margins and a feature finance wants to kill.
This is home turf for us. CloudZero maintains the pricing guides teams use for half this roster, Groq, Together AI, Bedrock, Vertex AI, DeepSeek, and Mistral among them, and we’ve shipped the other half of the problem too: Bedrock bills broken down to the individual user, because a lump-sum AI invoice answers nothing. That vantage point shapes how this guide judges providers.
The cost-per-outcome test
Every list in this category ranks sticker prices. Sticker prices are where cost analysis starts, not where it ends, and treating them as the whole answer is how teams pick a “cheap” provider and end up spending more. So every provider below gets scored on five questions:
- Quantization transparency: is the bargain rate serving the full-precision model, or an FP8 or FP4 variant under the same name? Same model name, different quality, different retry rate.
- Effective throughput: tokens per second is a cost line in disguise. A model that streams 3x faster finishes agent tasks sooner, holds less state, and burns fewer timeout retries.
- Reliability tax: peak-hour capacity errors, cold starts, and rate limits convert into retries and fallbacks that land on your bill, not the provider’s.
- Discount structure: batch APIs commonly run 50% off with a 24-hour turnaround, and Together, Fireworks, Groq, and Novita all offer one, per Morph’s pricing tracker. If half your workload isn’t latency-sensitive, the sticker was never your price.
- Attribution: can you tell from the provider’s billing which product, feature, or customer drove the spend? Almost none can, which is where this article ends up.
Cheapest per token is not cheapest per successful task. Everything below is scored with that lens.
Cheapest LLM inference providers: the price table
Llama 3.3 70B Instruct is the most widely hosted open model, which makes it the cleanest benchmark for hunting the cheapest LLM inference rates. Per million tokens, input/output, on-demand serverless list prices:
| Provider | Llama 3.3 70B (in/out per M tokens) | Notes |
|---|---|---|
| DeepInfra | $0.10 / $0.32 | cheapest full listing; quantized Turbo tier lower still |
| OpenRouter | $0.10 / $0.32 | aggregator; small fee on credits |
| Novita | $0.135 / $0.40 | budget tier |
| Groq | $0.59 / $0.79 | custom LPU silicon, speed premium |
| Amazon Bedrock | ~$0.72 blended | hyperscaler endpoint |
| Fireworks | ~$0.90 to $1.20 | speed-optimized GPU serving |
| Together AI | ~$1.04 | breadth and fine-tuning platform |
Note: Rates verified August to September 2026 via Infrabase and Saturn Cloud; the spread on the same model name is the entire point. DeepSeek V4 Pro repeats the pattern: Fireworks, Together, and Baseten all list an identical $1.74/$3.48 reference rate while DeepInfra ($1.30/$2.60) and Novita ($1.60/$3.20) undercut it, per Morph. Check the live pricing page before committing volume. These numbers have a shelf life measured in weeks.
Fastest LLM inference: the custom silicon tier
If your product feels slow, no discount fixes it. These three built or bought non-GPU hardware and compete on raw speed; you pay a premium per token and buy it back in latency. For the fastest LLM inference available, this is the tier.
Groq
Groq’s LPU silicon is purpose-built for the token-by-token decode phase GPUs handle less efficiently, and it shows: consistently low time to first token (~0.7 seconds on gpt-oss-120B) and roughly 478 tokens per second on the same model, per measurements compiled by Infrabase.
The fine print: a deliberately curated catalog (Llama, gpt-oss, Qwen3 32B, and Kimi K2), known peak-hour capacity errors, and custom fine-tune hosting gated behind Enterprise approval. Also note the ownership question: NVIDIA licensed Groq’s inference technology in a reported $20 billion deal in December 2025, taking founder Jonathan Ross and most of the engineering team with it. GroqCloud still operates independently under CEO Simon Edwards and the published rate card is unchanged, but build for portability.
Cost-per-outcome verdict: strong for latency-critical supported models; pair with a GPU fallback for high availability.
Our Groq pricing breakdown has the full rate card.
Cerebras
Cerebras holds the throughput crown and it isn’t close: the company reports around 3,000 tokens per second on gpt-oss-120B on its wafer-scale engine, with Artificial Analysis measuring roughly 1,700 sustained, still far ahead of the field. The catalog is tiny, one production model plus previews, which is the price of the speed.
Cost-per-outcome verdict: unmatched when your model’s in the catalog and throughput dominates your economics; not a general-purpose host.
SambaNova
SambaNova publishes 435 tokens per second on MiniMax M2.7 and over 600 on gpt-oss-120B, class-leading for frontier-size open weights, with linear per-token pricing, per Inworld’s throughput guide. It’s the least known of the silicon trio, which cuts both ways on ecosystem maturity.
Cost-per-outcome verdict: serious throughput per dollar on supported large models; confirm catalog fit before you fall in love.
Breadth and price platforms: the GPU serving middle
The center of the market, where most production workloads actually live: wide catalogs, OpenAI-compatible endpoints, and a price war that works in your favor.
Together AI
Together AI pairs one of the widest catalogs with fine-tuning and serving on one platform, plus ATLAS speculative decoding for structured, cache-friendly workloads. Its per-token rates sit at the premium end of GPU serving (~$1.04 on the benchmark), which buys platform depth rather than the lowest sticker.
Cost-per-outcome verdict: strong for teams iterating on fine-tunes; the batch discount matters for offline work. Full numbers in our Together AI pricing guide.
Fireworks AI
Fireworks does one thing very well: serve optimized open weights fast, with aggressive pricing, batch discounts, speculative decoding, and fine-tune-and-serve on one API. It clusters mid-pack on price and near the top of GPU-based speed.
Cost-per-outcome verdict: one of the best speed-to-price ratios on GPUs, and the default shortlist entry for production open-model serving.
DeepInfra
DeepInfra is the price floor of the serious tier: $0.10/$0.32 on the benchmark, gpt-oss-120B at roughly $0.05 blended, and the widest catalog of current open models (Kimi K2, Qwen3.5, GLM-5, DeepSeek V4, Nemotron), per Infrabase. Its very cheapest rates are quantized Turbo tiers, so test question one applies.
Cost-per-outcome verdict: the cost-per-token champion; verify the precision tier matches your quality bar before declaring victory.
Novita
Novita competes one notch above DeepInfra’s floor ($0.135/$0.40 on the benchmark) with a broad catalog and a 50% batch API. It’s a budget workhorse with less brand gravity than the platforms above it, which matters mostly in enterprise procurement meetings.
Cost-per-outcome verdict: a legitimate low-cost contender; benchmark reliability on your own traffic first.
Nebius
Nebius is two businesses wearing one logo: a neocloud renting raw GPUs and a token factory serving open models per token. That dual structure suits teams who expect to graduate from serverless to dedicated capacity without a vendor migration.
Cost-per-outcome verdict: solid mid-market rates with a built-in scale-up path; model the crossover point before you hit it.
Best inference platforms for custom models
Somewhere around your first serious fine-tune, the question flips from “which hosted model” to “how do we serve our model.” Different tier, different winners.
Baseten
Baseten is built for that flip: its Truss abstraction gives real control over custom serving, per-minute dedicated GPU billing runs from a T4 at about $0.011 a minute to a B200 at about $0.166, scale-to-zero bounds spiky workloads, and its delivery network cut cold starts roughly 2 to 3x versus its own prior baseline, per Deepak Gupta’s platform review. Popular open models are also served per-token.
Cost-per-outcome verdict: strong for custom and fine-tuned models with a platform engineer on staff; the wrong tool for calling a hosted Llama endpoint this afternoon.
Replicate
Replicate, which Cloudflare agreed to acquire in November 2025, is where experiments start: a huge community model library, per-use pricing, and minimal setup. The acquisition signals an edge-inference future worth watching.
Cost-per-outcome verdict: excellent for prototyping and side projects; at sustained production volume, the dedicated platforms above win the math.
Modal
Modal hands you serverless GPU containers instead of a model menu: bring any model or pipeline, pay per second of compute, scale to zero. It’s infrastructure with an inference-shaped hole in it, beloved by teams who want full stack control without owning capacity.
Cost-per-outcome verdict: the flexible power option; your bill depends on your engineering more than any list price.
Anyscale
Anyscale, which Nscale agreed to acquire in 2026, brings the Ray lineage to serving (Ray itself moved to the PyTorch Foundation in late 2025), aimed at teams whose AI workloads sprawl beyond single-endpoint inference.
Cost-per-outcome verdict: strongest when inference is one stage of a bigger distributed pipeline; oversized for straightforward serverless calls.
Hyperscaler serverless inference
Zero new vendors, procurement already loves them, ecosystem gravity included in the price.
Amazon Bedrock
Amazon Bedrock serves open and proprietary models inside AWS at per-token rates (the benchmark lands around $0.72 blended), plus intelligent prompt routing, which AWS says can cut costs up to about 30% by steering each prompt to the cheapest model in the same family that can answer it.
Cost-per-outcome verdict: the path of least resistance for AWS shops; watch cross-region and provisioned-throughput lines. Our Bedrock pricing guide covers the whole model.
Google Vertex AI
Google Vertex AI hosts Gemini alongside open models with enterprise controls and GCP integration. As with Bedrock, the premium over specialists buys governance and unified procurement, not the lowest token price.
Cost-per-outcome verdict: strongest inside GCP estates; compare specialist rates before routing high-volume open-model traffic here. Details in our Vertex AI pricing guide.
Microsoft Foundry serverless
Microsoft Foundry’s serverless endpoints, renamed from Azure AI Foundry on January 1, 2026 with pay-per-token standard deployments unchanged, bring open and partner models to Azure enterprises in the compliance idiom Azure buyers expect.
Cost-per-outcome verdict: the same tradeoff as its hyperscaler peers, convenience over sticker.
Hyperbolic
Hyperbolic is the wildcard: a decentralized GPU network serving open models at aggressive rates by aggregating underutilized capacity. Genuinely interesting economics, younger operational maturity than everything above it.
Cost-per-outcome verdict: worth a benchmark for cost-tolerant workloads; keep a conventional fallback warm.
Honorable mentions
- Lambda Inference (per-token serving from the GPU cloud veteran)
- Cloudflare Workers AI (edge inference for latency-sensitive light models)
- GMI Cloud (reliability-positioned serving)
- Scaleway (the EU-hosted option at ~€0.90 per million tokens each way on the benchmark)
- RunPod serverless (GPU pods with a serverless layer)
- Inworld (realtime routing across 220+ models for consumer apps)
- OneInfer (kernel-optimized multimodal serving)
Boundary note: OpenRouter appears in the price table because it matches the cheapest Llama rates, but it’s an aggregator routing to providers on this list, one hop and a small fee in exchange for zero lock-in. It lives fully in our gateways guide.
Every provider you add is another invoice
Now the ending the pricing trackers never write. Say you nail this: Groq on the latency-critical path, DeepInfra for bulk classification, Baseten for the fine-tune, Bedrock because legal likes AWS paper. Congratulations, you’ve optimized cost per token four times over, and created four invoices in four formats on four schedules, none of which says which product or customer the spend actually served.
That’s question five of the cost-per-outcome test, and no provider answers it, because no provider can see past its own meter. Multi-provider inference is precisely the fragmentation CloudZero closes: provider and gateway telemetry lands in one place, gets allocated to the products, features, and customers behind it, and comes back as unit economics instead of a stack of PDFs.
That’s not a roadmap slide. The Bedrock bill-by-user work already breaks a hyperscaler’s lump-sum AI charges down to the individual person and team, and the connector catalog covers OpenAI, Anthropic, and the cloud bills around them.
So run the playbook in order: pick providers with the cost-per-outcome test, route them through a gateway so the telemetry is yours, and land the stream in a ledger that answers the only question the board asks about inference spend: what did we get for it?
You’ve got the shortlist. Now see what your inference spend looks like when all of it lands in one ledger: schedule a CloudZero demo with your own provider mix, or take the self-guided product tour first and poke around at your own pace.
Frequently asked questions about LLM inference providers
