LLM Inference Providers Compared: Groq, Together AI, DeepInfra, Replicate, Modal and Baseten
You've prototyped on a frontier model API, the bill is starting to matter, and someone suggests "just run an open model somewhere cheaper." Then you open six tabs (Groq, Together AI, DeepInfra, Replicate, Modal, Baseten) and realise they aren't selling the same thing. Some sell tokens. Some sell GPU time. Some sell a place to run your own container that happens to have a GPU in it.
This post sorts them by what you're actually paying for, with prices as of November 2026 from each provider's own pricing page. I don't quote speed benchmarks unless the provider publishes the number itself, and where I do, it's their number, not a test.
Three things you can buy
- Tokens on a shared model. The provider runs a popular open model and bills per million input and output tokens. You get no idle cost and no control over hardware. Groq, Together AI serverless, DeepInfra and Baseten's Model APIs all do this.
- A dedicated endpoint. The provider runs a model on GPUs reserved for you and bills per GPU-hour or minute, with or without traffic. Together, DeepInfra and Baseten offer this.
- Your own code on rented GPUs. You bring a container or a Python function, and the platform bills by the second while it runs. Modal is built around this, and Replicate does it for private models.
Most teams should start at 1 and move right only when a number forces them to: steady high volume, a fine-tuned model nobody hosts, or a latency target the shared tier can't meet.
The token sellers
These are the easiest switch from OpenAI, because they speak the same API. Each one documents an OpenAI-compatible base URL:
from openai import OpenAI
import os
# Groq: https://api.groq.com/openai/v1
# Together: https://api.together.ai/v1
# DeepInfra: https://api.deepinfra.com/v1/openai
client = OpenAI(
base_url="https://api.together.ai/v1",
api_key=os.environ["TOGETHER_API_KEY"],
)
Sources: Groq, Together, DeepInfra. "Compatible" has limits. Groq's page, for example, says logprobs, logit_bias and top_logprobs return a 400 error and that n must be 1, so test the parameters you rely on.
To compare like with like, here's one open model that several providers list, OpenAI's gpt-oss-120b, per million tokens as of November 2026:
| Provider | Input | Output | Source |
|---|---|---|---|
| Groq | $0.15 | $0.60 | models page |
| Together AI | $0.15 | $0.60 | pricing |
| Baseten Model APIs | $0.10 | $0.50 | pricing |
DeepInfra's pricing page didn't list that model when I checked, but its Llama-3.3-70B-Instruct-Turbo was $0.10 in and $0.32 out, and Qwen3-32B $0.08 in and $0.28 out.
What separates them beyond price:
- Groq builds its own inference hardware (groq.com describes its LPX working alongside Nvidia GPUs), and its models page lists its own speed figures: 500 tokens per second for gpt-oss-120b and 1,000 for gpt-oss-20b. Those are Groq's numbers. The catalogue is short, and some models (like
llama-3.3-70b-versatile) show "Contact Sales" in place of a price. On ownership: Groq announced on December 24, 2025 a non-exclusive licensing deal with Nvidia. Its founder and president moved to Nvidia, Simon Edwards became CEO, and Groq said it stays independent and that GroqCloud "will continue to operate without interruption." There's a free plan with lower rate limits. - Together AI has the broadest menu: serverless tokens, dedicated endpoints, GPU clusters and fine-tuning (for example $0.34 per million training tokens for supervised fine-tuning of Llama 3.1 8B, with a $4 job minimum, per its pricing page).
- DeepInfra has service tiers that multiply the base price: Priority at 1.5× for faster time to first token, Flex at 0.8× for cheaper and slower. It also caps accounts at 200 concurrent requests unless you talk to sales, and you add a card or prepay before using it.
Our Groq vs Together AI and Together AI vs DeepInfra comparisons go further on each pair.
The GPU-time sellers
Once you're paying for GPU time, the question becomes: what do I pay when nobody is calling? Here's one H100 across providers, as listed on each pricing page in November 2026. They aren't identical products, so read the "what you get" column.
| Provider | H100 price | Billing unit | What you get |
|---|---|---|---|
| DeepInfra | $2.90/hour | minute granularity | Dedicated GPU for a custom LLM (pricing) |
| Modal | $0.001097/second (about $3.95/hour) | per second | Your function in a container; scales to zero (pricing) |
| Together AI | $5.49/hour | listed per GPU-hour | Single-tenant dedicated endpoint (pricing) |
| Replicate | $0.001525/second ($5.49/hour) | per second | Private model on dedicated hardware (pricing) |
| Baseten | $0.10833/minute ($6.50/hour) | per minute | Dedicated deployment (pricing) |
Together also sells raw H100s in GPU clusters at $3.99/hour on demand, which isn't the same product as its managed endpoint.
The idle-cost rules are where real bills diverge:
- Replicate says private models bill for every period the instance is online, including setup, active processing and idle time while it waits for requests. Fast-booting fine-tunes are the exception and bill only while active. Replicate has also changed hands: it announced on November 17, 2025 that it is joining Cloudflare, saying "The API isn't changing" and that it continues as a distinct brand.
- Modal stops charging once an app scales to zero containers, but by default a container stays alive for 60 seconds after its last input, and you pay for that. Plans add a monthly fee: Starter is $0 with $30 of credits, Team is $250 with $100 of credits. Since September 1, 2026, Starter's shared endpoint tokens bill from the first request rather than drawing on the credit.
- Baseten lists the same per-minute GPU rates for dedicated inference and training.
Here's what Modal's model looks like in code, from its guide:
import modal
app = modal.App("example-inference")
image = modal.Image.debian_slim().uv_pip_install("transformers[torch]")
@app.function(gpu="h100", image=image)
def chat(prompt: str | None = None) -> list[dict]:
...
You run it with modal run path/to/file.py. No Dockerfile, no YAML. That's the appeal, and it's also why Modal is a better fit for batch jobs and custom pipelines than for "just give me an endpoint." See Modal vs Baseten and Hugging Face vs Replicate for more.
A decision rule
- Prototype or spiky traffic on a popular open model: a token seller. Pick on price for your model and on the API parameters you need.
- Latency is the product (voice, live agents): try Groq first, and measure on your own prompts rather than trusting anyone's chart, including theirs.
- Steady, high volume on one model: do the maths on a dedicated endpoint. Divide the hourly price by the tokens you'd actually push through it in an hour; if that beats the per-token rate with room to spare, switch.
- Fine-tuned or unusual models, or pipelines with pre- and post-processing: Modal, Replicate private models or Baseten dedicated.
- Data must stay on your machine: none of the above. See running local LLMs for development.
Before any of that, count your tokens. Our token counter gives you the input side of the estimate, and our Groq vs OpenAI comparison shows what changes when you leave a frontier API.
Written by AIDToolStack Editorial
Researched from official documentation, specifications and vendor pages, listed under Sources below. Last checked November 2026.
Sources
- console.groq.com/docs/openai
- docs.together.ai/docs/openai-api-compatibility
- docs.deepinfra.com/chat/overview
- console.groq.com/docs/models
- www.together.ai/pricing
- www.baseten.co/pricing/
- deepinfra.com/pricing
- groq.com/pricing
- groq.com/newsroom/groq-and-nvidia-enter-non-exclusive-inference-technology-licensing-agreement-to-accelerate-ai-inference-at-global-scale
- console.groq.com/docs/rate-limits
- modal.com/pricing
- replicate.com/pricing
- replicate.com/blog/replicate-cloudflare
- modal.com/docs/guide
Related Posts
Explore on AIDToolStack
Discover tools and servers featured in our directory.
AI Tools
MCP Servers
Sponsor Our Newsletter
Reach thousands of developers who are actively evaluating AI tools, MCP servers, and dev infrastructure. Our weekly newsletter goes to engaged technical decision-makers.
All sponsored content is clearly labeled per our editorial policy.