Self-hosted vs API: the actual math at 1M / 10M / 100M tokens/month

"Should we self-host or pay an API?" is the question that triggers most engagements. The answer is usage-dependent, and the crossover point is lower than most teams assume. This post does the math three times — at 1M, 10M, and 100M tokens per month — with all assumptions disclosed.

All numbers are 2026 list prices, rounded for readability. Substitute your own assumptions where ours don't match. The shape of the answer is the durable part.

The model: API side

GPT-4o-mini at $0.15 input / $0.60 output per million tokens (2026 OpenAI pricing).

Cost per request, assuming 2:1 input:output ratio:

$0.15 × (2/3) + $0.60 × (1/3) = $0.10 + $0.20 = $0.30 per million tokens total

(We round to $0.30/M for the rest of this analysis.)

So:

1M tokens/mo = $0.30
10M tokens/mo = $3
100M tokens/mo = $30
1B tokens/mo = $300
10B tokens/mo = $3,000

These are small numbers. For most companies, GPT-4o-mini at 10B tokens/month — already a lot of traffic — is $3K/mo. API economics are wonderful at low volume.

The model: self-hosted side

A Llama-3.3 70B Instruct or Qwen 2.5 72B serving comparable-quality responses on owned hardware.

Hardware capex: A workstation with 2× A100 40GB or 1× H100 80GB. List price for the H100 server (full build, not just the card): ~$50,000. Amortized over 3 years: $16,700/year, or $1,400/month.

Electricity: ~700W under sustained load. At $0.15/kWh, that's $76/month if running 24/7.

Cooling / rack space (data center) or air conditioning (office): ~20% of electricity → $15/month.

Network egress (when serving over LAN): negligible. Over the internet, a separate calculation.

Engineering overhead: the gnarly one. A self-hosted LLM service needs someone watching it. We assume 0.05 FTE (~2 hours/week) at $150/hour fully-loaded = $1,300/month.

Total fixed monthly cost: ~$2,800.

These numbers can change a lot:

Renting (not buying) the GPU: $1.50-3/hr on Lambda Labs / Runpod for an H100 = $1,100-2,200/mo at 24/7. Less capex risk, more recurring.
Smaller hardware: 7B model on a $2,000 4090 workstation → $55/mo amortized, basically free at 24/7.
More engineering: enterprise contexts can spend 0.25 FTE = $6,500/mo on the same service.

We use $2,800/mo for the H100-class build, $300/mo for a 7B-on-4090 setup.

Throughput

Self-hosted only saves money if it covers the workload. Indicative throughputs:

70B Q4 on H100 with vLLM, concurrency 16:

~620 tokens/sec sustained
1.6M tokens/hour
1.16B tokens/month (24/7)

7B Q4 on a 4090 with llama.cpp, concurrency 4:

~140 tokens/sec sustained
0.5M tokens/hour
365M tokens/month (24/7)

So the H100 build covers up to ~1B tokens/month before saturation. The 4090 build covers ~300M.

The math at each tier

1M tokens/month

API: $0.30
Self-host H100 build: $2,800
Self-host 4090 build: $300

Verdict: API wins by 1000-9000×. Don't self-host until your usage is meaningful.

10M tokens/month

API: $3
Self-host H100 build: $2,800
Self-host 4090 build: $300

API still wins by 100×. The 4090 build is now within an order of magnitude but still loses.

100M tokens/month

API: $30
Self-host H100 build: $2,800
Self-host 4090 build: $300

API still wins. The H100 build is barely break-even with itself.

1B tokens/month

API: $300
Self-host H100 build: $2,800
Self-host 4090 build: $300 (but you're now at 4090 capacity ceiling)

Self-host 4090 wins. H100 build still 9× more than API.

10B tokens/month

API: $3,000
Self-host H100 build: $2,800
Self-host 4090 build: capacity exceeded (would need ~30 4090 builds)

H100 build wins by ~$200/mo. Crossover.

100B tokens/month

API: $30,000
Self-host H100 build: $2,800 (capacity exceeded, need 100 H100 builds → $280,000/mo)
Self-host H100 cluster (10 H100s, dedicated team): $50,000/mo

API wins again — because you'd need a real data center build to keep up, and the dedicated team alone eats the API delta.

Where the crossover actually lives

For a 70B-class workload, the API crossover is around 10B tokens/month for a single H100 build. Below that, API is cheaper. Above that, on-prem starts winning until you hit the multi-server zone where the engineering overhead catches up.

For a 7B-class workload on a single 4090, the crossover is at about 1B tokens/month.

Most companies are at 10-100M tokens/month. They sit firmly in API territory. The companies for whom self-hosting is the obvious answer are doing 1B+ tokens/month, and care about other factors that change the math.

What else changes the math

The pure token-cost math is one input. Other factors that often dominate:

Data residency. "All user data must stay in our VPC." API is off the table; cost discussion is moot.

Latency. On-prem inference is 0-2ms network round-trip. API is 50-300ms depending on geography. For chat where TTFT matters, on-prem is structurally faster.

Rate limits. OpenAI tier 5 lets you do ~30K requests/minute. Above that you're negotiating. On-prem has no rate limit beyond your hardware.

Vendor lock-in. APIs change pricing and deprecate models. On-prem models you've fine-tuned aren't going away.

Customizability. Custom calibration, LoRA fine-tuning, prompt compression — all require model weights you control. API only gives you the inference.

Predictable spend. $2,800/mo flat is easier to budget than $300-30,000/mo variable. Finance teams prefer the flat number even at slightly higher mean cost.

When self-hosting wins despite worse math

A few engagements where on-prem made sense even though the API was cheaper:

Regulated industry (health, finance, legal) where the data residency requirement was a hard constraint.
High-variance traffic where the API rate limits would have throttled the busy hour.
Customized model (LoRA, custom calibration) where the customer was getting genuine accuracy lift that the API couldn't match.
Burst seasonality where reserved hardware sat idle 80% of the time but covered the December peak that would have hit API rate limits.

When self-hosting wins for one of these reasons, the cost math doesn't have to. It's the right business answer regardless.

The honest summary

Less than 1B tokens/month for general workloads: stay on API. The math is so lopsided that self-hosting only makes sense for non-cost reasons.
1-10B tokens/month: think about it carefully. The crossover is in this range, and the qualitative factors (residency, latency, customization) usually push the decision.
>10B tokens/month: self-hosting is cheaper, but the engineering complexity is real. Make sure you have the team for it.

We've been on both sides of this conversation. Self-hosting isn't always the right answer just because it can be. It's the right answer when it solves a problem the API can't, or when the math is comfortably on its side.