← All articles · Hardware
On-prem vs cloud
The break-even point between buying GPUs and renting them comes faster than most teams expect — Blueprint shows you exactly where.
3 min read
The two cost models
On-prem is capital expense + power + amortization. Buy an RTX 4090 once ($1,800), use it for 24 months, pay $40/mo for power at typical utilization. Per-hour cost over its useful life is roughly $0.30-0.50 depending on duty cycle.
Cloud is operating expense per hour. A single A100 80GB on Lambda / Runpod / Vast is $1.10-$1.80/hour. AWS, GCP, Azure mark it up significantly — $3-5/hour is typical for on-demand pricing.
The cards on the Hardware tab compute both and show you the per-hour delta, plus the months-to-break-even given your workload's hourly utilization.
When cloud wins
- Bursty workload: you need 10× capacity for 8 hours per month and idle the rest of the time. The cloud's per-second billing recovers the unused hours; on-prem hardware sits depreciating.
- Need a GPU you can't buy at retail: H100s and H200s are effectively cloud-only for most teams. Consumer cards top out at 24 GB (RTX 4090); going larger means going to cloud or paying enterprise prices for an A6000.
- Short-horizon project: a one-shot fine-tune that takes 40 GPU- hours costs $80 in the cloud and saves you the entire capex conversation.
When on-prem wins
- 24/7 production serving: a single A100 running constantly costs $9,600/year in the cloud (at $1.10/hr × 8,760 hours). The same card to buy is ~$12,000 retail; break-even is ~13 months. Past that, on-prem is pure savings.
- Data sensitivity: customer prompts never leaving your network is a compliance requirement in many regulated industries (healthcare, legal, finance, defense). Cloud means cloud-provider sees the traffic, even if encrypted in transit.
- You're already a hardware shop: existing power, cooling, bandwidth, rack space, ops headcount. The marginal cost of adding GPUs is just the cards themselves.
How to read the break-even card
The Hardware tab's break-even card takes your chosen GPU + an estimated utilization percentage (the fraction of hours per month you'll actually run inference, not idle) and computes the cumulative cost curve for both. The crossover month is where on-prem starts saving you money.
If your utilization is below 20%, cloud almost always wins. Above 70%, on-prem almost always wins. Between those, it depends on the specific card and your time horizon.
Common pitfalls
- Forgetting GPU resale value. A 2-year-old A6000 still sells for ~50% of retail. Cloud has no asset to recover.
- Underestimating cloud egress. Pulling model files or logs off the cloud platform can add up — multi-hundred-dollar bills for what seems like routine ops.
- Overestimating utilization. Most teams claim "24/7 production" and run 30% of those hours actually inferring. Be honest with the utilization slider.
Need help with this in production?
Inspire AI Lab runs LLM optimization engagements end-to-end using Blueprint. If you'd rather hand the work to us instead of running it yourself, book a 30-minute review.