← All articles · Hardware
Sizing hardware
Read the tier recommendations, understand why each one's labeled what it is, and pick the box that fits your workload without overpaying.
2 min read
The three-tier model
The Hardware tab classifies your chosen (model, quant, context, batch) into three hardware tiers:
- Laptop — 8-24 GB consumer GPU (RTX 4060, 4070, 4080 mobile, M-series Macs with unified memory). Suitable for: dev work, single- user inference, demos. Not suitable for production serving.
- Workstation / single-server — 24-80 GB pro/server GPU (RTX 4090, A5000, A6000, L40S). Suitable for: small-team internal tools, single-tenant production with under 10 concurrent requests, calibration + fine-tuning of models under 30B.
- Datacenter — 80+ GB datacenter GPU (A100 80GB, H100, H200) or multi-GPU sled. Suitable for: serving 70B+ models, batched production traffic, fine-tuning at scale.
The cards on the Hardware tab show recommended GPUs in each tier with their approximate retail price (consumer) or cloud hourly rate (datacenter).
How the tier is picked
The tier is the smallest GPU whose VRAM exceeds the total computed in the VRAM math (weights + KV cache + activations + overhead) for your chosen config. If you're at 23 GB total and the recommendation is "workstation / RTX 4090 (24 GB)," that's a tight fit — you'll OOM on context overflow. The card flags this with a yellow band.
When to upgrade tiers
- You're at 90%+ VRAM utilization during steady-state serving. Headroom matters: long requests, KV cache spillage, prompt-cache evictions all push usage above your average.
- Your latency floor is too high. A laptop GPU has limited memory bandwidth (200-500 GB/s) compared to datacenter parts (3-5 TB/s). If your time-to-first-token is unacceptable, throwing a bigger GPU at it often helps more than throwing a smaller model at it.
- You're paying for downtime. Production traffic with even modest uptime requirements (99%+) eventually needs hardware that can survive a card failure — the workstation tier is single-GPU; the datacenter tier supports redundant sleds.
Common pitfalls
- Buying for peak workload that never arrives. Right-size for your 90th-percentile real load, not your hypothetical max.
- Underestimating power. Datacenter GPUs draw 350-700W each. A desktop PSU that handles one consumer card won't run two A6000s. The recommendations card flags power budget where it's a concern.
- Forgetting the CPU + RAM + storage. A 70B model needs 50+ GB of system RAM if you're not putting it all on GPU (CPU offload), and fast NVMe to load it quickly. The recommendations assume a balanced build; don't pair an H100 with a 16 GB / SATA-SSD box.
Need help with this in production?
Inspire AI Lab runs LLM optimization engagements end-to-end using Blueprint. If you'd rather hand the work to us instead of running it yourself, book a 30-minute review.