Skip to content
Blueprint

Your LLM stack, on your hardware

Run. Tune. Operate.

Self-host open LLMs on your hardware. Tune them for your workload with custom calibration, LoRA fine-tuning, and runtime optimizations. Operate them as production tooling — always-on, live metrics, multi-engine without code changes.

Download BlueprintView source on GitHubOr try the planner in your browser

Apache 2.0, free for any use. Built and maintained by Inspire AI Lab — we use Blueprint on every engagement. Want us to run an engagement? Book a 30-min review.

What Blueprint does

Three things. Tightly integrated. One install.

Self-hosting is necessary but not sufficient — running the model is the easy part. The value is in tuning it for the work you actually do, and operating it like production code, not a notebook experiment.

Run

Self-host on your hardware

Pick a model, size it for your machine, install and serve in one path. No per-token API bill, no data leaving your network.

  • Curated catalog with VRAM math + cost math (try in browser)
  • Multi-engine: llama.cpp · vLLM · TensorRT-LLM
  • OpenAI-compatible API on a port you choose
  • GPU + CPU sizing for laptops, on-prem servers, and rented clouds

Tune

Optimize for your workload

Off-the-shelf quantizations are calibrated to web text, not your data. Five techniques to close the gap — each costs less than calling GPT-4 for a week.

  • Custom imatrix calibration → custom quantization
  • LoRA / QLoRA fine-tune on your data
  • LLMLingua prompt compression (2–10× shrink)
  • Semantic routing across small + large models
  • Prompt cache for repeated prefixes

Operate

Production-ready, day one

Runs as a service that survives reboots. Live metrics, one dashboard, no SSH-into-the-server.

  • Windows Service / Linux systemd unit registration
  • Live GPU / VRAM / RAM + request metrics dashboard
  • Swap engines without changing your application code
  • Single binary install — no Docker, no Python env you have to debug

How we optimize

Five techniques. Each closes a real gap. Each is shipped.

"Optimize your LLM" can mean a lot of things — most of which never get measured. These five are the techniques we use on every engagement, with the gap each one closes named explicitly.

  1. 01

    Custom imatrix calibration

    What
    Run llama-imatrix against your prompts to capture per-layer activation statistics. Quantize with that imatrix so the bits land where YOUR workload actually pushes activation magnitude.
    Why
    Off-the-shelf Q4 quants are calibrated to web-text corpora like C4. Your customer-support transcripts, your SQL prompts, your legal contracts look nothing like that — and they pay the accuracy bill.
    Proof
    Custom Q4_K_M typically beats the stock pre-quant by 8–15% on the workload that matters, at the same file size.
  2. 02

    Domain adaptation (LoRA / QLoRA)

    What
    Fine-tune a low-rank adapter on your data using the bundled Python sidecar. QLoRA lets you do this on a single consumer 24GB GPU; LoRA scales up to A100/H100 when needed.
    Why
    Prompt engineering plateaus fast. Past that ceiling, the model needs to learn your domain — vocabulary, format, decisions you'd make. Adapters are 50× smaller than the base model and load into llama.cpp, vLLM, and TensorRT-LLM cleanly.
    Proof
    A 7B base + a 50MB LoRA routinely beats a 70B base on the trained task — at 5–10× lower latency and a fraction of the VRAM.
  3. 03

    Prompt compression (LLMLingua)

    What
    Run prompts through a small compression model that strips low-information tokens before the prompt hits your main model. 2× to 10× shrink with measured downstream-accuracy retention.
    Why
    If you're paying for context — and you are, in latency and VRAM — the cheap tokens are the redundant ones. LLMLingua identifies them.
    Proof
    Microsoft Research: 2–20× compression on long-context QA with <2% accuracy degradation. Blueprint runs the published model unmodified.
  4. 04

    Semantic routing

    What
    Classify incoming queries by complexity before routing. Easy ones go to a small fast model; hard ones go to your big model. Escalation rules are yours to set.
    Why
    You don't need a 70B for 'what time is it.' You don't need GPT-4 for FAQ deflection. Routing collapses the average cost per query to a fraction without dropping the ceiling on hard ones.
    Proof
    Common pattern: 60–80% of traffic served by a fast 3B; the long tail served by a 70B. Average cost drops by an order of magnitude.
  5. 05

    Prompt cache

    What
    Cache the KV-state of common prefixes (system prompts, retrieval context, few-shot examples). Next request with the same prefix skips the prefill phase entirely.
    Why
    Most production traffic shares a 1–4 kilo-token system prompt across every request. That prefix gets re-tokenized + re-prefilled every time unless something catches it.
    Proof
    Cache hit on a 4k-token shared prefix cuts time-to-first-token by 80–95% and frees the GPU for new work.

Walk through all five on your workload, on a screenshare.

Book a 30-min demo

Who uses it

Three audiences. Same tool. Same deliverable shape.

Consultants

Running LLM optimization engagements

The workflow IS the engagement. Calibrate on the client’s prompts; fine-tune on their data; hand over a folder and an invoice. Same toolkit we use at Inspire AI Lab — yours, free.

Platform teams

Self-hosting your own LLM stack

Skip the bespoke tooling. Use the same calibrate → train → serve → optimize loop your CTO would have built in-house, except it exists, works today, and you can read every line of it.

Practitioners learning

The full pipeline in 10 minutes

Three bundled demo datasets — customer support intent, SQL generation, contract Q&A — walk you through the whole pipeline end to end. No signup, no gate.

Why this exists

Built by an engineering firm. Sold to engineers.

We're Inspire AI Lab — a small firm that runs LLM optimization engagements. Every project looked the same: calibrate a quant for the client's vocabulary, fine-tune a LoRA on their data, pick the right runtime, set up the production cost optimizations, write a report. We were doing the same handful of steps by hand on every project, so we turned them into a tool.

Blueprint is what we use ourselves. It's Apache 2.0 — yours to read, fork, deploy, modify. The consulting practice is how we keep the lights on; the toolkit is how we move faster than firms that don't have one.

If you'd rather hand the engagement to us — calibration, fine-tuning, production deployment, ongoing operations — that's the consulting offer. Book a 30-min review →

Get going

Three ways to start, depending on where you are.

01

Try the planner

Browser-only catalog + VRAM math. Pick a model, see what it’d cost. No download. ~5 minutes.

Open Plan

02

Download the app

Blueprint for Windows, macOS, or Linux. Apache 2.0. No license key, no signup. Run the full workflow.

Download

03

Have us deploy it

A 30-minute review covers your workload, hardware, security posture. Then we run the engagement.

Book a demo

Apache 2.0. Source on GitHub. Consulting at inspireailab.com.