Skip to content
Blueprint

← All articles · Hosts

Adding a host

Register a Linux GPU box so Blueprint can SSH in, push-install the svc, and drive llama-server on it from one desktop.

3 min read

What a "host" is

A Linux machine you OWN end-to-end via SSH. Blueprint:

  • Installs the blueprint-svc binary on it (push-install)
  • Drives the supervisor through an SSH-tunneled HTTP control plane
  • Caches the bearer token for that host in your OS keychain so subsequent connects skip the SFTP read

Distinct from a Remote server (OpenAI-compatible endpoint someone else operates) — hosts are yours, remote servers are theirs. See Remote servers for the other path.

What you need on the host

  • Linux: tested on Ubuntu 22.04 / 24.04, Debian 12. RHEL family should work but isn't routinely tested.
  • SSH access via a key (not password). Standard config in ~/.ssh/config or paste the key path directly in the Add Host form.
  • A GPU if you want it to serve at any reasonable speed (Blueprint doesn't enforce this; you'll just get CPU-only llama.cpp if no NVIDIA driver is present).
  • Outbound internet for pulling models from HuggingFace.

How to add

  1. Hosts tab → Add host.
  2. Fill in:
    • Label — what shows up in the host selector. Free text.
    • User — SSH login (root, ubuntu, ec2-user, etc.).
    • Host — hostname or IP.
    • Port — defaults to 22.
    • Key path — path to your private key file (~/.ssh/id_ed25519). Optional; if empty, Blueprint uses SSH_AUTH_SOCK (your ssh-agent).
    • Roledev / shared / prod. Affects guardrails on destructive actions.
  3. Test connection — sanity check that SSH works.
  4. Save.

The host now appears in the registry. Push-install is the next step (see Push install).

How the connection works

When you click Connect for a host:

  1. Blueprint opens an SSH session using your key.
  2. Reads the svc bearer token from ~/.blueprint/svc-token on the remote (or pulls it from the local OS keychain if cached).
  3. Hits /v1/health through the SSH tunnel to confirm the blueprint-svc is alive.
  4. Caches the connection in memory for subsequent calls.

After connect, the host selector in the top-right title bar lets you switch between "Local" and any connected host. The Dashboard tabs become host-aware — Models, Inference, etc. read from the active host.

How the keychain caching works

First Connect: SSH fetches the token, stores it in the OS keychain (Windows Credential Manager / macOS Keychain / Linux Secret Service) keyed by host ID.

Subsequent Connect: keychain hit skips the SFTP read. If the cached token is rejected (svc rotated the token), Blueprint refetches via SSH and updates the cache. See the keychain doc for the detailed flow.

Common pitfalls

  • "SSH connect succeeded but Connect fails": SSH is fine but the svc isn't installed on the remote. Run Push install.
  • Key with passphrase: Blueprint can't prompt you mid-connect. Either use a passphrase-less key or load it into ssh-agent before launching Blueprint.
  • Firewall on the host blocking outbound HF downloads: hosts behind corporate firewalls need an HTTP proxy set in their env (HF_HUB_USER_AGENT_ORIGIN, https_proxy) before pulls work.
  • Multiple hosts with the same hostname (e.g. EC2 default DNS names): label them distinctively or you'll lose track of which is which in the selector.

Need help with this in production?

Inspire AI Lab runs LLM optimization engagements end-to-end using Blueprint. If you'd rather hand the work to us instead of running it yourself, book a 30-minute review.