Blueprint

Adaptation

LoRA, QLoRA, SFT, DPO, distillation, continued pre-training.

adaptation·Jun 26, 2026·5 min

QLoRA on a consumer GPU: fitting a 13B fine-tune in 24GB

You don't need an H100 to fine-tune a 13B model. With 4-bit base quantization, paged optimizer state, and gradient checkpointing, a 4090 will train one in an evening. Here's the recipe and the trap doors.

adaptation·Jun 26, 2026·8 min

LoRA from first principles: rank, alpha, and why low-rank works

LoRA freezes the base model and learns two small matrices whose product gets added to the original weights. That's the whole idea. Here's why it works, how to pick rank and alpha, and the failure modes we've hit on real engagements.