Fine-tuning Llama-3.3 8B on a contract-review corpus with QLoRA

QLoRA on a 13B was the headline of a recent post — fitting a 13B fine-tune in 24GB. This walkthrough does the same with Llama-3.3 8B on a more focused task: contract clause classification. The result is a 7B-class model that scores higher on the target task than the off-the-shelf 70B, at 10× lower inference cost.

We run the whole thing through Blueprint's Train card. The Python sidecar handles peft + transformers + trl; the GUI handles config, monitoring, and adapter export.

The dataset

We use a public subset of CUAD — a corpus of commercial contracts annotated for 41 clause types (governing law, indemnification, change of control, etc.). For this walkthrough we narrow to 8 of the most common clause types and 3000 labeled examples.

Data shape (JSONL):

{"text": "This Agreement shall be governed by and construed in accordance with the laws of the State of New York...", "label": "governing_law"}
{"text": "Each party shall indemnify and hold harmless the other party from and against any and all losses...", "label": "indemnification"}

In a real engagement you'd build the equivalent from your client's contract corpus. The structure is the same; only the labels and examples change.

Train card setup

In Blueprint's Train card:

• Base model: Llama-3.3 8B Instruct (we already have it from earlier walkthroughs)
• Dataset: point at the JSONL file
• Method: QLoRA
• LoRA config:
  • rank: 16
  • alpha: 32
  • target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
  • dropout: 0.05
• Training args:
  • epochs: 3
  • per-device batch size: 2
  • gradient accumulation: 8 (effective batch 16)
  • learning rate: 2e-4 cosine
  • max sequence length: 1024
  • warmup ratio: 0.03
  • bf16 mixed precision
  • gradient checkpointing: on
  • optimizer: paged_adamw_8bit

These are the defaults Blueprint suggests for an 8B QLoRA. We've used the same config on a dozen engagement-shaped datasets and they hold up.

Hit Start.

What happens

Under the hood, Blueprint spawns the Python sidecar, which loads:

bnb = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)
model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.3-8B-Instruct",
    quantization_config=bnb,
    device_map="auto",
)
model = prepare_model_for_kbit_training(model)
model = get_peft_model(model, lora_config)
trainer = SFTTrainer(model=model, train_dataset=..., args=...)
trainer.train()

Live training metrics stream to the Train card:

• Step time
• Train loss + eval loss
• Tokens/sec
• VRAM peak

On a 4090 with this config, step time is ~2.4s, effective batch 16, 3000 examples × 3 epochs = ~28 minutes per epoch, ~85 minutes total. Reasonable evening run.

What the loss curve looks like

For a well-configured run on this dataset:

• Epoch 1 train loss: 1.2 → 0.8 (rapid drop)
• Epoch 2 train loss: 0.8 → 0.5 (still moving)
• Epoch 3 train loss: 0.5 → 0.4 (plateauing)
• Eval loss tracks train loss within 0.05 — no overfitting

If eval starts pulling away from train (overfitting), shorten to 2 epochs or raise dropout to 0.1. If both are still moving rapidly at epoch 3, extend to 5 epochs and re-eval.

Export the adapter

When training finishes, Blueprint emits the adapter in two formats:

• adapter.safetensors — for vLLM / TensorRT-LLM
• adapter.gguf — for llama.cpp via --lora-scaled <path> <scale>

The .gguf is ~50MB. The .safetensors is ~200MB. Either is tiny compared to the 8B base.

Evaluation

Blueprint's Eval panel runs the held-out test set (500 examples we held out at the start) through both the base model and the LoRA-adapted model. Each returns a predicted clause label; we score against the reference.

Results on this CUAD subset:

The fine-tuned 8B beats the off-the-shelf 70B on the trained task by 4.3 percentage points. It serves at roughly an order of magnitude lower inference cost. This is the LoRA value proposition in three numbers.

Serving the adapter

Two options:

Option A: Merge the adapter into the base. Blueprint's Train card has a "Merge into base" button. It produces a new full Q4_K_M GGUF that includes the LoRA's effect. Serves identically to a normal model; no special flags. Slightly slower one-time setup, faster inference.

Option B: Keep separate. Pass --lora-scaled adapter.gguf 1.0 to llama-server. The adapter loads at runtime and applies during inference. Slightly slower inference per request, but you can swap adapters per request — useful if you have multiple LoRAs for different tasks.

For a production deployment where the model serves a single workload, merge. For an engagement where you're prototyping multiple adapters, keep separate.

What goes wrong

A few failure modes we've hit on real engagements:

Loss explodes to NaN within 100 steps. Wrong compute dtype. Force bnb_4bit_compute_dtype=torch.bfloat16 on Ampere+ GPUs.

Eval accuracy didn't move. Almost always under-trained. Run more epochs or raise learning rate to 3e-4. Eval set might also be too easy (>90% achievable with prompt engineering alone) — re-bench against the base with a structured prompt first.

Eval accuracy went up, MMLU dropped 5+ points. Over-fitting / format collapse. Lower rank, add general-purpose data to the mix (1-5% of total), or shorten training.

Adapter loads but inference outputs are garbage. Usually a tokenizer mismatch — make sure the inference path uses the same tokenizer as training. Blueprint's Deploy panel pins the tokenizer, so this is rare with Blueprint but common with hand-rolled setups.

What to try next

• Same flow with rank 32 — does the extra capacity help?
• Continued pre-training on raw contract text first, then SFT — does the domain priming improve final accuracy?
• DPO on top of the SFT-tuned model, using pairs of "this clause classification is correct" vs "this one is wrong" — does preference alignment beat pure SFT?

Each is a different button click and a different overnight run. The pattern compounds across engagements.