Fine-Tune an Open LLM to Make It Yours
Adapt an open-weight model to your domain with a small dataset.
Run this workflow
See exactly what it produces before you build it.
Build this with your agent
One copy-paste hands Claude Code, Codex, or Cursor the full recipe, steps included, nothing to fetch.
Intended Use
Teams with a domain-specific task where the base model is close but not quite right and a few hundred examples exist.
Not for
- General-knowledge improvement (use a better base model)
- Tasks without a clear input→output structure
The Stack
Tested Against
vllm@0.6axolotl@0.4Side effects & data flow
- Network
- huggingface.co
- Writes
- ./fine-tune-output/, ~/.cache/huggingface/
- Credentials
- HF_TOKEN
Prerequisites
- A GPU or rented cloud GPU
- A few hundred examples
Steps
- 1
Prepare data + LoRA
Curate instruction pairs and run a LoRA fine-tune, then serve with vLLM.
Eval, 1 fixture
Last passed: verified 1mo agodomain-improvementrubrictimeout 600s · max $0.5Judge: claude-sonnet-4-5 Rubric: Pass if the fine-tuned model's accuracy on the held-out eval is at least 10 percentage points higher than the base model on the same set.
Results
137K views, 1.1K shares, top guide on the sub.
Did this work for you?
Our CI checks the setup runs. You tell us if the whole thing worked. Tell us straight.
Related workflows
- Validate a WrenAI semantic model's references before an agent queries through it
- Wire the DeepWiki MCP into your agent so it looks up repos instead of hallucinating
- Verify an agent-skills plugin before you ship or install it
- WebMCP: declare a site's agent tools, and gate the ones that spend money
- Wire GLM-5.2 into Hermes: valid route, 64k-context check, no key in config
- Voyager pattern: validate a procedural skill store before you trust a saved skill
Liked this workflow?
Get new verified workflows in WebAfterAI, three issues a week (Tue, Thu, Sat).