Unsloth: write parametric memory in with a fine-tune config

The Stack

Steps

1. 1

   Write the fine-tune config and validate it

   Define the target model, context length, LoRA rank, and training hyperparameters. CI checks the config parses and all required fields are present. The actual fine-tune (which needs CUDA and your dataset) is fenced.

   cat > finetune-config.json <<'JSON'
   {
     "model_name": "unsloth/Meta-Llama-3.1-8B-bnb-4bit",
     "max_seq_length": 2048,
     "load_in_4bit": true,
     "lora": {
       "r": 16,
       "lora_alpha": 16,
       "lora_dropout": 0,
       "target_modules": ["q_proj", "k_proj", "v_proj", "o_proj"]
     },
     "training": {
       "per_device_train_batch_size": 2,
       "gradient_accumulation_steps": 4,
       "warmup_steps": 5,
       "max_steps": 60,
       "learning_rate": 2e-4,
       "output_dir": "./outputs"
     }
   }
   JSON
   node -e '
   const fs = require("fs");
   const c = JSON.parse(fs.readFileSync("finetune-config.json", "utf8"));
   function bad(m) { console.error("BAD: " + m); process.exit(1); }
   if (!c.model_name || typeof c.model_name !== "string") bad("model_name must be a string");
   if (!c.max_seq_length || typeof c.max_seq_length !== "number") bad("max_seq_length must be a number");
   const lora = c.lora || {};
   if (!lora.r || typeof lora.r !== "number") bad("lora.r (rank) must be a number");
   const train = c.training || {};
   if (!train.learning_rate) bad("training.learning_rate must be set");
   if (!train.output_dir) bad("training.output_dir must be set");
   console.log("config OK: fine-tune " + c.model_name + " at seq=" + c.max_seq_length + " LoRA r=" + lora.r + " lr=" + train.learning_rate);
   '

2. 2

   Run the fine-tune (the GPU step, not checked by CI)

   Install Unsloth (pip install unsloth), load the model with FastLanguageModel.from_pretrained() using your LoRA config, then run SFTTrainer on your dataset. Evaluate on a hold-out set before shipping. Training and quality are fenced.