TodayTools

Fine-tuning AI models flashcards

Teach a ready model your task, style and format.

Donate
Beginner friendly, starts from zero
Section
How to study
0
Fine-tuning AI modelsStart from zero · 1 of 12

What is fine-tuning, in one plain sentence?

Fine-tuning is giving an AI that already knows a lot some extra lessons from your own examples, so it learns your style or your task.

A trained chef learning your family's recipes.

All topics

Fine-tuning AI models at a glance

Fine-tuning is giving an AI model that already knows a lot extra lessons from your own examples, so it learns your style or task.

Like a new hire who already speaks the language well: you only teach him how your shop does things.

Words you will meet

Model
A program that learned patterns from data and uses them to answer
Prompt
The message you give the model
Token
A small piece of a word; models read and charge in tokens
Weights
The billions of numbers inside a model that training nudges
Base model
A model trained on general text, before your lessons
Training set
The examples the model learns from
Validation set
Examples kept aside to test it on
GPU
A special chip that does training maths fast

The steps

  1. 1. Start from zero (12 cards): What fine-tuning is, the words you need, and your first training example.
  2. 2. Core basics (16 cards): When to fine-tune, data format, epochs, overfitting and memory budgets.
  3. 3. Next steps (16 cards): LoRA and QLoRA settings, SFT with TRL, loss masking and checkpoints.
  4. 4. Go deeper (16 cards): Merging, DPO, GRPO, forgetting, evaluation and shipping adapters.
All 12 cards in Start from zero
What is fine-tuning, in one plain sentence?
Fine-tuning is giving an AI model that already knows a lot some extra lessons from your own examples, so it learns your style or your task.
Before fine-tuning, how did you get an AI to answer your way?
You pasted long instructions and examples into every single message, and it still drifted from your format now and then. Fine-tuning bakes the habit in.
Where might you already meet a fine-tuned AI?
A support chat that always answers in one company's friendly tone, a tool that sorts complaints to the right team, or a writing helper that knows a firm's house style.
What everyday situation is fine-tuning most like?
A new hire who already speaks English well. You spend two weeks showing him how your shop writes receipts and replies; you never teach him the language again.
What do 'AI model' and 'LLM' mean?
A model is a computer program that learned patterns from lots of data and uses them to answer. An LLM (large language model) is a model trained on huge amounts of text, so it can read and write.
What are a 'prompt' and a 'token'?
A prompt is the message you give the model. Models read text in tokens, small pieces of words, and long prompts mean more tokens, which costs more and takes longer.
What are 'training', 'weights' and 'parameters'?
Training is showing a model examples and nudging it each time it gets one wrong. The numbers it nudges are its weights, also called parameters; big models have billions.
What do 'pretraining', 'base model' and 'GPU' mean?
Pretraining is the first huge training run on general text, done once by a big lab. The result is a base model. A GPU is a special chip that does the maths of training fast.
What are a 'dataset', a 'training set', a 'validation set' and an 'eval'?
A dataset is your collection of examples. Most go into the training set to learn from; a few are kept aside as a validation set to test on. An eval is a scoring run on examples the model never trained on.
What do you need before starting this topic?
Nothing to install for the ideas. To do one yourself you need a few hundred good examples and either a cloud fine-tuning service or a computer with a strong GPU; reading a little Python and JSON helps.
Your first tiny win: what is this one training example saying?
It is one example written in JSON, a simple text format for data. The user line is the customer's question; the assistant line is the ideal reply you want the model to learn.
What will you be able to do after this topic?
Decide when fine-tuning beats a better prompt, prepare a clean training file, run a small fine-tune on one GPU, and prove with an eval that it really helped.
Fine-tuning AI models cheatsheet

Choose a method

  • Prompting + examples First try, zero training
  • RAG Fresh or private facts
  • SFT (LoRA) Format, tone, narrow skills
  • QLoRA SFT on one consumer GPU
  • DPO Prefer better over worse answers
  • GRPO / RL Checkable tasks: maths, code, tools
  • Distillation Big-model skill in a small model

LoRA starting values

  • r 16 (8 to 64)
  • lora_alpha r or 2r
  • lora_dropout 0.05
  • target_modules "all-linear"
  • learning_rate 1e-4 to 2e-4
  • epochs 1 to 3
  • full fine-tune lr 1e-5 to 2e-5

TRL config

  • SFTConfig(max_length=2048) Sequence cap
  • assistant_only_loss=True Learn answers only
  • packing=True Fill sequences, less padding
  • DPOConfig(beta=0.1) Distance from reference
  • GRPOConfig(num_generations=8) Samples per prompt
  • peft_config=LoraConfig(...) Train an adapter

Memory savers

  • bf16=True Half-size activations, stable range
  • load_in_4bit (NF4) 4-bit frozen base
  • gradient_checkpointing=True Recompute activations
  • gradient_accumulation_steps Big batch from small ones
  • optim="paged_adamw_8bit" Smaller optimizer states
  • Shorter max_length Activations scale with length

Ship it

  • save_pretrained(adapter_dir) Adapter only, MBs
  • merge_and_unload() One standalone model
  • push_to_hub(repo, private=True) Share with the team
  • ollama create -q q4_K_M Quantise for local use
  • vLLM --enable-lora Serve many adapters on one base
Donate