TodayTools

Build your own AI model flashcards

From tensors to a small transformer you trained yourself.

Donate
Beginner friendly, starts from zero
Section
How to study
0
Build your own AI modelStart from zero · 1 of 12

What is an AI model, in plain words?

A program that learned a skill from many examples instead of being given rules by a person. Show it thousands of cat photos and it learns to spot cats.

Rules written by people versus skills learned from examples.

All topics

Build your own AI model at a glance

An AI model is a program that learns a skill from many examples instead of following rules a person wrote.

Like a student who does practice sums with an answer key, checks each answer and slowly gets better.

Words you will meet

Weights
The many adjustable numbers inside a model that decide its answer
Training
The practice loop that adjusts the weights using examples
Loss
One number that says how wrong the model was; lower is better
Tensor
A grid of numbers, like a list, a table or a stack of tables
Layer
One step of maths inside the model; models stack many
Batch
A small group of examples handled at once
Epoch
One full pass through all your training data
GPU
A chip that does thousands of small sums at once, making training faster

The steps

  1. 1. Start from zero (12 cards): What a model is, the key words in plain English, and your first tensor.
  2. 2. Core basics (16 cards): Tensors, loss, gradients, optimizers and your first network.
  3. 3. Next steps (16 cards): The training loop, data loading, embeddings, attention and sanity checks.
  4. 4. Go deeper (16 cards): Tokenizers, schedules, mixed precision, scale, checkpoints and modern blocks.
All 12 cards in Start from zero
What is an AI model, in plain words?
A program that learned a skill from many examples instead of being given rules by a person. Show it thousands of cat photos and it learns to spot cats.
How did computers do these jobs before AI models?
People wrote every rule by hand, like 'if the email says FREE MONEY, mark it spam'. Real life has too many cases, so hand rules kept missing things.
Where do you already meet AI models every day?
The next-word suggestions on your phone keyboard, photo search for 'beach', the spam folder in your email and the voice that answers when you talk to your phone.
What is an everyday picture of how a model learns?
A student doing practice sums with an answer key. He guesses, checks the key, sees how far off he was and adjusts. After many sums he gets most right.
What are inputs, outputs and weights?
The input is what goes in, like a photo. The output is the answer that comes out, like 'cat'. Weights are the many adjustable numbers inside that decide the answer.
What do training, data and a class mean?
Data is the pile of examples you learn from. Training is the practice loop that adjusts the weights using that data. A class is one possible answer group, like cat, dog or bird.
What is a loss, and how is it like a test score?
The loss is one number that says how wrong the model was. Low loss means good answers. Training keeps nudging the weights to make the loss smaller.
What is a tensor?
A grid of numbers. One row is a list, rows and columns make a table, and a photo is a stack of tables (red, green and blue). Models only ever see tensors.
What are a layer, a batch and an epoch?
A layer is one step of maths inside the model; models stack many. A batch is a small group of examples handled at once. An epoch is one full pass through all your data.
What are a CPU and a GPU, and why do people talk about GPUs?
The CPU is the main chip in a computer and does one job at a time very well. A GPU is a graphics chip that does thousands of small sums at once, which is what training needs.
What do you need before starting, and what is a first tiny win?
Python and a free library called PyTorch; a normal laptop is enough. Two lines make your first tensor and print its shape, which is its size in each direction.
What will you be able to do after this topic?
Turn data into tensors, build a small model from layers, train it until the loss drops, check it on new data and save it to use again.
Build your own AI model cheatsheet

Tensors

  • torch.tensor(data) Tensor from Python data
  • torch.randn(B, T, D) Random normal tensor
  • x.shape, x.dtype, x.device The three debug checks
  • x.to(device) Move to GPU or CPU
  • x.view(B, -1) / x.reshape(...) Change shape
  • x.transpose(1, 2) Swap two dimensions
  • x @ w Matrix multiply

Building blocks

  • nn.Linear(in, out) Weights times input plus bias
  • nn.Embedding(vocab, dim) Token id to vector
  • nn.ReLU() / nn.GELU() Nonlinearity
  • nn.LayerNorm(dim) / nn.RMSNorm(dim) Normalise activations
  • nn.Dropout(0.1) Regularise during training
  • F.scaled_dot_product_attention(q, k, v, is_causal=True) Fused attention

Training step

  • model.train() Training mode
  • loss = loss_fn(model(x), y) Forward and loss
  • optimizer.zero_grad() Clear old gradients
  • loss.backward() Compute gradients
  • clip_grad_norm_(params, 1.0) Cap gradient size
  • optimizer.step(); scheduler.step() Update weights and LR
  • model.eval(); torch.no_grad() Evaluation mode

Losses and optimizers

  • nn.CrossEntropyLoss() Classes and next token, takes logits
  • nn.MSELoss() / nn.L1Loss() Regression
  • nn.BCEWithLogitsLoss() Yes/no or multi-label
  • AdamW(lr=3e-4, weight_decay=0.01) Default optimizer
  • Warmup + cosine decay Default LR schedule

Sanity numbers

  • Step-0 loss about ln(classes or vocab)
  • Perplexity exp(loss per token)
  • Chinchilla data about 20 tokens per parameter
  • Training FLOPs about 6 x params x tokens
  • Adam memory about 16 bytes per parameter

Save and ship

  • torch.save(model.state_dict(), path) Weights only
  • torch.load(path, map_location="cpu") weights_only=True by default
  • safetensors.torch.save_file(sd, path) Safe, fast weight file
  • torch.onnx.export(model, args, path) Portable model for apps
  • torch.compile(model) Faster training and inference
Donate