Learning · hand-inked notes

Model Sketchbook

Architecture, drawn the way you study it.

Hand-inked digital notes for the models that matter — one page per shape, blue for explanation, red for what to remember. Built to grow into a shelf of a hundred.

1/ 9pages on the shelf

Ink— structure & boxes

Blue pencil— explanations

Red pencil— what to remember

Transformers

Attention, language models, and the stack that runs them

1/4

2018 · decoder-only

Live

GPT (124M)

Token → embedding → twelve residual blocks → logits. The shape stays [batch, tokens, 768] the whole way up.

GPTdecoderattentionBPE

Open page

2017 · Attention Is All You Need

Coming

Transformer (original)

The 2017 blueprint: QKV attention, residuals, and the feed-forward that widens then shrinks.

Attentionencoder-decoder

2018 · encoder-only

Coming

BERT base

Same transformer block, different job: fill the blanks from both sides.

BERTMLMencoder

2023 · open decoder

Coming

LLaMA block

What changed after GPT-2: norm placement, activation, and how position is woven into Q/K.

LLaMARoPESwiGLU

Vision

Convolutions, residuals, and how images become features

0/2

2015 · residual learning

Coming

ResNet-50

The + that changed everything: identity shortcuts through bottleneck blocks.

CNNresidualImageNet

2015 · biomedical → generative

Coming

U-Net

Contract, expand, and stitch fine detail back in — the backbone of many diffusion UNets.

segmentationskips

Diffusion & generative

Noise in, structure out — generation as a reverse process

0/1

2020 · denoising diffusion

Coming

DDPM (diffusion)

Train a net to undo a little noise. Sample by starting from pure noise and stepping back.

DDPMgenerative

Classic nets

The shapes that taught us how deep learning works

0/1

2013 · latent generative

Coming

Variational Autoencoder

Latent mean and variance, reparameterisation trick, and the ELBO intuition in one page.

VAElatent

Serving & systems

What happens after the forward pass leaves the notebook

0/1

serving · decode path

Coming

KV cache (inference)

Prefill vs decode, what gets cached, and how batching changes the memory story.

inferencelatencymemory