Weeks 6–7: NanoGPT On A CPU

Objective

Learn the workflow around GPT-style models without pretending a MacBook Air should train a serious language model from scratch.

Required Videos

• Andrej Karpathy: “Let’s build GPT”
• Optional: a short tokenizer explainer if tokenization feels magical

Tasks

1. Run python prepare_data.py.
2. Inspect the generated train and validation token files.
3. Read config_cpu.md before changing any settings.
4. Use experiment_log.md to plan at least three tiny, fair experiments.

Deliverables

• Prepared data files
• A filled experiment log
• One trusted lesson from comparing runs

Checkpoint Questions

• Why is small data still useful for learning?
• Why do we split train and validation?
• What makes an experiment fair?
• What should never be attempted on a CPU-only laptop in this course?