How LLMs work

Open the box you have been using every day.

1. Tokens and tokenization Explain why an LLM cannot reliably count the letters in a word.
2. Embeddings: meaning as coordinates Describe how similar meanings end up close together in space.
3. Attention and the transformer Explain what 'attention' lets a model do that earlier models could not.
4. Build GPT from scratch Implement a small working GPT and understand every line.
5. Pretraining, fine-tuning and RLHF Name the three stages that turn raw text prediction into an assistant.
6. Why LLMs hallucinate Explain hallucination as a property of the objective, not a bug to be patched.

Ends with a graded checkpoint: 10 questions, pass mark 70%.