Audio walkthrough of Chip Huyen's Chapter 7: Finetuning. The hierarchy: prompting → RAG → fine-tuning, in that order. Most engineers reach for fine-tuning too early, before exhausting prompting and RAG — a major interview red flag. Covers: full fine-tuning vs PEFT (parameter-efficient), LoRA / QLoRA and why they work despite updating <1% of weights, instruction tuning vs RLHF vs DPO, the data quality > quantity principle, catastrophic forgetting, before/after eval discipline, when to fine-tune vs RAG vs prompt.
Key interview point: knowing when NOT to fine-tune is as important as knowing how. Vertex AI customers ask for fine-tuning constantly when they don't actually need it — knowing when to push back is the AI engineer's craft.
