Chapter 7: Finetuning to Follow Instructions

• create-preference-data-ollama.ipynb: A notebook that creates a synthetic dataset for preference finetuning dataset using Llama 3.1 and Ollama

• dpo-from-scratch.ipynb: This notebook implements Direct Preference Optimization (DPO) for LLM alignment