Skip to main content

Ctrl+K

LLMs from Scratch

Build a Large Language Model (From Scratch)

Setup

Optional Setup Instructions

Chapters

Chapter 1: Understanding Large Language Models
Chapter 2: Working with Text Data
Chapter 3: Coding Attention Mechanisms
Chapter 4: Implementing a GPT Model from Scratch to Generate Text
- Chapter 4: Implementing a GPT Model from Scratch To Generate Text
  - Chapter 4: Implementing a GPT model from Scratch To Generate Text
  - Chapter 4 Exercise solutions
- Chapter 4: Implementing a GPT Model from Scratch To Generate Text
  - FLOPS Analysis
Chapter 5: Pretraining on Unlabeled Data
Chapter 6: Finetuning for Classification
Chapter 7: Finetuning to Follow Instructions

Appendices

Appendix A: Introduction to PyTorch
- Appendix A: Introduction to PyTorch
- Python and Environment Setup Recommendations
Appendix D: Adding Bells and Whistles to the Training Loop
- Appendix D: Adding Bells and Whistles to the Training Loop
Appendix E: Parameter-efficient Finetuning with LoRA
- Appendix E: Parameter-efficient Finetuning with LoRA

Repository
Suggest edit
Open issue

.md

Appendix D: Adding Bells and Whistles to the Training Loop

Appendix D: Adding Bells and Whistles to the Training Loop#

01_main-chapter-code contains the main chapter code.

previous

Python and Environment Setup Recommendations

next

Appendix D: Adding Bells and Whistles to the Training Loop

By Sebastian Raschka

© Copyright 2025.