Skip to main content

Ctrl+K

LLMs from Scratch

Build a Large Language Model (From Scratch)

Setup

Optional Setup Instructions

Chapters

Chapter 1: Understanding Large Language Models
Chapter 2: Working with Text Data
Chapter 3: Coding Attention Mechanisms
Chapter 4: Implementing a GPT Model from Scratch to Generate Text
- Chapter 4: Implementing a GPT Model from Scratch To Generate Text
  - Chapter 4: Implementing a GPT model from Scratch To Generate Text
  - Chapter 4 Exercise solutions
- Chapter 4: Implementing a GPT Model from Scratch To Generate Text
  - FLOPS Analysis
Chapter 5: Pretraining on Unlabeled Data
Chapter 6: Finetuning for Classification
Chapter 7: Finetuning to Follow Instructions

Appendices

Appendix A: Introduction to PyTorch
- Appendix A: Introduction to PyTorch
- Python and Environment Setup Recommendations
Appendix D: Adding Bells and Whistles to the Training Loop
- Appendix D: Adding Bells and Whistles to the Training Loop
Appendix E: Parameter-efficient Finetuning with LoRA
- Appendix E: Parameter-efficient Finetuning with LoRA

Repository
Suggest edit
Open issue

.md

Python and Environment Setup Recommendations

Python and Environment Setup Recommendations#

Please see the README.md in the setup directory for Python installation and setup recommendations.

previous

Exercise A.1

next

Appendix D: Adding Bells and Whistles to the Training Loop

By Sebastian Raschka

© Copyright 2025.