NanoGPT: train on Shakespeare and inspect the code

This tutorial uses NanoGPT to explain how a local LLM trains and generates text. The speaker walks through Python setup, terminal navigation, a Conda environment and cloning Andrej Karpathy's project. His system has Python 3.13.7, while the training environment uses Python 3.11. He also checks GPU support on a Mac.

The first exercise prepares Shakespeare text and trains a small model for 5,000 iterations. The speaker reports that training took about three hours on his Mac M1 Pro with 32 GB RAM; that is an example runtime, not a hardware requirement. He explains the training log, follows the loss and generates text influenced by the dataset. Tokenization and next-token prediction provide context for a simplified explanation of Transformers, followed by a tour of the model and training code.

For the chat exercise, the tutorial switches to the speaker's NanoGPT2 project and a base GPT-2 model. He says the Shakespeare model is insufficient for this stage. The supplied dataset contains 50 simulated human-assistant conversations, and the fine-tuning run reads it five times. A final question about France's capital produces an uneven response. The speaker cautions against expecting too much from this model. The exercise teaches local AI training and fine-tuning rather than establishing the chatbot's reliability.