LitGPT setup: download, start fine-tuning and serve a model

This tutorial, uploaded on June 3, 2024, walks through LitGPT as a command-line tool for working with a local LLM. No LitGPT release version or separate recording date is supplied, so the installation steps and capabilities describe the upload-era demonstration.

The presenter uses an Ubuntu system and a sponsor-provided VM with a GPU containing 48 GB of VRAM. That is the demonstration hardware, not a stated minimum requirement. He creates a Conda environment, installs LitGPT with pip, lists downloadable models and downloads a Microsoft model for the chat demonstration. He then loads its saved checkpoints and tries interactive prompts.

For fine-tuning, he downloads a dataset and starts a run configured to use a 0.1 fraction of the data. The output includes a LoRA configuration. He interrupts training before completion, then explains how a completed model's output directory would supply the checkpoint for chat. The tutorial therefore does not establish the quality of a finished fine-tune.

The final demonstration serves a checkpoint over HTTP on port 8000. A Python request from another terminal on the same server asks the model to correct a typo. The presenter also discusses network access and pretraining scripts, but does not demonstrate a separate client machine or a pretraining run.