LM Studio setup: models, memory and offline chat

This beginner tutorial explains how to run models locally with LM Studio on a Mac or PC. The speaker compares its chat interface with ChatGPT, then walks through choosing a model in Discover, downloading its weights, loading it into memory and sending a prompt.

The explanation separates training from inference and describes how parameter counts and quantization affect hardware needs. Q4 generally uses less memory, while Q8 retains more precision. Storage capacity alone does not determine whether a model will run: the walkthrough distinguishes GPU VRAM on Windows PCs from shared CPU and GPU memory on Apple Silicon Macs. Model size, context length and additional runtime memory all affect what fits.

In the demo, the speaker turns off Wi-Fi and continues chatting with the downloaded model. This presents an offline ChatGPT alternative, with inference taking place on the computer. The speaker also discusses document summaries and questions about PDFs, while noting that these uses need the right local setup.

For slow responses, the tutorial suggests trying a smaller model or stronger quantization. It recommends leaving most advanced settings at their defaults initially and explains why longer context needs more memory. The speaker describes repeated local inference as avoiding provider message caps and per-token charges; hardware still limits speed.