IBM Granite 4.1 8B: local setup and coding tests

This tutorial installs IBM Granite 4.1 8B as a local LLM on Ubuntu and tests its responses in a notebook. The presenter uses an Nvidia RTX 6000 with 48 GB of VRAM. He reports a download of about 17.6 GB and GPU memory use just under 19 GB once the model loads. These figures describe his setup rather than a minimum hardware requirement.

The presenter describes the Granite 4.1 family as Apache 2 licensed, with 3B, 8B and 30B options. He cites a 131K-token context window and support for 12 languages, then explains IBM's staged training and response-quality filtering.

The main coding test asks Granite to write a Python file watcher that finds functions and classes missing docstrings through AST parsing. A separate copy of the model runs through Ollama and supplies responses through a local API endpoint. The presenter reports that the generated script detects a changed file and adds docstrings, which he checks in VS Code. He also notes that the code uses an older OpenAI SDK style.

Further tests request a fictional weather report in the supported languages and solve a snail-climbing math problem. The presenter asks native speakers to assess the language output and believes the model's 20-day math answer is correct. RAG and tool calling receive brief positive comments, but he leaves those tests for another video.