Granite 4.2 3B vs 8B: local setup and reasoning tests

This video compares IBM's Granite 4.2 3B and 8B through local setup and two practical prompts. The presenter uses Ubuntu on a system with 48 GB of VRAM, downloads the models through Hugging Face, and serves them with vLLM using a reasoning parser plugin supplied with the model card. He describes the models as Apache 2 licensed and reports a 128K context window for 3B.

The training discussion explains a limitation relevant to an AI agent: according to the presenter, 3B skips the agentic training stages that 8B and 30B receive. He nevertheless tests 3B through Hermes Agent, asking for a single HTML file with continent tabs, every country's national drink, and text in native languages. It claims to save the file, but he finds no file. After manually saving the output, he reports incomplete country coverage and tabs that do not work.

The 8B model creates a file and produces a better page, but still omits many countries. Reported memory use is about 4.5 GB for 3B and over 45 GB for 8B in these configurations. The presenter attributes the latter to a larger KV cache; these figures are not minimum hardware requirements.

A final 8B prompt asks it to weigh chicken safety against cost. It selects the option described as 99% spoiled, which the presenter rejects. These examples expose failures in this local LLM test rather than establish general benchmark performance.