SmolVLM

SmolVLM is a compact vision language model for developers building applications that interpret images and answer questions about them. It can run on-device, including on laptops and smaller local setups.

• Image questions and descriptions: Generate descriptions of pictures, answer questions about diagrams, or create stories based on visual content.
• Document reading: Ask for information shown in document images, such as invoice dates, and get text answers.
• Multiple-image conversations: Combine several images with text in one conversation to ask questions about them together.
• Custom training: Use SmolVLM Instruct for interactive applications or SmolVLM-Base as a starting point for fine-tuning; Transformers supports full fine-tuning and optional LoRA or QLoRA, while TRL supports preference tuning.

SmolVLM works with Hugging Face Transformers on CPU or CUDA GPUs. Published inference tests with one or two input images used about 5 GB of GPU memory; that figure describes those tests rather than a fixed requirement for every workload. VLMEvalKit supports evaluation of the model and fine-tuned variants.

The model is open source under Apache 2.0 and permits commercial use. Model checkpoints, training datasets, recipes, and training tools are available under the same license.