FastVLM

FastVLM is a family of local vision-language models for developers and researchers building applications that answer questions about images. It focuses on reducing the delay before an answer starts, especially with high-resolution images.

• Image understanding: FastViTHD, its vision encoder, processes images with fewer tokens to reduce image encoding time before the language model responds.
• Model sizes: FastVLM-0.5B, FastVLM-1.5B and FastVLM-7B provide smaller and larger model choices, with pretrained PyTorch checkpoints available for each.
• Apple device inference: The project includes export code for running models on Apple Silicon and a demo app for on-device use on iPhone, iPad and Mac.
• Training and fine-tuning: Developers can train or adapt FastVLM variants through the LLaVA codebase.

The project's evaluations report faster first responses than LLaVA-OneVision-0.5B for the smallest variant, alongside a smaller vision encoder. Larger variants use Qwen2-7B and a single image encoder, with faster first responses than Cambrian-1-8B in those evaluations.

FastVLM is a model development project with inference examples and a device demo. Apple Silicon inference requires exporting the PyTorch checkpoints to a compatible format.