Falcon

Falcon-H1 is a family of language models from the Technology Innovation Institute for developers building AI applications on their own hardware. Smaller models suit edge devices, while larger variants target server deployments.

• Language and coding tasks: The models handle instruction following, maths, coding and translation, with training languages including Arabic, English, Hindi, Japanese and Chinese.
• Long context: Context windows reach up to 256K tokens for long documents and extended conversations, with capacity depending on the model.
• Local runtime choices: GGUF and Hugging Face formats support CPU or GPU inference through llama.cpp, Hugging Face Transformers, vLLM and SGLang; the family also supports MLX.
• Application serving: SGLang can expose an OpenAI-compatible endpoint, and OpenWebUI can provide browser chat with a locally served model.
• Fine-tuning: Compatible frameworks include OUMI, Llama-Factory and Unsloth for adapting models to specific tasks.

Falcon-H1 combines attention with Mamba2 state space models in a hybrid architecture designed to reduce memory use and accelerate inference. Hardware needs depend on the selected model and runtime.

Local inference runs on your own CPU or GPU. The hosted chat interface and Hugging Face demo are separate online options.