Intel IPEX-LLM

A local LLM acceleration library for Intel CPUs, GPUs and NPUs, with Ollama and llama.cpp integration; the project is archived.

Intel IPEX-LLM is a library for developers running or fine-tuning language models on Intel hardware. The project is archived and no longer maintained, and Intel has identified known security issues in the code.

Intel hardware support: Run models on Intel CPUs, Core Ultra integrated GPUs and NPUs, or discrete Arc, Flex and Max GPUs, including workloads that use multiple GPUs.
Existing model tools: Use Intel acceleration with Ollama, llama.cpp and HuggingFace Transformers, or serve models through vLLM.
Low-bit inference: Run supported models such as Llama, Mistral, Qwen, DeepSeek and Gemma at reduced precision; FlashMoE supports DeepSeek V3/R1 and Qwen3MoE on Intel Arc GPUs.
Application integrations: Connect local models to LangChain and LlamaIndex applications, with integrations for GraphRAG and RAGFlow; supported vision models include Qwen-VL and MiniCPM-V.

GPU support for the llama.cpp package covers Windows and Linux, while its NPU package runs on Windows. Docker images support GPU inference and model serving on your own hardware. IPEX-LLM is open source under Apache 2.0. Intel no longer accepts patches or provides maintenance, bug fixes or updates.