2,000+ AI tools you can run locally Add tool

GGUF and ggml

An open-source C/C++ machine learning library supports CPU, GPU, NPU and browser backends, with quantized computation under the MIT license.

ggml is a library for developers building machine learning runtimes and local AI applications. It provides tensor computation across different kinds of hardware; GGUF is the model file format covered by the project.

Hardware backends: Build applications that use CPU, GPU or NPU computation, with browser support as well.
Quantization: Use 2- to 8-bit integer representations or MXFP4 and NVFP4 microscaling formats for model computation.
Platform portability: Target x86, ARM, RISC-V, LoongArch, PowerPC and s390x, or use WebAssembly for browser applications.
Optimized computation: Use SIMD kernels on x86, ARM and RISC-V, with no memory allocations during runtime.

The C/C++ implementation has no dependencies and uses the MIT license. It's an open-source development library for integrating machine learning into software, so users need an application built around it to interact with models.

Inference Libraries and Bindings Web Quantization

Similar to GGUF and ggml

llama.cpp

92 videos

An MIT-licensed local LLM engine runs language and vision-language models on CPUs or GPUs, with quantization, hybrid execution and Docker support.

134.9KUpdated 21 hours agoMIT

Web#Code execution#GGUF#Hugging Face integration

llama.cpp is a local LLM inference engine for developers building AI applications or running models on hardware they control. It supports language and vision-language models, with command-line tools and a server API for using the engine directly or connecting other software.

Inference Libraries and BindingsInference Servers and ClustersLocal Model RunnersSelf-Hosted Chat Web UIs

MLX

25 videos

An open-source machine learning array framework with shared CPU and GPU memory on Apple silicon and Linux CPU or CUDA support.

29.2KUpdated 1 week agoMIT

macOS · Linux#Distributed execution#LoRA

MLX is a machine learning array framework for researchers and developers building and testing models on their own hardware. It supports model training and inference, with APIs for numerical work and neural networks.

Inference Libraries and BindingsTraining Frameworks

MLX LM

10 videos

A local LLM Python package runs text generation and fine-tuning on Apple Silicon with MLX, Hugging Face model support and quantization.

7.7KUpdated 1 week agoMIT

macOS#Batch processing#Distributed execution#Hugging Face integration

MLX LM is a Python package for developers and researchers who want to generate text and adapt language models on Apple Silicon Macs. It uses MLX for local inference and training, with access through Python or a terminal.

Inference Libraries and BindingsLLM Fine-TuningLocal Model RunnersQuantization and Model ConversionTerminal Chat Clients

DwarfStar

6 videos

A local LLM engine runs DeepSeek, GLM and Qwen models on Apple Silicon, NVIDIA and AMD hardware, with SSD streaming for models larger than RAM.

24.4KUpdated 2 weeks agoMIT

macOS#Distributed execution#GGUF#Multimodal input

DwarfStar is a local LLM inference engine for people who want to run large DeepSeek, GLM and Qwen models on their own computers or servers. It focuses on selected models and hardware setups, and requires the project's own GGUF files rather than accepting arbitrary GGUF downloads.

Inference Servers and ClustersLocal Model RunnersTerminal Chat ClientsTerminal Coding Agents

Qwen3

36 videos

A local LLM family with public weights, chat and reasoning variants, and CPU or GPU support through Ollama, llama.cpp, and LM Studio.

27.7KUpdated 10 months ago

#Batch processing#GGUF#Hugging Face integration

Qwen3 is a language model family from Alibaba Cloud's Qwen team for people building chat applications or running models on their own hardware. Its public weights include dense and mixture-of-experts models in smaller and larger sizes.

Open-Weight LLMs

LM Studio

91 videos

A local LLM app runs downloaded models for chat and agent tasks, with MLX and llama.cpp support and voice transcription processed on your device.

lmstudio.aiComputer and Browser Agents

macOS · Windows · Linux#llama.cpp backend#MCP#MLX

LM Studio runs downloaded language models for conversations and tasks on your computer. Its Bionic agent helps people create documents, write software and carry out work using local or frontier open models.

Computer and Browser AgentsDesktop Chat AppsLocal Model RunnersPersonal AI AssistantsSpeech-to-Text and Transcription