Best Open-Weight LLMs to Run Locally
Language models you can download and run, such as Qwen3 and gpt-oss, for chat, reasoning and tool calling.
A family of AI models for self-hosted agents, with open weights and training data, supporting Ollama and llama.cpp on NVIDIA GPUs.
2.3KUpdated 1 week agoApache-2.0
Linux#GGUF#Guardrails#Hugging Face integration
Nemotron is NVIDIA's family of AI models for developers building agents that reason, write code and use tools. Teams can evaluate and adapt the models using published weights, training data and training recipes.
Datasets and Synthetic DataEmbedding and Reranker ModelsLLM Fine-TuningOpen-Weight LLMsVision-Language Models+0
An open-source LLM architecture with constant-memory inference and parallel training, plus Python and PyTorch code under Apache 2.0.
14.8KUpdated 1 week agoApache-2.0
RWKV is a language model architecture for developers and researchers building LLM applications or training models. Its recurrent design keeps inference memory constant as text grows, making it relevant when a growing context cache would strain available memory.
LLM Fine-TuningOpen-Weight LLMs+0
A self-hosted LLM with fast and slow reasoning modes, a 256K context window, and support for Transformers and vLLM.
819Updated 1 year ago
#Quantization#Tool calling
Hunyuan-A13B is Tencent's language model for developers and researchers building self-hosted AI applications. It handles general text and reasoning tasks, with optimization for AI agent workloads.
Open-Weight LLMs+0
An open-source AI model family you can run locally through Ollama or LM Studio, with language, vision and speech models under Apache 2.0.
44Updated 2 months agoApache-2.0
Linux#Guardrails#Hugging Face integration#LM Studio integration
Granite is IBM's family of AI models for developers and businesses building applications on their own hardware or servers. You can customize the models for enterprise tasks, including coding assistants and document analysis.
Coding ModelsEmbedding and Reranker ModelsOpen-Weight LLMsSpeech-to-Text ModelsVision-Language Models+0
An open-source sequence modeling library with reusable neural network blocks, pretrained language models and local inference through PyTorch.
19KUpdated 4 months agoApache-2.0
Linux#Batch processing#Hugging Face integration
Mamba is a Python library for researchers and developers building sequence models, including language models. Its selective state space architecture processes sequences in linear time and is designed for efficient GPU execution.
Inference Libraries and BindingsOpen-Weight LLMs+0
Local LLM families for text generation and reasoning offer downloadable weights, with smaller R1 variants based on Qwen2.5 and Llama.
92KUpdated 2 years agoMIT
#Hugging Face integration
DeepSeek-V3 and R1 are downloadable language models for developers who want to run text generation and reasoning on their own hardware. V3 is a general language model; R1 focuses on problems that require extended reasoning.
Open-Weight LLMs+0
Local LLM models support English, Korean and Spanish, with reasoning and tool use, an on-device variant, and GGUF, GPTQ and AWQ formats.
107Updated 1 year ago
#GGUF#Hugging Face integration#Multilingual
EXAONE is a family of language models from LG AI Research for developers building multilingual AI applications on their own hardware. It combines everyday language tasks with reasoning for harder problems and supports agents that call external tools.
Open-Weight LLMs+0
A local LLM family with downloadable text and vision models, base and chat variants, and custom licensing for research and commercial use.
7.7KUpdated 1 year ago
#Hugging Face integration#Multimodal input#Quantization
Llama is Meta's family of language models for developers, researchers and businesses building AI applications on their own hardware or servers. Downloadable weights let you choose a model for your application and available hardware.
Open-Weight LLMsVision-Language Models+0
An open-source local LLM family for assistants, coding and tool use, with GGUF and MLX downloads under Apache 2.0.
11.9KUpdated 6 days agoApache-2.0
macOS · iOS · Android#GGUF#Hugging Face integration#llama.cpp backend
MiniCPM is a family of compact language models for developers building local assistants and coding agents. It targets on-device workloads and hardware with limited computing resources.
Edge and Embedded AIOpen-Weight LLMs+0
An open source local LLM with 3B parameters, direct and reasoning response modes, multilingual support and a context window of up to 128k tokens.
4KUpdated 1 month agoApache-2.0
#Hugging Face integration#Multilingual#Tool calling
SmolLM3 is a compact language model from Hugging Face for developers building local AI applications and researchers studying model training. It has 3B parameters and comes in base and instruction-tuned variants.
Open-Weight LLMs+0
A self-hosted, open weights LLM with document citations and tool calling, available under the noncommercial CC-BY-NC-4.0 license.
huggingface.coOpen-Weight LLMs
#Guardrails#Hugging Face integration#Multilingual
Command A is an open weights language model from Cohere and Cohere Labs for researchers and developers building document assistants, chatbots and AI agents. It handles text input and output, with a focus on multilingual business tasks.
Open-Weight LLMs+0
A bilingual local LLM family supports English and Chinese chat and fine-tuning, with quantized models compatible with consumer GPUs.
7.8KUpdated 2 years agoApache-2.0
Docker#Hugging Face integration#llama.cpp backend#Multilingual
Yi is a family of language models from 01.AI for developers and researchers who want to run English and Chinese AI on their own hardware. It includes chat models for conversation and base models you can adapt to specific tasks.
Open-Weight LLMs+0
An open-source LLM family for chat, reasoning and coding, with Hugging Face Transformers support and a variant for million-token context.
7.3KUpdated 1 year agoApache-2.0
#Hugging Face integration#Tool calling#Web search
InternLM is a family of downloadable language models for developers and researchers building AI assistants or adapting models to their own applications. It includes base models for further training and chat models tuned to follow instructions.
Open-Weight LLMs+0
An open-source financial LLM framework analyzes news and forecasts stock movements, with local models or cloud inference through OpenAI and MiniMax.
21.6KUpdated 3 weeks agoMIT
Web#Hugging Face integration#LoRA#Multilingual
FinGPT is a financial LLM framework for researchers and developers building tools that interpret market news and company information. It includes trained models, financial datasets, and tools for adapting models to specific finance tasks.
LLM Evaluation and TestingLLM Fine-TuningLLM Libraries and SDKsOpen-Weight LLMs+0
An open-weight local LLM family for phones, PCs and servers, with Ollama and LM Studio support and a JAX library for fine-tuning.
5.8KUpdated 1 month agoApache-2.0
Android#LM Studio integration#LoRA#Multilingual
Gemma is Google DeepMind's family of open-weight AI models for developers building applications on their own hardware. It includes compact models for mobile and IoT devices and larger models for reasoning on personal computers and servers.
Edge and Embedded AIEmbedding and Reranker ModelsLLM Fine-TuningOpen-Weight LLMsVision-Language Models+0
An audio AI model family understands speech and sound, generates spoken replies, and offers Apache 2.0 mini models for local GPU deployment.
1.5KUpdated 8 months agoApache-2.0
Docker · Web#Distributed execution#Hugging Face integration#Multilingual
Step-Audio 2 is an audio AI model family for developers building voice assistants and applications that interpret recordings. It can reason about what people say and how they say it, as well as sounds beyond speech.
Open-Weight LLMsSpeech-to-Text ModelsText-to-Speech ModelsTranslationVoice Assistants+0
Open-weight reasoning models run locally through Ollama, vLLM or Transformers and carry the Apache 2.0 license.
20.5KUpdated 2 weeks agoApache-2.0
macOS · Linux#Hugging Face integration#LM Studio integration#Ollama integration
gpt-oss is a pair of OpenAI reasoning models for developers who want to run a local LLM or serve models on their own hardware. The smaller model targets local and specialized applications; the larger one targets general-purpose tasks that need more reasoning.
Open-Weight LLMs+0
A local LLM family for coding, maths and multilingual tasks, available in GGUF and Hugging Face formats for CPU or GPU inference.
128Updated 1 year ago
macOS#GGUF#Hugging Face integration#llama.cpp backend
Falcon-H1 is a family of language models from the Technology Innovation Institute for developers building AI applications on their own hardware. Smaller models suit edge devices, while larger variants target server deployments.
Open-Weight LLMs+0
A local conversational LLM that supports tool requests, CPU and NVIDIA GPU inference, and quantized models for lower memory use.
12.4KUpdated 2 months agoApache-2.0
Linux · Web#Distributed execution#Hugging Face integration#Multilingual
MOSS is a conversational language model from Fudan University for developers and researchers who want to run and adapt a local LLM. It includes model weights, dialogue datasets and fine-tuning code.
LLM Fine-TuningOpen-Weight LLMs+0
A local LLM family with public weights, chat and reasoning variants, and CPU or GPU support through Ollama, llama.cpp, and LM Studio.
27.7KUpdated 10 months ago
#Batch processing#GGUF#Hugging Face integration
Qwen3 is a language model family from Alibaba Cloud's Qwen team for people building chat applications or running models on their own hardware. Its public weights include dense and mixture-of-experts models in smaller and larger sizes.
Open-Weight LLMs+0
An open-source language model family with downloadable weights for local use, training data, and variants for chat and step-by-step reasoning.
1.8KUpdated 23 hours agoApache-2.0
#Tool calling
OLMo is a language model family from Ai2 for researchers and developers who want to run models on their own hardware or study how they're trained. It provides model weights, datasets, code and reports for examining and adapting the models.
Open-Weight LLMs+0
An on-device AI model turns text into tool calls, extracts structured data and creates local search embeddings on phones, wearables and computers.
14.7KUpdated 3 days agoApache-2.0
macOS · Windows · Linux · iOS · Android · Web#Hugging Face integration#LoRA#Multilingual
Needle is an AI model for developers building device assistants that turn requests into actions. It prioritizes automation over general chat, with smaller model variants for devices that have limited resources.
Edge and Embedded AILLM Fine-TuningLLM Libraries and SDKsOn-Device and In-Browser AIOpen-Weight LLMs+0
A self-hosted LLM handles long documents with a 256K-token context window, supports vLLM and Transformers, and uses the Jamba Open Model License.
huggingface.coOpen-Weight LLMs
#Hugging Face integration#LoRA#Multilingual
Jamba is AI21's language model family for teams building document-based assistants and business applications on their own servers. The Large model targets lengthy context, but its hardware requirements put it in server territory: quantized deployment uses eight GPUs with 80GB of memory each.
Open-Weight LLMs+0
An open-source LLM for self-hosted AI agents and coding, with thinking and direct-response modes, MIT licensing, and vLLM support.
huggingface.coCoding Models
#Hugging Face integration#LoRA#Multilingual
GLM-4.5 is an open-source language model family for developers building AI agents and coding tools on their own servers. It supports reasoning and tool use, but its supplied inference configurations require server-class NVIDIA hardware.
Coding ModelsOpen-Weight LLMs+0
An open-source local LLM for reasoning and math that runs offline, supports fine-tuning, and is available through Ollama and Hugging Face.
3.9KUpdated 2 weeks agoMIT
#Hugging Face integration#Ollama integration#Works offline
Phi-4 is Microsoft's language model for developers building applications that need complex reasoning and math problem solving. It can run on your own hardware without a cloud connection.
Open-Weight LLMs+0
A free, open-source local LLM training toolkit with dense and mixture-of-experts models, Apache 2.0 licensing, and Ollama compatibility.
65.2KUpdated 3 weeks agoApache-2.0
Linux · Web#Distributed execution#Hugging Face integration#llama.cpp backend
MiniMind is a toolkit for learners and developers who want to understand language models by training small ones on their own hardware. Its core training algorithms use native PyTorch code that you can inspect and change.
LLM Fine-TuningOpen-Weight LLMsTraining Frameworks+0
A self-hosted reasoning LLM with a million-token context window and function calling, deployable through vLLM or Transformers.
3.2KUpdated 1 year agoApache-2.0
#Hugging Face integration#Multilingual#Tool calling
MiniMax-M1 is a reasoning model for developers building AI agents or working on coding and mathematical problems with long inputs. You can deploy it on your own infrastructure.
Coding ModelsOpen-Weight LLMs+0
An open-source audio AI model for offline transcription, translation and recording Q&A, with vLLM and Transformers support under Apache 2.0.
10.8KUpdated 5 months agoApache-2.0
#Hugging Face integration#Multilingual#Multimodal input
Voxtral Small is an audio and text model for developers building self-hosted speech applications. The separate mistral-inference library is archived and no longer maintained; this model runs with vLLM or Hugging Face Transformers.
Open-Weight LLMsSpeech-to-Text and TranscriptionSpeech-to-Text ModelsTranslation+0
Open-weight LLMs support self-hosted text generation, with function calling in Mistral Large and hosted inference through an API or cloud providers.
10.8KUpdated 5 months agoApache-2.0
Docker#Hugging Face integration#Multimodal input#Tool calling
Mistral Small and Large are downloadable language models for developers building chat and text applications on their own infrastructure. Their reference library, mistral-inference, is archived and no longer maintained.
Open-Weight LLMsVision-Language Models+0
LogotyperGot an idea for an AI tool? I'm Eugene, the designer and developer behind Localhosted. I'll help you build it, from the first sketch to launch.
Start your projectLogotyperGot an idea for an AI tool? I'm Eugene, the designer and developer behind Localhosted. I'll help you build it, from the first sketch to launch.
Start your project
