Tools to Fine-Tune LLMs Locally

Soup is a local LLM fine-tuning tool for developers and teams adapting models to their own datasets. It brings training, evaluation and export into one workflow that runs on your own hardware without requiring a cloud service.

Axolotl is a free, open-source LLM fine-tuning framework for developers and researchers adapting models to their own data. It supports training on local hardware or cloud infrastructure you control.

OpenRLHF is a self-hosted Python framework for researchers and teams training language models with human feedback or custom rewards. It supports fine-tuning and reinforcement learning on your own GPU hardware, including training agents that interact with an environment.

Ludwig is a framework for developers and researchers training custom AI models without writing their own training loops. It handles preprocessing, training and evaluation from a model definition, covering LLMs as well as tabular and multimodal tasks.

FastChat is a self-hosted LLM platform for developers and researchers who want to serve chat models on their own hardware and compare their responses. It powers Chatbot Arena and includes a locally hosted side-by-side comparison interface. The Python code is open source under Apache 2.0; model licenses apply separately, including Llama's license for Vicuna.

GPT-NeoX is EleutherAI's framework for researchers and teams training language models with billions of parameters on their own GPU infrastructure. It focuses on training from scratch, with support for fine-tuning existing checkpoints.

Colossal-AI is a Python framework for developers and researchers training or serving large AI models on their own GPUs. It provides tools to spread workloads across GPUs and work within available memory.

AgileRL is an open-source Python framework for teams training AI agents on their own tasks, including LLM agents and agents for robotics, finance or logistics. Its evolutionary hyperparameter tuning adjusts training settings during a run, reducing the need for separate tuning trials. The framework uses the Apache 2.0 license.

PEFT is a library for developers adapting pretrained AI models on their own hardware. It trains a small portion of a model's parameters to reduce the compute and storage costs of full fine-tuning.

FinGPT is a financial LLM framework for researchers and developers building tools that interpret market news and company information. It includes trained models, financial datasets, and tools for adapting models to specific finance tasks.

Gemma is Google DeepMind's family of open-weight AI models for developers building applications on their own hardware. It includes compact models for mobile and IoT devices and larger models for reasoning on personal computers and servers.

bitsandbytes is a Python library for developers who need to run or fine-tune large language models on their own hardware with less memory. It works within PyTorch workflows.

DB-GPT is a self-hosted AI data assistant for teams analyzing business data and developers building data applications. It connects plain-language questions to database queries and code-based analysis.

Unsloth is a desktop app for people who want to chat with models, train them on their own data, and create media on their own hardware. Its no-code interface brings these tasks into one workspace.

AI Toolkit (ostris) is a training suite for people fine-tuning image and video generation models on their own hardware or rented servers. It provides a browser interface and command-line access to training jobs.

verl is a Python library for researchers and engineers training large language models on their own hardware. It implements HybridFlow for reinforcement learning after initial training, with support for supervised fine-tuning too.

MLX VLM is a Python package for developers and researchers who want to run multimodal models on their own Mac. It supports local inference and fine-tuning through MLX.

MOSS is a conversational language model from Fudan University for developers and researchers who want to run and adapt a local LLM. It includes model weights, dialogue datasets and fine-tuning code.

H2O LLM Studio helps teams adapt language models to their own datasets through a browser interface, without writing training code. It's aimed at organizations building models for specific tasks.

Needle is an AI model for developers building device assistants that turn requests into actions. It prioritizes automation over general chat, with smaller model variants for devices that have limited resources.

ms-swift is a Python framework for developers and researchers training and deploying language and multimodal models on their own hardware. It covers model adaptation through evaluation and serving, with support for Qwen3, DeepSeek-R1, Llama4, Mistral, Qwen3-VL and InternVL3.5.

ClearML is a self-hostable MLOps platform for researchers and engineering teams managing model development and production. It connects experiment records with datasets, training jobs and model deployment so teams can trace results back to the work that produced them.

npcpy is a Python library for developers and researchers building AI agents and applications with language models. It supports local LLM backends as well as cloud providers, so projects can choose where model requests run.

LitGPT is a Python toolkit for developers and researchers who want to train and deploy language models on their own hardware. Its directly editable model implementations suit custom applications and research experiments.

NeMo RL is NVIDIA's Python library for researchers and engineering teams adapting language and multimodal models on their own hardware. It supports experiments on a GPU workstation and distributed training across multiple nodes.

MLX Tune is a Python library for adapting AI models on an Apple Silicon Mac. Formerly unsloth-mlx, it is an independent project and is not affiliated with Unsloth. Its Unsloth-compatible API lets you prototype training locally with minimal changes to existing scripts, then move to NVIDIA hardware with Unsloth. Training runs through Apple's MLX framework and doesn't require a cloud GPU.

MaxText is an LLM training library for researchers and engineers building models from scratch or adapting existing ones. It provides a reference implementation teams can modify for their own model designs, with training that scales from a single host to large clusters.

torchtitan is a training platform for researchers and developers experimenting with generative AI models on their own GPU servers or clusters. Its replaceable components and extension points support work on model architectures and training infrastructure.

MLX LM is a Python package for developers and researchers who want to generate text and adapt language models on Apple Silicon Macs. It uses MLX for local inference and training, with access through Python or a terminal.

SkyRL is a Python library for researchers and developers who want to train LLMs with reinforcement learning on their own hardware. Its Tinker API support lets you run existing Tinker training scripts against a server you control without changing the scripts. It's open source under Apache 2.0.

MiniMind is a toolkit for learners and developers who want to understand language models by training small ones on their own hardware. Its core training algorithms use native PyTorch code that you can inspect and change.

Agent Lightning is a Python framework for developers and researchers training AI agents through reinforcement learning. It keeps the agent's existing tools, context and task environment involved in training, so feedback comes from the workflow the agent actually uses.

TensorZero is a self-hosted platform for developers who need to compare and improve models in LLM applications. The project is archived and no longer maintained.

Intel IPEX-LLM is a library for developers running or fine-tuning language models on Intel hardware. The project is archived and no longer maintained, and Intel has identified known security issues in the code.