Tools to Fine-Tune LLMs Locally

text-generation-webui, also called TextGen, runs language models on your own hardware for chat, writing and application integrations. You can use it as a desktop app or a self-hosted browser interface.

Nemotron is NVIDIA's family of AI models for developers building agents that reason, write code and use tools. Teams can evaluate and adapt the models using published weights, training data and training recipes.

HarnessX is a Python framework for developers building AI agents whose behavior they can reuse and adapt. It separates model choice from tool use, memory and context handling, so changing an agent's role doesn't require rewriting it.

Kubeflow is a self-hosted AI platform for teams developing and running machine learning workloads on Kubernetes. Its modular projects cover interactive development, training and the work of moving models into production.

RWKV is a language model architecture for developers and researchers building LLM applications or training models. Its recurrent design keeps inference memory constant as text grows, making it relevant when a growing context cache would strain available memory.

Kaito manages LLM inference, fine-tuning and document retrieval services in Kubernetes. It's for teams that want to run models in their own cluster with less manual work sizing GPU capacity and managing deployments.

StarCoder2 is a family of code generation models for developers who want code completion on their own hardware. These models predict continuations of code; they aren't instruction models, so conversational requests aren't their intended use.

KTransformers is a research framework for developers running or fine-tuning large language models on their own hardware. It focuses on mixture-of-experts (MoE) models, using CPU resources alongside GPUs to work within limited GPU memory.

AutoTrain supports model training on your own hardware through AutoTrain Advanced, a no-code application for people fine-tuning LLMs or building models from their data. Advanced can run locally or on cloud infrastructure. The project is no longer maintained, and it won't receive bug fixes or new features.

DataDreamer is a Python library for researchers and developers who need to create training datasets and adapt LLMs to specific tasks. It connects data preparation and model training in reproducible workflows.

Liger Kernel is a Python library for developers training or fine-tuning LLMs on their own hardware. Fused kernels reduce memory use during model operations and post-training.

TRL is a Python library for researchers and developers adapting foundation models with their own datasets. It builds on Hugging Face Transformers and supports training on a single GPU or across multiple machines.

DSPy is a Python framework for developers building language model applications with local backends such as Ollama on a laptop or SGLang on a GPU server. It connects application code to those model servers, which handle inference on your hardware. It also supports cloud providers, so where model requests go depends on the backend you choose.

Oumi helps ML researchers and teams build specialized AI models for their own tasks. It brings data preparation, training, evaluation, and deployment into one platform, with reusable recipes and components you can customize.

HQQ is a Python library for developers compressing language and vision models to fit the memory available on their own hardware. Its Half-Quadratic Quantization method doesn't require a calibration dataset.

Reef is self-hosted infrastructure for developers who want AI agents to learn from their interactions. It connects live requests and feedback to changes in model weights or the prompts, rules, and skills that guide an agent.

Second Me is an open-source prototype for people who want to train and host an AI representation of themselves. It uses personal memories to shape a model around their identity and context.

PaddleNLP is a Python toolkit for developers training and deploying language models on their own infrastructure. Built on PaddlePaddle, it covers pretraining, fine-tuning, model compression and inference across several accelerator families.

Augmentoolkit turns documents into datasets for training an LLM on a particular subject. It's for people building custom models around material such as research papers or fictional lore.

AReaL is open-source reinforcement learning infrastructure for researchers and developers training LLMs and AI agents on their own compute. Its asynchronous training approach targets large reasoning models and agent applications, with support for single-node runs and distributed training. It uses the Apache 2.0 license.

OpenPipe ART is a Python framework for developers who want to train multi-step AI agents on the tasks their applications actually perform. It uses reinforcement learning through GRPO (Group Relative Policy Optimization) to improve model behavior based on experience and task rewards. It's open source under Apache 2.0.

NeMo Automodel is a Python library for researchers and developers who want to train or adapt AI models on their own GPU workstations or clusters. Its PyTorch-based training works with Hugging Face models without converting checkpoints.

FastVLM is a family of local vision-language models for developers and researchers building applications that answer questions about images. It focuses on reducing the delay before an answer starts, especially with high-resolution images.

LazyLLM is a Python framework for developers building applications with multiple AI agents. It brings application workflows, model training and evaluation together so teams can refine a prototype using task-specific data.

Open Instruct is AllenAI's toolkit for researchers and developers adapting pretrained language models with public datasets. It includes the training recipes behind TÜLU and downloadable checkpoints for comparing training stages.

LLaMA-Factory is a local model training platform for developers and researchers who want to adapt pretrained models to their own tasks. Its web UI lets you fine-tune models without writing code.

Kiln is a desktop workbench for teams building and testing AI applications. Engineers, QA staff and subject matter experts can work with the same tasks and datasets, including contributing ratings and examples without writing code.

Ferret is a local vision-language model for researchers studying how AI understands specific parts of an image. It supports questions about selected regions and can identify image regions in its answers. Its code and data are restricted to research use.

torchtune is a Python library for developers and researchers who want to adapt LLMs on their own hardware with editable PyTorch training code. The project is no longer actively maintained.

LLaVA is a vision-language model project for researchers and developers who want to ask questions about images on their own hardware.

nanochat is a toolkit for researchers and developers who want to train an LLM and inspect the pipeline themselves. Its small Python codebase covers the stages between preparing a tokenizer and chatting with the resulting model.

DistillKit is a Python toolkit for developers and researchers training smaller language models to learn from larger ones. It addresses the memory and storage demands of distillation, particularly when you want to reuse one teacher's outputs across several student models.

RD Agent is a self-hosted AI research agent for data scientists, researchers and quantitative developers. It runs experiments on a Linux machine or server, while chat and embedding requests use providers through LiteLLM, including OpenAI and Azure OpenAI. Those remote configurations require API credentials and internet access; hosting the agent yourself doesn't move their inference onto your hardware.

XTuner is an LLM training engine for researchers working with very large mixture-of-experts (MoE) models. Its focus is reducing memory pressure and improving distributed training efficiency when model experts receive uneven workloads.