mergekit

mergekit combines pretrained language models into one model for developers and researchers who want to combine specialized skills or balance model behaviors. It works directly with model weights, without additional training or access to the original training data.

• Model merging: Combine checkpoints with weighted averaging, SLERP, TIES, DARE or task arithmetic, depending on the relationship between the models and the skills you want to retain.
• Model assembly: Build models from selected layers, combine dense models into a mixture of experts, or chain merges so later stages use earlier results.
• LoRA extraction: Extract PEFT-compatible low-rank approximations from fine-tuned models for use as adapters.
• Tokenizer alignment: Transplant tokenizers between models to align vocabularies, including for draft models used in speculative decoding.
• PyTorch checkpoints: Apply merge algorithms to .pt and .safetensors checkpoints beyond Hugging Face Transformers; this mode doesn't support layer slicing or tokenizer configuration.

The Python toolkit runs locally and is open source under LGPL-3.0. Supported model families include Llama, Mistral, GPT-NeoX and StableLM. Merges can run entirely on CPU or use GPU acceleration with as little as 8 GB of VRAM. It loads weights as needed to reduce memory use when input models don't fit in memory at once.