SpecForge

SpecForge is an open-source framework for teams training draft models to speed up LLM inference in SGLang. It suits developers who want to train on their own hardware and use the resulting drafts directly with SGLang serving.

• SGLang compatibility: Export trained draft models for SGLang, with MTP drafts merged into the base model.
• Draft model methods: Train EAGLE3, P-EAGLE, DFlash, DFlash2, DSpark, Domino and MTP drafts through a shared runtime with common data handling, evaluation and export.
• Live or precollected training data: Train with target-model hidden states captured during training or collected beforehand, with the target model and trainer sharing hardware or running on separate infrastructure where the method supports it.
• Ready-made drafts: Download SpecBundle models from Hugging Face for LLM families including Llama, Qwen, DeepSeek and gpt-oss, with SGLang serving settings and benchmark results.

SpecForge supports NVIDIA GPUs through CUDA, AMD GPUs through ROCm and Ascend NPUs. Training can scale from a single GPU to multiple servers. Supported training arrangements differ by draft method, so each method doesn't support every hardware layout or data-capture mode. The Python framework uses PyTorch and carries the MIT license. SpecBundle's performance dashboard compares draft acceptance length and inference speedup across benchmarks.