Nunchaku
An open-source diffusion inference engine that runs 4-bit models such as FLUX locally on NVIDIA GPUs, with ComfyUI integration and Apache 2.0 licensing.
Nunchaku is a local diffusion inference engine for developers and image-generation users who want to run models such as FLUX with less GPU memory. It focuses on faster generation with compressed models on NVIDIA hardware.
- Lower memory use: SVDQuant compresses model weights and activations to 4 bits while aiming to retain visual fidelity, reducing the memory needed for FLUX.1.
- Faster generation: The engine reduces computation overhead, and lower memory use can avoid CPU offloading when a model fits on the GPU.
- Generation interfaces: A ComfyUI plugin brings Nunchaku into visual workflows, while Gradio demo apps provide another way to try the engine.
- Custom model quantization: The related DeepCompressor library provides tools for preparing custom quantized models for use with the engine.
Reported performance results cover NVIDIA RTX 4090 desktop and laptop GPUs and RTX 5090 hardware. The project also includes visual-quality comparisons for PixArt-∑. Speed and memory gains depend on the model, precision and GPU used in those comparisons.
Nunchaku is open source under Apache 2.0 and runs inference on your own GPU. Nunchux also offers a separate hosted API for image and video generation. Requests to that service run on Nunchux infrastructure.
Logotyper
