BrushNet

BrushNet adds text-guided image inpainting to pretrained diffusion models, letting you fill selected areas of an image while retaining the surrounding content. It's for developers and researchers who want to run image editing on their own hardware and work with an existing model's visual style.

It accepts an image, a mask marking the area to edit, and a text prompt. Its dual-branch design processes the masked image separately from the diffusion model's noisy image representation, giving generation detailed guidance from the original picture. An adjustable control strength changes how closely the result follows that guidance.

BrushNet provides checkpoints for Stable Diffusion 1.5 and Stable Diffusion XL, with separate training for object-shaped segmentation masks and more general, irregular masks. It also works with community models such as DreamShaper, epiCRealism, MeinaMix and Realistic Vision. Examples cover photographs, anime, pencil drawings and watercolor, so the choice of base model matters for the look of the edit.

The Python and PyTorch implementation includes inference, a Gradio demo, and training on custom datasets. BrushData supplies segmentation annotations for training, while BrushBench pairs images with human-annotated masks and captions for evaluation. The pretrained model targets general scenes; specialized uses such as product displays or virtual try-on may require training on your own data.