IP-Adapter

IP-Adapter lets you guide Stable Diffusion with a reference image while still using text to describe the result you want. It's for artists and developers who want image references in their local AI workflows. The main code and standard adapter weights use Apache 2.0; the separate FaceID variants are restricted to research use.

The adapter works with Stable Diffusion 1.5 and SDXL, including custom models fine-tuned from the same base. You can use an image alone or combine it with a text prompt, adjusting how closely the output follows the reference. This gives you a way to explore variations without describing every visual detail in words.

Its supported tasks include image variations, image-to-image generation and inpainting guided by a reference. ControlNet and T2I-Adapter integration let you combine that reference with structural guidance. IP-Adapter Plus uses finer visual features, while face-focused variants accept a face image as the prompt.

IP-Adapter is an adapter for an existing diffusion model, so it fits into a generation workflow you already use. Diffusers includes support, and integrations are available for ComfyUI, WebUI and InvokeAI. It also supports safetensors model files and provides training code for developers who want to adapt it to their own datasets.

Reference framing matters: the default CLIP image processor crops images to the center, so square references work best and details near the edges of a non-square image can be lost.