IP-Adapter

IP-Adapter adds reference-image prompting to Stable Diffusion for artists and developers who want to guide generation with an image rather than describe every detail in words. It works with an existing text-to-image model and can combine image references with text prompts.

• Reference-guided generation: Use an image prompt to create variations, guide image-to-image generation, or direct inpainting with Stable Diffusion 1.5.
• Text and image prompts: Combine a visual reference with written instructions to guide the generated result.
• Structural control: Pair image prompts with ControlNet or T2I-Adapter; SDXL also supports image variations and ControlNet guidance.
• Workflow integrations: Use the adapter through Diffusers or integrations for ComfyUI, WebUI and InvokeAI.

The project uses the Apache 2.0 license. It supports Stable Diffusion 1.5 and SDXL, plus custom models fine-tuned from the same base model. It's an adapter rather than a standalone image-generation app, so it needs a compatible diffusion model and workflow. It supports safetensors weights and includes training code for developers who want to train an adapter on their own dataset.