AnimateDiff
An open-source AI animation module for Stable Diffusion that runs locally, works with personalized models, and supports image or sketch guidance.
AnimateDiff adds text-driven animation to personalized Stable Diffusion models without requiring separate training for each model. It's for artists and developers who want to generate motion while keeping the visual style of a chosen image model. The Python implementation runs locally and includes a Gradio browser interface.
Its reusable motion module works with models based on Stable Diffusion 1.5, including customizations made with LoRA and DreamBooth. Examples include ToonYou, Realistic Vision and majicMIX Realistic. This lets you choose a model for its appearance and use the motion module to generate animated output. AnimateDiff also works through Hugging Face Diffusers.
For more control over the result, MotionLoRA supports camera movements such as zooms, pans, tilts and rotation. SparseCtrl accepts RGB images or sketches to guide animation content, so a text prompt isn't the only way to shape a scene. For image animation and interpolation, images generated by the same community model are recommended to keep the style consistent.
The project code and released motion modules use Apache 2.0. Base Stable Diffusion weights and personalized checkpoints have their own model terms, which also apply when generating animations. Its focus on compatibility with community image models comes with limits: output can show flicker, and general text-to-video quality is limited because the approach isn't specifically optimized for that use.
