Favicon of Stable Diffusion Core ML

Stable Diffusion Core ML

An on-device image generation toolkit for macOS, iOS and iPadOS that converts Stable Diffusion models to Core ML and embeds them in Swift apps.

Stable Diffusion Core ML is a developer toolkit for adding local AI image generation to Mac, iPhone, and iPad apps. It pairs model conversion tools with a Swift package for embedding generation in native software.

  • Native app integration: The Swift package runs converted Stable Diffusion models through Apple's Core ML framework.
  • Model conversion: The Python package converts PyTorch models to Core ML and also supports image generation with Hugging Face diffusers.
  • Model choice: Ready-made Core ML weights on Hugging Face Hub include Stable Diffusion 1.4, 1.5, 2.1, XL, and 3 Medium, including an XL base-plus-refiner pipeline.
  • Compressed weights: Quantized and mixed-bit model variants reduce model size for supported deployments, including iOS versions of Stable Diffusion XL.
  • Multilingual prompts: Apple's system text encoder supports multilingual image generation with suitably fine-tuned models; its assets download on first use.

The toolkit is open source under MIT and targets Apple Silicon through Core ML. Models can use GPU or Neural Engine execution depending on the pipeline. Stable Diffusion 3's MMDiT component requires CPU and GPU execution rather than the Neural Engine, and larger conversion resolutions require more memory.

Similar to Stable Diffusion Core ML