Stable Diffusion Core ML

Stable Diffusion Core ML is an MIT-licensed, open-source toolkit for developers adding on-device image generation to Mac, iPhone and iPad apps. It runs diffusion models through Apple's Core ML framework, with support for Apple Silicon GPUs and the Neural Engine. Image generation runs locally.

The project pairs a Python package with a Swift package. Python converts PyTorch models into Core ML format and generates images with Hugging Face diffusers. The Swift package brings those converted models into native apps. It's aimed at developers who need an image generation component they can embed in their own software.

Supported models include Stable Diffusion 1.4, 1.5 and 2.1, along with Stable Diffusion XL and Stable Diffusion 3 Medium. Hugging Face Hub provides ready-made Core ML weights, including compressed variants and an SDXL base-plus-refiner pipeline. Developers can also convert other Stable Diffusion models.

Weight compression reduces model size and memory use, which helps larger models fit on mobile devices. Mixed-bit compression adjusts precision across layers to balance size against changes in model output. Hardware support starts with M1 Macs and iPads, and A14 iPhones; Stable Diffusion 3 uses CPU and GPU execution rather than the Neural Engine.

The toolkit also supports Apple's system multilingual text encoder for image generation with suitably fine-tuned models. That workflow requires downloading the text encoder's assets on first use.