AudioLDM 2

AudioLDM 2 is a local AI audio generator for sound designers and researchers experimenting with synthetic sound. It uses pretrained models to produce audio from text and provides a Gradio browser interface alongside command-line access.

• Sound effects and music: Text prompts describe the audio you want, with checkpoints for general sound generation and a dedicated music model.
• Speech generation: Supply the words and a speaker description to generate speech using models trained on GigaSpeech or LJSpeech.
• Audio editing: The repository supports audio super-resolution and inpainting.
• Diffusers integration: Hugging Face Diffusers provides another local implementation that supports audio of arbitrary length and runs faster than the native implementation.

The Python tool runs on CPU, NVIDIA GPUs through CUDA, or Apple Silicon through MPS. MPS computation requires about 20 GB of RAM. Results can vary with hardware and random seed, including output that sounds unusual or has low quality.

Local generation runs on your hardware; the Hugging Face demo is hosted separately. Official model checkpoints are available through the Hugging Face Hub.

AudioLDM 2 is distributed as inference code and downloadable sound, music and speech checkpoints. The repository license is CC BY-NC-SA 4.0. The official AudioLDM 2 model card likewise identifies CC BY-NC-SA 4.0, restricting commercial use.