ncnn
An on-device AI inference framework runs models on mobile, desktop and browser targets, with CPU and Vulkan GPU support.
ncnn is a C++ inference framework for developers adding on-device AI to mobile, desktop and embedded applications. It focuses on a small memory footprint and doesn't require third-party runtime dependencies.
- CPU and GPU inference: Run models on CPUs, including optimized ARM execution, or use Vulkan acceleration on supported platforms.
- Model conversion: Bring PyTorch and ONNX models into ncnn through pnnx, with compatibility converters for older Caffe, MXNet and Darknet models.
- Constrained hardware: Model optimization, fp16 computation and int8 quantized inference help adapt workloads to devices with limited resources.
- Application integration: Use C++, C or Python interfaces and add custom layers when a model needs them.
Supported targets include Android, iOS, Linux, Windows, macOS and browsers through WebAssembly, plus embedded devices such as Raspberry Pi and NVIDIA Jetson. GPU acceleration is optional.
Computer vision workloads include MobileNet, ResNet, YOLO, segmentation, pose estimation and OCR. Community projects also cover image generation and audio; ncnn_llm provides language model, embedding and vision-language examples. Model compatibility depends on which operations ncnn supports.
Logotyper
