ncnn
An AI inference framework for Android, iOS, desktop and browsers, with CPU and Vulkan GPU support and PyTorch and ONNX model conversion.
ncnn is a C++ framework for developers building on-device AI into mobile, desktop and embedded applications. Its focus is running neural networks with a small memory footprint and no third-party runtime dependencies. Models run on the target device's CPU or a supported Vulkan GPU.
It supports Android, iOS, Linux, Windows and macOS, plus browsers through WebAssembly. Embedded targets include Raspberry Pi and NVIDIA Jetson. CPU optimizations include ARM NEON and multicore execution, so GPU acceleration is optional.
The pnnx conversion tool brings PyTorch and ONNX models into ncnn's own format. Compatibility converters also cover older Caffe, MXNet and Darknet models. Developers can work through C++, C or Python interfaces, add custom layers, and use model optimization, fp16 computation and int8 quantized inference to suit constrained hardware.
Computer vision is a major use case, with support for networks such as MobileNet, ResNet and YOLO, alongside segmentation, pose estimation and OCR examples. Community projects extend its use to speech recognition and image generation. The ncnn_llm project provides language model, embedding and vision-language examples; compatibility depends on the operations each model needs.
ncnn is open source under the BSD 3-Clause license. Tencent uses it in applications including WeChat and QQ.
