PaddleNLP
An open-source LLM toolkit built on PaddlePaddle for training, compression and inference on NVIDIA GPUs and other accelerators.
PaddleNLP is a Python toolkit for developers training and deploying language models on their own infrastructure. Built on PaddlePaddle, it covers pretraining, fine-tuning, model compression and inference across several accelerator families.
- Hardware choice: Train and run language models on NVIDIA GPUs, Kunlun XPUs, Ascend NPUs, Enflame GCUs and Hygon DCUs through a shared toolkit interface.
- Pretrained models: Work with LLaMA, Qwen, Mistral, Gemma, Baichuan, Bloom, ChatGLM and OPT, with a common API for loading models and tokenizers.
- Distributed training: Combine data, tensor and pipeline parallelism, and resume saved training checkpoints with a different allocation of machine resources through Unified Checkpoint.
- Fine-tuning and compression: Adapt pretrained models using padding reduction and FlashMask, then use compression and quantization tools for deployment.
- Parallel inference: Run inference with parallel execution and fused operations; the toolkit also supports natural language understanding tasks such as sentiment analysis.
PaddleNLP is open source under Apache 2.0. It supports deployment on your own hardware, including Docker, while AI Studio provides a separate hosted notebook environment for tutorials and examples.
