Nemotron 3 Ultra: architecture and training explained

NVIDIA's researchers discuss how Nemotron 3 Ultra was trained and why they prioritized inference speed for AI agent workloads. This is a technical Q&A rather than a deployment demo or a guide to running a local LLM on consumer hardware.

The main topic is multi-teacher on-policy distillation, or MOPD. The team describes training specialist teachers in parallel, combining their capabilities into one student model, then repeating the process after identifying weaknesses. They distinguish this approach from weight merging: teacher feedback applies to prompts in the relevant domain. At the time of the stream, the researchers planned to release the teachers but still had safety evaluations to complete.

The discussion also covers asynchronous reinforcement learning, synthetic prompts and reward profiling. Researchers explain how prompt pass rates help select a training curriculum and how targeted data can address skipped verification steps or tool-use mistakes.

For inference speed, the team credits the Mamba 2 hybrid architecture, multi-token prediction and limits on reasoning length. The host points to Hugging Face model availability, cloud deployment options and GitHub adaptation recipes. The session does not establish minimum hardware requirements for self-hosted use.