NVIDIA Cosmos
A physical AI platform with downloadable world models for NVIDIA hardware, synthetic video generation, robot policy learning, and hosted browser access.
NVIDIA Cosmos is a platform of world models for developers building robots, autonomous vehicles, and industrial vision systems. Its downloadable models run on your own NVIDIA hardware; NVIDIA also offers a hosted catalog for browser access. Cosmos combines visual reasoning, simulation, and action generation in a shared Mixture-of-Transformers architecture.
The models can interpret objects, interactions, and intent in images or video, and generate possible future scenes from text, visual input, sound, and actions. Developers can use those outputs as synthetic training data or compare behaviors in physics-grounded simulations before testing them in the real world. Robot policy learning supports adaptation to specific tasks, camera layouts, and robot embodiments. Video analysis covers live feeds and recorded footage, including contextual alerts, captioning, and search through NVIDIA Metropolis Blueprint.
Hardware needs vary by model. Cosmos3-Edge supports Jetson AGX Orin and Thor, while Nano and Super target workstation or data-center GPUs such as RTX PRO 6000, H100, H200, and B200. Supported backends include Diffusers, Transformers, vLLM, TensorRT-LLM, and NIM, with OpenAI-compatible API serving available.
Cosmos Framework provides training and optimization tools. Cosmos Curator filters, annotates, and deduplicates sensor data, while Cosmos Evaluator scores generated outputs. The video quickstart requires approved Hugging Face access to a guardrail model. Generation uses guardrails by default, but outputs can still contain inconsistent motion, changing object shapes, or implausible physics; safety-critical applications need additional validation.
