World Foundation Model
for Robotics
A 1.36B parameter Diffusion Transformer trained on 147K robot video clips, learning to encode robot manipulation into a continuous latent space.
Architecture
Text-conditioned video diffusion using EDM (Elucidated Diffusion Models) with a DiT backbone and NVIDIA Cosmos video tokenizer.
1024-dim
16L, d=2048
Decoder
25f @ 256x256
Training Data Reconstructions
Robot videos encoded through the Cosmos CV8x8x8 tokenizer into a 16-channel latent space, then decoded back. The tokenizer achieves 34-41 dB PSNR — near-lossless reconstruction.
Latent Space Interpolation
Linear interpolation between two robot video latents produces smooth, coherent transitions — demonstrating that the model has learned a structured, continuous representation of robot behavior.
Learned Video Variations (SDEdit)
Starting from a real robot video, we add controlled noise and denoise using the trained DiT — generating plausible variations of the original scene. The model has learned local structure of robot manipulation.
What's Next
Learn to Build This
This World Foundation Model is one of the core topics in our Modern Robot Learning Bootcamp V2. Go from diffusion policy fundamentals to deploying world models on real SO-101 robot hardware.
Join the Bootcamp