DreamDojo
- latent actions
- 2026
- Cosmos-Predict2.5
- World action model
DreamDojo is a controllable world model, not a deployed policy: it learns continuous latent actions from tens of thousands of hours of egocentric human video, then is post-trained on robot data.
Key facts
Rows below are the register record in lib/models.ts. A dash means the value is not confirmed from a primary source. Classification follows Reuss, NVIDIA Technical Blog (15 June 2026) and the model's own paper.
| Name | DreamDojo |
|---|---|
| Lab | — |
| Year | 2026 |
| Paradigm | latent actions |
| Backbone | Cosmos-Predict2.5 |
| Source | https://arxiv.org/abs/2602.06949 |
What the paper claims
- The authors pretrain on DreamDojo-HV, which they describe as 44,000 hours of egocentric human video, using continuous latent actions as a unified proxy because most of that video has no robot action labels. Gao et al., 2026
- The world model is built on pretrained Cosmos-Predict2.5. After post-training on small-scale target-robot data, a Self Forcing distillation pipeline is reported to run at 10.81 frames per second at 640×480. Gao et al., 2026
- They list downstream uses of the world model: live teleoperation, policy evaluation without real-world deployment, and model-based planning. Gao et al., 2026
What it evaluates
- They report systematic evaluation on multiple out-of-distribution benchmarks for open-world, contact-rich simulation, including zero-shot generalization to unseen objects and novel environments after robot post-training. Gao et al., 2026
On this register
Latent actions. Learns actions as hidden codes from video. That axis is defined in what is a world action model? and in Reuss, NVIDIA Technical Blog (15 June 2026).
Other latent actions models on the register: Being-H0.7.
No other register row currently lists the backbone Cosmos-Predict2.5. Matching is exact; nearby names such as Wan 2.1 versus Wan 2.2 are kept separate.
Limits the authors state
- The authors say simulation of uncommon actions such as slapping and fast waving remains imperfect; absolute success rates inside DreamDojo are often higher than in the real world, so failure modes are not faithfully generated; the model does not naturally support multi-view simulation; and inference speed still has headroom. Gao et al., 2026