UniPi
- inverse dynamics
- 2023
- Video diffusion (~1.7B)
- World action model
UniPi treats a robot policy as a text-conditioned video generator: it first draws the future, then reads the actions out of that video.
Key facts
Rows below are the register record in lib/models.ts. A dash means the value is not confirmed from a primary source. Classification follows Reuss, NVIDIA Technical Blog (15 June 2026) and the model's own paper.
| Name | UniPi |
|---|---|
| Lab | — |
| Year | 2023 |
| Paradigm | inverse dynamics |
| Backbone | Video diffusion (~1.7B) |
| Source | https://proceedings.neurips.cc/paper_files/paper/2023/file/1d5b9233ad716a43be5c0d3023cb82d0-Paper-Conference.pdf |
What the paper claims
- The paper casts sequential decision making as text-conditioned video generation. A planner synthesizes future frames for a language-specified goal, then an inverse-dynamics model extracts the control actions from those frames. Du et al., NeurIPS 2023
- The authors argue that video is a shared interface across environments with different state and action spaces, and that language is a shared interface for task specification. Du et al., NeurIPS 2023
- They instantiate the planner as a video diffusion model and say the same formulation can transfer knowledge from internet videos into realistic robot-motion plans. Du et al., NeurIPS 2023
- Reuss's field map files UniPi as the early inverse-dynamics recipe: generate a future video plan, then recover actions, using a CNN video-diffusion stack of about 1.7 billion parameters trained from scratch. Reuss, NVIDIA Technical Blog (15 June 2026)
What it evaluates
- The paper reports combinatorial generalization to novel geometric relations between objects, multi-task learning from language-conditioned goals, and hierarchical, steerable video plans. Du et al., NeurIPS 2023
- It also reports internet-scale knowledge transfer: a video generator pretrained on large-scale text-video data synthesizing robot motion videos from language instructions. Du et al., NeurIPS 2023
On this register
Inverse dynamics. Generates the future video, then infers the actions that would produce it. That axis is defined in what is a world action model? and in Reuss, NVIDIA Technical Blog (15 June 2026).
Other inverse dynamics models on the register: Video Prediction Policy, mimic-video, LingBot-VA, DVA.
No other register row currently lists the backbone Video diffusion (~1.7B). Matching is exact; nearby names such as Wan 2.1 versus Wan 2.2 are kept separate.