World models are becoming a foundational technology in artificial intelligence, even if most people never interact with them directly. These systems use visual inputs to generate 3D environments that simulate real-world physics, and they are already being used to train robots and autonomous vehicles. The work is largely invisible, but its effects are beginning to reach the physical world.
From Pokémon Go scans to robot training
Niantic, the company behind Pokémon Go, sold the game last year and pivoted toward world models. Brian McClendon, chief technology officer of Niantic Spatial, says the company is focused on reconstructing the real world. Its trove of environmental scans, including data collected from players, is being used to build digital replicas. McClendon describes a process where someone can photograph a space where a robot is meant to operate, generate a 3D model, and then run millions of simulations to train the robot on that environment’s parameters. The robot practices without moving through the physical space.
Teaching machines to predict what happens next
Oliver Cameron, CEO of the AI lab Odyssey, explains the difference between language models and world models by analogy: a language model predicts the next word in a sentence, while a world model takes an of the world and predicts the next visual state. These generated 3D scenes serve as digital training grounds that look and feel real. For robots, that means practicing tasks and navigation in simulated environments before operating in actual homes or workplaces.
Driverless cars rehearse dangerous roads
Autonomous vehicles are another major application. Jamie Shotton, chief scientist at Wayve, says a world model must react to actions taken in the world, which requires an understanding of physics, cause and effect, and how other objects and people respond. Wayve’s generative world model is powered in part by dashcam footage, and it is used to recreate hazardous roads and unsafe conditions that would be impractical to test in real life. The goal is to let software experience split-second scenarios without putting people at risk.
Investors and researchers pile in
The technology has attracted significant attention and money. Tencent is investing in world models for video games, where physics and 3D rendering are already central. Google is expected to continue touting its own improvements. Yann LeCun, Meta’s former chief AI scientist, recently raised over a billion dollars for his startup Advanced Machines Intelligence. Stanford computer scientist Fei-Fei Li launched World Labs in 2024, and AMD acquired the company this year for more than $8 billion.
A new kind of gig work
Building these simulations requires real-world data, and that demand is creating work for people who record their daily activities. In one example, a person strapped an iPhone to their forehead and recorded hours of video at home—folding laundry, washing dishes, and pouring a drink—then sold the footage through an app for a modest payout. As companies race to improve world models, more of this kind of data collection may become available as gig work.
The result is a quiet feedback loop: ordinary moments are captured, turned into training data, and used to teach machines how to move through the physical world. The robots and cars may stay out of sight for now, but the simulations they learn from are already being built from the stuff of everyday life.
