A robot could watch a million videos of people picking up eggs. Yet when it reaches for one, it might still crush it.
The problem isn’t necessarily intelligence. It’s missing information.
The internet can show AI what people do. It doesn’t always reveal the forces that make those actions possible.
And that difference may explain why building a general-purpose robot is proving so much harder than building a chatbot.
Watching an Action Isn’t the Same as Performing It
Imagine watching someone pick up an egg.
You can see their fingers close around it. But the video doesn’t tell you exactly how much pressure they apply, how the shell responds, or when the egg begins to slip.
A robot must turn visual understanding into precise motor commands while responding to physical conditions that can change from moment to moment.
Physical Intelligence’s π₀ illustrates this challenge.
Its researchers combined internet-scale visual and language knowledge with real robot demonstrations, enabling the model to generate continuous actions up to 50 times per second.
Understanding an instruction is only the beginning. Executing it requires another kind of knowledge.
Google DeepMind encountered a related challenge with Open X-Embodiment.
More than 20 institutions pooled over one million real robot trajectories spanning 22 robot embodiments, allowing models to learn from experiences collected across different machines.
That collaboration revealed something important: unlike web text, useful robot experience often has to be deliberately collected.
The Physical World Doesn’t Offer Free Training Data
Collecting successful demonstrations is difficult enough.
But what happens when the egg slips?
A robot also needs to learn how to recognize mistakes, adjust its grip, and recover from unexpected situations.
Those experiences can be valuable, yet generating failures in the real world may consume operator time, damage objects, or endanger equipment.
This creates a problem that simply scaling a language model cannot solve.
Researchers are exploring complementary approaches.
Internet video provides broad knowledge of objects and behavior. Simulation allows robots to practice without repeatedly risking physical hardware. Real-world demonstrations help connect those learned behaviors to actual machines.
NVIDIA’s Isaac and Cosmos technologies illustrate this approach, using simulation and generated trajectories to expand robotic training opportunities.
But synthetic experience still needs validation against reality. Simulated friction, contact, and object behavior may differ from the physical world.
The emerging challenge isn’t simply building larger robot models. It’s producing reliable physical experience at scale.
The internet gave language models a vast library of human knowledge.
Robotics may have to build its own library of physical experience.
CONTINUE THE RESEARCH
TECH VISION MAGAZINE · AUGUST 25, 2026
No Ready-Made Tokens in the Physical World: Data Acquisition and Labeling Challenges for Embodied AI
Explore Open X-Embodiment, π₀, action representations, tactile sensing, teleoperation data factories, and the engineering challenges of scaling real-world robot learning.