GRASP Lab: teaching robots complex tasks with 5 minutes of video
Most robot learning research makes the same bet: scale the data and you scale the capability.
SymSkill, from the University of Pennsylvania's GRASP Laboratory, won the top prize at ICRA 2026 — the IEEE International Conference on Robotics and Automation, robotics research's most prestigious venue — by challenging that assumption directly.
The result: a robot arm that learned 12-step sequential tasks from 5 minutes of unstructured video and recovered from mid-task failures without restarting the sequence. SymSkill took ICRA 2026's Best Conference Paper Award and its Best Paper on Planning and Control.
It had already won a best paper at CoRL 2025 for the theoretical foundation it builds on.
The data bottleneck problem
Modern robot learning has a scaling problem. Imitation learning works well for individual actions but breaks down when tasks get sequential and complex. Ask a robot to remove a lid from a pan, pick up a banana, and place it on a plate in that order, and the required demonstration data grows fast. Add in real-time failure recovery — what happens when a step goes wrong mid-sequence — and the data requirement explodes.
Classical task-and-motion planning (TAMP) sidesteps the data problem by using symbolic logic: if you hand-code the rules, the planner can compose arbitrary sequences. But hand-coded symbolic representations don't generalize to new objects or environments, and classical planners are too slow for real-time physical recovery when things go wrong. Most labs either scale up demonstrations or accept that their planner can't handle physical messiness.
SymSkill does neither.
What SymSkill does differently
The system has 2 layers: a symbolic planning layer that learns the abstract structure of tasks (what objects exist, what states they can be in, what actions change those states), and a dynamical systems layer that controls the physical movement. The critical move is that both layers are co-invented jointly from the same unlabeled, unsegmented play data, not designed or hand-coded separately.
Feed the system 5 minutes of video showing unstructured object manipulation, and it infers the symbolic relationships itself.
Advisor Nadia Figueroa, a Penn mechanical engineering professor, described it:
"Think of it like creating roads. Once we set up the map and provide the rules of the road, the robot can be given a destination and use what it knows to get there without being shown the exact route."
Lead author Yifei Simon Shao:
"A robot might already know how to grasp an object, move it and place it somewhere else. But a real-world task could require those skills to happen in a particular sequence. Once it has learned all basic skills and understands how they relate to the world around it, it can start putting those skills together in ways it hasn't seen before."
The robot doesn't need to see every task sequence demonstrated. It learns primitives and recombines them. That's what 5 minutes of video buys.
The banana demo
In the real-world test: a Franka robotic arm removes a lid from a covered pan, picks up a banana, and places it on a plate. Concrete and mundane, but what's notable is the failure recovery. If a human interrupts the sequence, or if a step fails, the robot doesn't restart from the beginning. It replans from the current state and continues.
In simulation (RoboCasa), the system learned 12 individual tasks and composed them into multi-step plans without additional demonstrations, achieving 85% success on sequences it had never seen.
On physical hardware, under deliberate interference, the replanning capability held.
Why this matters for humanoids
SymSkill is currently a single robotic arm. The GRASP team is working toward bimanual and dual-arm extensions, which points directly at humanoid deployment. The data efficiency argument is particularly relevant there: you can't ask factory operators to generate thousands of demonstrations every time a robot needs to learn a new assembly procedure.
If a robot can learn a new 12-step task from 5 minutes of a human showing it the relevant objects, deployment economics change.
The contrast with the dominant industry approach is pointed.
Runway's Praxis-1, announced this week, bets on scaling internet video as training data: more video, better robot. SymSkill bets on structure: less data, better architecture. Both can be right simultaneously for different deployment contexts, but they represent genuinely different theories of where robot intelligence comes from.
Figueroa's idea?
"We can make robots more capable without simply throwing more data at the problem."
Sources
Written by
HMH