Researchers from Google, Princeton, Columbia and MIT developed a picking robot using physics and deep learning that can accurately toss random objects into bins two times faster than previous systems.
“This robot, like many others, is designed to tolerate the dynamics of the unstructured world,” mentioned Andy Zeng, Student Researcher of Robotics at Google. “But instead of just tolerating dynamics, can robots learn to use them advantageously, developing an “intuition” of physics that would allow them to complete tasks more efficiently? Perhaps in doing so, robots can improve their capabilities and acquire complex athletic skills like tossing, sliding, spinning, swinging, or catching, potentially leading to many useful applications, such as more efficient debris clearing robots in disaster response scenarios — where time is of the essence.”
Using an NVIDIA TITAN GPU and PyTorch, the researchers trained their deep neural networks to predict adjustments on top of the physics-based throwing velocity estimate, in order to compensate for unknown dynamics as well as the noise and variability of the real world.
Besides training, Tossingbot relies on the GPU for real-time 3D reconstruction of the scene with RGB-D cameras to capture and aggregate 3D data as the robot moves around. The robot improved itself over time through continual learning — successfully achieving an 85% throwing accuracy over the course of 14 hours.
According to the researchers, TossingBot can grasp and throw over 500 arbitrary objects into boxes located outside its maximum reach range.
Related resources
