Ken Goldberg is a professor and William S. Floyd Jr. at UC Berkeley. He is the Engineering Distinguished Chair, co-founder and chief scientist of the robotic packet sorting startup Ambidextrous, and a researcher at IEEE.
What roles will generative artificial intelligence play in the future of robotics?
Although the rumblings start a little earlier, 2023 will be remembered as the year when productive artificial intelligence transformed Robotics. Big language models like ChatGPT can enable robots and humans to communicate in natural language. Words have evolved over time to represent useful concepts, from “chair” to “chocolate” to “charisma.” Robotics have also discovered that large Vision-Language-Action models can be trained to facilitate robot perception and control the movements of robot arms and legs. Because training requires huge amounts of data, laboratories around the world are now collaborating to share data. The results continue to pour in, and although there are still open questions about generalization, the impact will be profound.
Another exciting topic is “Multimodal models” in two senses of multimodal model:
Multi-Modal in combining different input modes, e.g. Vision and Language. This feature is now expanded to include Haptic and Depth sensing and Robot Actions.
Multi-Mode in that it allows different actions in response to the same input condition. This is surprisingly common in robotics; For example, there are many ways to grasp an object. Standard deep models will “average” these grasping actions, which can lead to very weak grasps. A very exciting way to protect multimodal actions is Diffusion Policies developed by Shuran Song, now at Stanford.
What are your thoughts on the humanoid form factor?
I’ve always been skeptical of humanoids and legged robots because they can be overly sensational and inefficient, but after seeing the latest humanoid and quadruped robots from Boston Dynamics, Agility, and Unitree, I’m reconsidering. Tesla has the engineering capabilities to develop low-cost motors and gear systems at scale. Legged robots have many advantages over wheels when it comes to traversing steps, debris, and rugs in homes and factories. Bimanual (two-arm) robots are necessary for many tasks, but I still believe simple grippers will continue to be more reliable and cost-effective than five-fingered robot hands.
After manufacturing and warehouses, what is the next major category for robotics?
After the recent union wage regulations, I think we will see many more robots in production and warehouses than we do today. Recent developments in driverless taxis are impressive, especially in San Francisco, where driving conditions are more complex than in Phoenix. But I’m not convinced they can be cost-effective. For robot-assisted surgery, researchers are exploring “Augmented Dexterity,” where robots can improve surgical skills by performing low-level subtasks such as suturing.
How far away are true general-purpose robots?
I don’t expect to see true AGI and general purpose robots in the near future. Not a single roboticist I know is worried about robots stealing jobs or becoming our masters.
Will home robots (beyond vacuum cleaners) emerge in the next decade?
I predict that within the next decade we will have affordable home robots that can reduce clutter, picking up things like clothes, toys, and trash and putting them in appropriate bins. Like today’s vacuum cleaners, these robots will make mistakes from time to time, but the benefits to parents and seniors will outweigh the risks.
What important robotics story/trend doesn’t get enough coverage?
Robot motion planning. This is one of the oldest topics in robotics; How to control motor connections to move the robot tool and avoid obstacles. Many people think this problem is solved, but it is not.
Robot “singularities” are a fundamental problem for all robot arms; These are very different from Kurzweil’s hypothetical point in time when AI surpasses humans. Robot singularities are points in space where a robot stops unexpectedly and must be manually reset by a human operator. Singularities arise from the mathematics required to convert the desired straight-line motion of the gripper into corresponding motions for each of the six robot joint motors. At certain points in space this transformation becomes unstable (similar to a division by zero error) and the robot must be reset.