Are You Ready to Dance with Robots?

The world of art plays an important role in human lives. The art mesmerizes and inspires us. It unleashes the creative energy and challenges conventional thinking. It provokes new thoughts and compels us to ask new questions. Can robots play a role in the art world?

Fictional robots have been playing prominent roles in movies for many years. Star Wars movies will not be the same without C-3PO and R2D2. The use of robots in movies enables writers to create new plots and enables actors to interact with superhuman characters.

The field of robotics has made tremendous progress. We now have truly remarkable robots. Can these real robots influence the art world?

I had an opportunity to interview Huang Yi on Thursday September 24, 2015 in the Clarice Smith Performing Arts Center. He is one of the pioneers of a new form of dance. His partner is a Kuka robot!

Kogod Theater Stage (Photograph by Rebecca Copeland) 

He currently uses a large intimidating orange Kuka robot in his performances. He said that he liked the Kuka robot because of its form. He programs his “dance partner” to glide through a space in harmony with music. Huang Yi and the robot move in unison during the performance and are able to express emotions to complement and augment the ambiance created by the music. His thought provoking performance asks us to examine the relationship between humans and robots.

Huang Yi's Dance Partner
(Photograph by Rebecca Copeland)

Huang Yi likes the complete predictability of the robot moves. It makes the dance safe and enables him to keep the tempo high without worrying about the need to constantly watch the robot. Currently it takes him ten hours of programming to create one minute of performance.

I wonder how this form of dance will change as robots become more intelligent and safe? Safety will encourage many more people to explore dancing with robots. Intelligence will enable robots to react to human moves and hopefully it will become easier to create new dance moves.

Huang Yi in the lab with our Kuka robots
(Photograph by Rebecca Copeland) 

Some art students in the audience seem a bit concerned about the need to learn programming to master this new art form. Hopefully advances in the area of learning from demonstrations can eliminate this barrier.

I wonder how this art form will change if we had robots that can understand the human emotions and gauge the mood expressed by the music!

What will it take for you to dance with robots?

RoboSAM: A robot that is smart enough to call humans for help!

In my opinion, one of the most important attributes of being smart is the ability to seek help when needed. This requires realizing that help is needed and getting the right kind of help from the right source. Currently, robots do not have an ability to assess whether they can successfully complete a task or not. When instructed to do a task, they simply attempt to do it. Sometimes the task execution results in spectacular success that delights the spectators and other times it leads to an embarrassing failure that baffles everyone, except the person who programmed the robot. Clearly, if robots were to become smart, they will need to ask for help when they are unable to do a task. 

Occasional robot failures can be tolerated. However, using humans to frequently clean up the mess created by robots is simply not a viable business model for using robots. Currently, deploying robots in industrial applications requires the reliability of robotic task execution to be very high. This is accomplished by designing specialized hardware and software. Extensive system testing is needed to ensure that potential failure modes are well understood and contingency plans are developed to handle them. Typically, task execution failures shut down the line and require human intervention to clear the fault and restart the line. This type of intervention is very expensive and hence robots are not used on a task until extremely high-level reliability can be achieved. Customized hardware and software costs can only be justified if the production volume is sufficiently high and tasks are repetitive (e.g., automotive assembly lines). 

To understand the underlying challenges in robot deployment, consider the following scenario. A robot is capable of picking a part if it is presented to the robot at a certain location. However, if the part has shifted from its nominal location, the robot might not be able to grasp it. The robot does not simply know where the transition boundary between task execution success and failure lies. If the part is sufficiently distant from its expected location, as the robot attempts to grasp it, the robot might bump into it, push it further, and jam the material handling system. This can in turn trigger a system fault and shut down the system. 

In order to use robots in small production batch operations or non-repetitive tasks, we will need robots that are able to estimate the probability of task completion before beginning the task. This will enable robots to assess their own confidence in doing a task. If the robot does not have high confidence in completing a task, then it should call for help. This will enable human operators to provide the robot with needed assistance (e.g., better part pose estimation, invoking a different grasping strategy) and prevent major system faults that result from task execution failure. Please keep in mind that the human only needs to help the robot with the portion of the task that is proving to be challenging. The robot can do the rest itself. In most situations, providing task assistance help to robots is much cheaper than recovering from a system shutdown. 

My students have been building a robot to demonstrate this concept in the bin picking context. This project is called RoboSAM (ROBOtic Smart Assistant for Manufacturing). Bin picking capability is representative of a robot’s ability to perceive the desired object in the environment and to successfully pick it up and deliver it in a known pose. If the robot is not sure whether it can pick the desired part from a bin containing many different parts, then it calls a remotely located human operator for help. We call this operational concept human-on-call concept. This is fundamentally different from the human-in-the-loop concept that requires the human operator to actively monitor the manufacturing cell and take control away from the robot when the robot is about to make a mistake. The new concept requires the robot to call the human operator when it decides that it needs help. 

I believe that human-on-the-call concept is the right economic model for deploying robots. It enables humans to move away from doing boring routine tasks to do challenging tasks with which robots struggle. This model allows a single remotely situated human operator to help multiple robots on an “as needed” basis. It also enables robots to be deployed on tasks on which achieving very high success rate will be difficult. For the near foreseeable future, a large number of tasks in small and medium manufacturing companies fall in this category. 

People often ask what humans will do when robots become more widespread. In my opinion, humans will be needed to teach robots how to do different tasks and bail robots out when they are confused. The key will be to develop technologies that allow robots to ask for help when needed. Recent work in our lab is a step in that direction.