KUKA Innovation Award 2017

My group at the University of Southern California fielded a team in KUKA Innovation Award 2017 competition. Team members included Ariyan Kabir, Sarah Al-Hussaini, Abdullah Alsharhan, Vivek Annem, Iain Brookshaw, Qi Deng, Alec Kanyuck, Nithyananda Kumbla, Joshua Langsfeld, Rishi Malhan, Fadel Muci, Brual Shah, and Shantanu Thakar. 

After two preliminary rounds, our team was selected as one of the five finalists. Applicants for this award were expected to demonstrate an innovative robotic application using Kuka iiwa arms. Our team traveled to Hannover Messe in Germany to showcase our entry in the competition. This is one of the largest trade fair in the world. Usually, more than 250,000 visitors attend this trade fair. Distinguished visitors this year included Angela Merkel, Chancellor of Germany.

The focus of our application was automation of finishing processes such as grinding, sanding, and polishing. We combined planning, control, perception, learning, and augmented reality technologies to create a new robotic system for finishing operations. Our setup used two robots. The first robot held the part and the second robot manipulated the cleaning tool. We used external sensors to monitor the task progress.

Our application was significantly different from the traditional robotic applications in manufacturing. Robots in traditional manufacturing operations use pre-programmed motions to carry out the tasks. This idea only works when a robot is used is mass production application to make the same part over and over and this approach does not help in low volume production. An example of this is post-processing operations in additive manufacturing of custom parts. For metal based 3D printing, surface finishing operations are still manual and can take a very long time.

Angela Merkel, Chancellor of Germany walked past our booth (Image Source: Kuka)

Visitors at our Booth (Image Source: USC CAM)

Our Booth at Hannover Messe (Image Source: Kuka)

Picture at Awards Ceremony (Image Source: Kuka)

Our team with the Finalist Trophy (Image Source: USC CAM)

Manual surface finishing tasks are very tedious and time consuming and contribute significantly to the total cost in manufacturing. They also pose risks to the health of the workers. Our team believed that robots should do the tedious labor and humans should perform high level decision making in surface finishing operations. This way, we can increase the productivity of human operators and improve their quality of life.

The automated finishing system needed to manage the interaction between robots, tools, and the part to be finished. Robots needed to learn and optimize parameters on-the-fly for any given object and plan their moves. A perception system was also required for detection and localization, assessing surface quality, and ensuring safety. To achieve these goals, we integrated new planning and learning algorithms with the existing technology for perception and control.

Overall we received a lot of positive feedback on our demonstration. Many companies were interested in deploying our technology. Our team returned back to USC with a resolve to mature the technology and get it ready for deployment.

Are there any positive implications of autonomous cars on jobs in taxi industry?

Many people have raised concerns about the negative impact of autonomous cars on people’s ability to make living as Uber, Lyft, or regular taxi drivers.

Clearly autonomous cars will eliminate the need for drivers and hence people’s ability to make a living as drivers-for-hire. Let us dig a little bit deeper in this area and figure out if autonomous cars will create opportunities for people in the taxi industry to make money in some other way.

The cost associated with driving a personal car is approximately 50 cents per mile. This estimate includes the cost of a modest car, gasoline, insurance, and maintenance. The labor cost of drivers makes the cost of taxis four to six times higher. Taxi fares should go down dramatically once autonomous cars become mainstream. This will make taxi services a lot more affordable. Hopefully, this will encourage people to spend more time in taxis.

Once people start spending more time in taxis in complete privacy, they might want to spend that time more productively from work and/or leisure perspectives. Here are examples of activities that they might do in autonomous taxis:

• Doing video conferencing
• Eating
• Getting manicure/pedicures
• Personal grooming
• Watching movies
• Shopping
• Taking scenic detours
• Wine tasting
• Checking blood pressure

The above list just gives a few examples. Basically, autonomous cars will have captive customers and a wide variety of services can be offered to them by creative people. New technologies will create new service possibilities that cannot be imagined today. This should give creative people an opportunity to make money by offering services to people in autonomous taxis.

Designing such services and offering them will require a different skill set than driving. Hopefully, a proactive discussion about such service possibilities will help people get ready for the future when driving skills will be inadequate to earn a living.

I am curious to know your thoughts on what services can be offered to people spending time on autonomous taxis.

What can robotics community learn from artists with disabilities?

The robotics community aspires to build general purpose robots that can perform complex tasks effortlessly. In reality, we see the current generation of robots struggling to do even simple tasks.

We as roboticists admire human painters that can breathe life into canvasses with few brush strokes and sculptors in whose hands a marble slab melts like butter and an stunning awe-inspiring form emerges. We sigh with envy and hope that someday our robots will be good enough to carve a recognizable shape into the marble without us writing few hundred thousand lines of code.

We often look at human hands and eyes and marvel at the ingenious “design” behind the two. Everything from the available number of degrees of freedom to highly adaptive and high resolution sensing is truly remarkable. Human hands and eyes working in tandem endow artists with impressive hand-eye coordination capabilities that enable them to perform “miracles” and create mesmerizing art.
 
We compare human hands and eyes with the “clunky” hand designs and “dumb” cameras found in robots of today and resign to the fact that with current robotic hand and vision technologies, we are not going to get too far in terms of mimicking any impressive human feat. Do we need to wait for significantly improved robot hand and perception technology to build more capable robots or can we do better with what we already have?
 
I have been recently researching art created by artists with different types of disabilities. I am developing a very different perspective on whether the current hand and eye technology limitations are holding back the robotics community.  


I recently was introduced to paintings created by artists with severe visual impairment. A good starting point is “10 Remarkable Paintings by Blind and Visually Impaired Artists”.  This work is truly inspiring. Figures 1 and 2 show two representative paintings.

Figure 1: A painting by 
John Bramblitt (Image Source: http://illusion.scene360.com/art/78311/blind-artists/)

Figure 2: A painting by Eşref Armağan (Image Source: http://esrefarmagan.com/wp-content/uploads/2013/05/24.png)

Doug Landis is paralyzed from the neck down. He holds a pen in his mouth and creates amazing drawings by controlling the pen with his mouth. His art work is called mouth art. Figure 3 shows one of his drawings.
 

Figure 3: A painting by Doug Landis (Image Source: http://www.odditycentral.com/pics/doug-landis-mouth-art.html)

These examples show that humans are able to create amazing art despite serious physical handicaps. Something magical happens in the brain and it enables the artist to create amazing art by controlling the available sensing and manipulation modalities. Many years ago Matt Mason told me that "simple hands" are capable of doing quite a bit. My recent explorations seem to support that point of view. We need to develop a better understanding of what minimal sensory and manipulation capabilities are needed to create a piece of art.  

Hopefully, this post will inspire roboticists to stop waiting for the perfect robot hands and eyes. We ought to be able to do better with what we have right now.   

Latest manufacturing advances create new vulnerability to cyber threats

A number of recent reports have pointed out the vulnerability of manufacturers to cyber threats. Small manufacturers are an important part of the manufacturing supply chain and simply do not have expertise and resources to take proactive preventive actions against sophisticated cyber threats. These vulnerabilities can be exploited to cause disruptions to the supply chain. 

The introduction of advanced manufacturing technologies is expected to revolutionize manufacturing, enable innovation, and create new businesses. However, emerging manufacturing technologies will create new vulnerabilities from the cyberattack perspective. Here are representative examples of vulnerabilities created by the introduction of new manufacturing technologies.

• Digital Manufacturing: The manufacturing sector has moved away from paper–based blueprints and has embraced digital models. 3D models are being used to manage design and manufacturing processes and speed up the product development process. However, the reliance on digital data and models creates new vulnerabilities during cyberattacks.
• Network-Connected Machines: The Internet of the Things is revolutionizing manufacturing by finding applications in prognostics and health management, on-line process monitoring, and process optimization. It is expected to increase manufacturing resource availability, reduce energy and water consumption, and fundamentally alter the ways manufacturing equipment gets maintained and serviced. However, a machine connected on the Internet can be a target for a hacker.
• Cloud-Based Services: Cloud based services are increasingly being used to exploit big data related technologies to make sense of the data being generated by manufacturing enterprises. It can be leveraged to make smart decisions and improve the operational performance of the organization. However, the need to transfer data back and forth between the cloud and the manufacturing equipment creates new vulnerabilities.
• Automation: The use of robotic manipulators, 3D printers, and automated guided vehicles is expected to increase productivity in the manufacturing sector. These technologies can be run untended for days at a time. These technologies can not only reduce operational cost, but also offer new functional capabilities. For example, 3D printers can be used to fabricate designs that would have been impossible to make using traditional manufacturing methods. The absence of human operators means that tempering of the machine by a hacker is likely to go unnoticed for a considerable period of time and can cause serious problems.
• Miniaturization: Modern products increasingly use miniaturized subsystems. This delivers improved performance and packs many functions in a single product (e.g., smart phones). To meet this need, today’s manufacturing technology is able to create very small features. This also means that malicious tampering is very hard to detect. For example, a hacker can insert small features in a part being built on a high resolution 3D printer. Such features will be very difficult detect.
• Complexity: Modern manufacturing is a complex network consisting of hardware, software, and people connected over the network. This complexity will make it difficult to secure manufacturing enterprises from cyberattacks and detect an attack in progress in a timely manner.

Cyberattacks on a manufacturing enterprise can cause serious problems. The following list presents representative examples:

• Cyberattacks can be used to steal proprietary information and product designs.
• Activities in a factory can be monitored to develop reconnaissance on planned future missions and capabilities without even the need for stealing the product data.
• The digital data being used by the factory can be altered to make subtle changes in the products. These changes can sabotage the products or provide backdoor entry into the product.
• A hacked robot or automated guided vehicles can simply run around on the factory floor at a high speed and cause major damage to the expensive equipment on the shop floor in a matter of few minutes. A sensor reading can be modified during the process control loop execution and can be used to cause serious damage to the equipment and the product being made.
• A critical machine can be simply shut off by a hacker and cause major production disruptions. This can have a significant crippling impact on the downstream supply chain.
• Critical information and data stored on a computer can be corrupted and rendered useless. This can lead to the loss of critical knowledge and trade secrets.
• Infected machines and robots can cause physical injuries to people in the factories.
• Infected machines can trigger fire and other environmental hazards for the nearby residential communities.
• Shutting down of a factory by a cyberattack can have significant economic impact on the local community as many members of the community (e.g., food vendors, retailers) rely on the factory workers for their livelihood.

Significant progress has been made in the field of Cyber Security for Information Technology based systems. Securing manufacturing enterprise from cyberattacks presents many new challenges. Machines and equipment used on factories have a long life (e.g., 20 to 30 years). They have limited memory and computing power and often unable to run the latest security software. Upgrading them frequently is economically not viable. The strategy of simply shutting down a machine in the middle of an expensive build is also not practical as it will lead to significant waste. Most small manufacturing companies do have people with the right expertise to monitor and recognize cyber threats. The physical aspect of a manufacturing enterprise means that simply taking a machine off the network will not contain the damage. The robot may continue to move and keep causing physical damage despite being off the network.

Making manufacturing enterprises safe from cyberattacks is a challenging task. It will require developing new cyber-physical security technologies and training people to combat cyberattacks and take proactive measures to secure the equipment. Manufacturing companies will need to build a culture that ensures that people take appropriate preventive measures to reduce vulnerability to cyberattacks. The presence of WiFi-connected smart phones and smart watches on the factory floors poses a major challenge to securing the factory network.

Recent Computing Advances: Artificial Intelligence or Augmented Intelligence?

There have been significant advances in computing over the last twenty years. These advances enable computers to perform amazing feats that are far beyond the capabilities of humans. Here are few examples.

• Ability to “read” millions of documents in a few minutes and index the information contained in those documents
• Search for digital artifacts (e.g., documents, images, etc.) based on user specified criteria in a fraction of a second from a diverse set of digital repositories
• Detect complex patterns and anomalies in real-time in a datastream consisting of several gigabytes of data
• Systematically generate and explore millions of options within a few seconds to determine the optimal course of action

These capabilities have been harnessed to build systems that can beat human grandmasters in chess, design novel devices that can be patented, detect barely visible early stage tumors in CT images, and predict fraudulent credit card transactions with a high degree of accuracy.

The computing technologies behind the above mentioned capabilities are often referred to as Artificial Intelligence (AI). Some are predicting that AI will surpass human intelligence in the not too distant future. These predictions are beginning to alarm a segment of the general public. This raises several questions. Will human intelligence be relevant if AI continues to make rapid advances? What jobs will humans do if AI continues to surpass humans on tasks that are currently performed by intelligent humans?

I view recent computing advances from a different lens. I believe that these advances will augment human capabilities instead of competing with humans. Computers and humans have different strengths. In the previous paragraphs, I outlined some of the ways in which computers are excelling. Let us now review the particular strengths of human beings. Only humans can relate to human emotions and their underlying origins (many of these often defy logic in the mathematical sense!). I strongly believe that only humans can understand needs, wants, and desires of other human beings. Human contact and social interaction is the only way to lift the spirits of human beings and inspire them to do their best. Humans are very creative and capable of inventing artifacts and services that other human beings want. They possess the judgment to figure out how to assess values of different options in a complex decision making problem. They are capable of incorporating ethical, moral, cultural, and legal considerations in the decision making process. Humans also have an innate ability to generalize from a very limited number of observations to draw broader conclusions.

The latest advances in computing are freeing humans from biologically-imposed constraints on memory, computing speed, and communication bandwidth. We can view recent advances in computing as a way to augment human intelligence, therefore the term AI should stand for Augmented Intelligence. Using augmented intelligence, every human will be able to communicate with thousands of people at the same time, communicate in virtually every language, explore millions of options before making a decision, and access all known human knowledge. Leveraging augmented intelligence will enable humans to excel by exploiting their creative energy and their ability to connect with their fellow humans. I believe that humans armed with augmented intelligence will have the potential to improve virtually all facets of our lives. Here are ten ways augmented intelligence will help people in doing their jobs better: 

1. Currently educators are unable to provide personalized attention to students to suit their learning style and pace. With augmented intelligence, professors and teachers can use augmented intelligence to provide personalized attention to students by designing and grading tailored assignments to match the learning needs of each student. 
2. Nowadays, due to the large volume of the published research, practicing doctors are unable to remain current with the latest findings. Doctors can use augmented intelligence to ensure that each patient’s treatment plan is informed by the latest advances in the field by taking into account all the relevant latest research on treatment options and side effects of new medicines.
3. Most challenging problems require an interdisciplinary approach. For example, engineers want to take inspiration from biology. Engineers can use augmented intelligence to analyze all the relevant literature outside of their narrow field of expertise (e.g., biology) to draw inspiration and automatically construct models from the published experimental data. They can focus on asking the right questions and leave the more tedious work to the computers.
4. Law enforcement agents can use augmented intelligence to gather and analyze all the evidence instantaneously by searching and integrating information from many disparate information sources.
5. Designers can use augmented intelligence to synthesize optimal designs by rapidly generating and evaluating billions of options. They can use their insights and judgment to steer the computer search in the right direction. For example, this can help in designing betters drugs.
6. Marketing professionals can use augmented intelligence to mine the vast amount of social media data to understand the recent trends and customer needs. This can help them in developing the right marketing strategy in order to craft advertisement campaigns that appeal to the taste of their prospective customers.
7. Augmented intelligence can enable entertainers to stay in touch with their fans by providing personalized response on social media and analyzing fan feedback to create art that truly inspires.
8. Using augmented intelligence, writers can get access to documents written in virtually every language to draw upon and can transcend the language barrier to reach a worldwide audience.
9. Financial advisers can use augmented intelligence to offer affordable personalized advice to their clients by better understanding client needs and the market conditions in today’s fast changing world.
10. Public policy professionals can analyze the implications of their proposed policies by conducting extensive computer simulations and customized surveys. The execution of both of these tasks can be improved by using augmented intelligence to automate these tasks.
    

Unfortunately right now, access to augmented intelligence will be restricted to a privileged few. This will exacerbate the already existing digital divide. Society will have to work that much harder to ensure that access to augmented intelligence technology will be available to all.