Turning Lasers into Robotic Optical Hands for Manipulating Biological Cells

Newton’s scientific accomplishments are truly astonishing. One of his remarkable theories stated that light has momentum. If light has momentum, then it should be possible to move objects by shining a light on them. I am sure that it sounded like a crazy idea when Newton first proposed it.

Over the years, people have done numerous experiments to confirm this theory. This idea is so captivating that it even influenced the great George Lucas. Star Wars movies featured famous Lightsabers that utilized the special properties of the light to create a powerful Jedi weapon. But we have not seen such fantastic spectacles of light and matter interaction in our everyday macroscale world. Light has very small momentum. So moving a heavy couch by shining a laser on it remains in the realm of science fiction. Unfortunately, if you make the laser too powerful, it will simply evaporate the couch and set your house on fire.

A different picture emerges at the microscale. It is certainly possible to move tiny objects by shining a laser on them. But this mode of interaction does not offer much control. Ashkin in 1986 figured out a better way. He created optical traps that were able to hold tiny particles in place. The basic idea was to bend and focus a laser beam tightly using an objective lens. Once the object enters the laser beam, the laser starts interacting with it and pushing it towards the focal point where it gets trapped.

We can imagine the laser as a collection of rays. These rays are reflected and refracted by objects that intercept them. As the rays are bent, their momentum changes and they exert force on the object. This phenomenon can be visualized as interactions between a stationary ball and a moving ball. The direction of the motion of the moving ball changes as it strikes the stationary ball. Hence it exerts a force on the stationary ball. Ashkin found out that once the effect of all the rays in the laser beam was accounted for, the direction of the resultant force was such that the object was pushed towards the focal point of the beam. So as the object entered the laser beam, it was simply pushed towards the focal point and once it reached that point it remained there. In essence, the focused laser beam created a particle trap. A trapped particle can be moved by moving the laser beam. Thus, the laser has been turned into a tweezing tool for grabbing small particles and moving them. Moving optical traps are often referred as optical tweezers.

Numerous groups have used optical traps to manipulate biological cells and study them. In fact, many important discoveries in biology have been made using optical traps. Biologists are primarily interested in fundamental scientific discoveries. So they are happy to create and move optical traps using tele-operation. Just like tele-operated robots, tele-operated optical traps have many inherent limitations. They are slow, require significant expertise, and limit what kind of manipulation is possible.

I was introduced to optical tweezers in 2004 during my sabbatical at the National Institute of Standards and Technology. Thank you Arvind Balijepalli and Tom LeBrun! I am interested in robotics. So once I learned about optical traps, I became interested in turning them into robotic hands for automatically manipulating biological cells. In many situations, directly trapping biological cells can cause complications. Cells might have an irregular shape and they might be susceptible to damage due to direct exposure to the laser.

We decided to take a different path. Rather than building robotic optical tweezers, we wanted to build robotic optical hands. The idea was to use the laser to trap and move microspheres made out of silica or polystyrenes. These microspheres can serve as “fingers” for gripping or pushing cells. So in our idea, the laser would act as an optical hand (i.e., hand of a ghost) and microspheres would act as “fingers”. This idea enabled free floating “fingers” with no physical hand attached to them. This would be truly an alien hand with no biological counterpart on Earth! We could have as many “fingers” as we wanted by splitting the laser beam to create multiple traps. We could also have multiple hands if we wanted! It was a crazy idea. But it removed many constraints associated with conventional microscale robotic grippers and offered several new possibilities. Soon we were hooked to make this idea a reality.

There were numerous challenges. Microspheres and cells float in the liquid medium and exhibit Brownian motion. We had to detect these objects in the scene, plan the next trap location, and make sure that microspheres and cells moved the way we wanted them to move. However, we only had a few milliseconds to do image analysis, planning, and control. Images are noisy and the environment has significant uncertainty. Moreover, the motions of all the hands and fingers need to be exquisitely coordinated. So this was a really tough robotics problem. +ashis banerjee , +Sagar Chowdhury , +Petr Svec , and +Atul Thakur worked incredibly hard to solve the challenging planning, perception, and control problems to realize this vision. They built upon the basic software capability provided by Andrew Pomerance. Wolfgang Losert and Chenlu Wang provided valuable help in conducting the experiments. Thank you National Science  Foundation for supporting this work! 
 
Our adventures in this area began by concurrently trapping multiple microspheres and moving them into an ensemble. We then used that ensemble to hold a cell and move it. We also developed the capability to move the cell into its desired location by pushing on it using a microsphere. If a cell is very sensitive to the laser, then we can use an intermediate microsphere as a tool, so that the microsphere “finger” being trapped by the laser does not touch the cell and ensures physical separation between the cell and the laser. Please see below the video of our robotic optical hand.

Hopefully, our colleagues in biology and medicine would be able to think about new scientific theories  that can be enabled by the above described robotic optical hands and their variants. Possibilities range from understanding the behavior of cancer cells to understanding how cells communicate.

Robotics solutions that enable precise automated manipulation of individual cells are expected to revolutionize medicine and biology. Our explorations in this area have taught us that robotics at the microscale requires out-of-the-box thinking. We are now busy coming up with even crazier ideas to marry robotics and biology. So stay tuned for updates.

Recent Advances in Industrial Robots and Their Implications on Manufacturing

Industrial robots (e.g., ABB, PUMA) have been quite successful in mass production assembly lines. For example, they are routinely used to weld, paint, and join parts in automobile industry. However, small and medium manufacturers (SMM) in the US have largely stayed away from using industrial robots. They continue to rely on manual labor and this makes it hard for them to compete with overseas suppliers with low labor costs.

The National Association of Manufacturers (NAM) defines small manufacturers as companies with 500 or fewer employees and medium-sized manufacturers as companies with 2,500 or fewer employees. The NAM estimates that that the US has close to 300,000 SMM, representing a very important segment of the manufacturing sector. As we move towards shorter product life cycles and customized products, the future of manufacturing in the US will depend upon the ability of SMM to remain cost competitive.

This blog post explores the reasons behind the lack of adoption of industrial robotics technology by SMM and recent advances in robotics that might change the status quo.

Let us explore a representative scenario to understand the limitations of the current industrial robots and why they are not used by SMM. Imagine that you are working in a small company and building a prototype of new medical device. You are under extreme time pressure to meet an important deadline. As you are assembling the device, you realize the bracket is too compliant. You need to laser cut it again in a much stiffer material. The good news is that it will only take six minutes to cut the bracket. But the logistics associated with it will take an hour. You really need to continue assembling the rest of the assembly and testing the controller. You simply don’t have an hour to spend and can certainly use an assistant right now!

Here is what you would like your assistant to do - walk over to the material storage area, locate the right material, pick up the material, take it to the laser cutter, open the laser cutter, place the material in it, press the button to start cutting, wait for the part to finish, open the laser cutter, pick up the part, clean it, and bring it to you. Obviously human assistants can do all of these tasks without even flexing their cognitive muscles. I am sure that they can do all of these tasks while texting and surfing the net on their smart phones! Unfortunately the current industrial robots simply cannot do these tasks. So you simply cannot get a robot assistant today!

Robots that rule the assembly line have the following four limitations. First, they are immobile. They cannot go to the task location. The work has to be brought to them. Second, their dexterity is extremely limited. Simple tasks such as opening shelves and precisely placing and securing a previously unseen part in a machine are out of their capabilities. Third, it takes a long time to program them. So using robots on no-repetitive tasks is simply counter-productive. Finally, robots cannot work in the close proximity of humans because of safety concerns. So you can forget about a robot assistant walking over and handing you a tool or a part to assist you on the shop floor.

Most SMM use highly automated machines (e.g., CNC machines, laser cutter, water-jet cutters, CNC press-brakes, 3D printers). However, SMM shop floors tend to be unstructured and often go through changes to meet the needs of the projects at hand. Main sources of manual labor in SMM are material transport and handling, machine setup and calibration, inspection, clean-up, and packaging. Unfortunately, the current industrial robots that are designed for mass production assembly lines are of not much use in these tasks. So industrial robots offer very little value to SMM!

Recent advances in robotics are challenging the status quo and aiming to turn robots into important tools for SMM. I would like to share the following important trends:

• Mobile manipulators are robots that can transport themselves to the work site. I recently saw demonstrations of mobile manipulators developed by Kuka that show impressive capabilities. This capability will be very useful in expanding the role of robots in manufacturing, particularly from the SMM point of view.

• Dexterity has been a major obstacle to the widespread use of robots in manufacturing. Recent developments on robot hands are targeting to overcome this obstacle (e.g., Schunk and Barrett hands). 3D printing enables users to quickly create their own customized grippers in few hours.

• Baxter from Rethink Robotics is aiming to eliminate the need for writing code to program robots. Instead, robots can be programmed by demonstrating the tasks. This is expected to empower workers on the shop floor. They will be able to start utilizing robots without the need to wait for a robot programmer to assist them.

• Recent advances in human-safe robots are enabling robots to work in the close proximity of humans. Kuka lightweight arm and Baxter are representative examples of advances in this area. Many researchers are developing methods to track human operators in the workspace to make robots aware of humans in the workspace and change planned robot motions to avert injury to humans. For example, +Krishnanand Kaipa , +Carlos Morato , and +Boxuan Zhao  in my lab have developed a system to monitor a human operator working in the close proximity of a robot using four Microsoft Kinect sensors. This information is used by the robot to update its plan. The video of this system is shown below.

I believe that ultimately the convergence of the above mentioned technologies will create the second generation of industrial robots that will revolutionize the manufacturing industry. 

Once the demand increases for these robots, the cost for them will start coming down. There is no reason why low-end industrial robots cannot be sold for less than ten thousand dollars once the economy of scale kicks in. This in turn will make robots affordable for SMM and manufacturing cost-competitive in high wage countries.