Nowadays, there is a lot of excitement about the low-cost computing, low-cost sensing, and universal connectivity. These technologies can be combined together to create the next generation of appliances with truly remarkable capabilities. In fact, companies are beginning to imagine a new industry around this idea (e.g., Industrial Internet by General Electric).
Terms “smart” and “intelligent” have been widely used for the last three decades to market really “dumb” devices. So we will need a new term to characterize the next generation of appliances. I am not creative enough to come up with a good buzzword. So I will just use the term “genius” to refer to the next generation appliances in this post.
From a purely technological perspective, it is feasible to create a “genius” refrigerator and have the following conversation with it in the morning as you open it to get your breakfast.
“Good morning!” your refrigerator greets you in a perky voice.
“Good morning,” you reply in your before-coffee half-sleepy voice.
“By the way, you need a haircut. Should I make an appointment for you?”
“I want to wait for couple of weeks.”
“Are you sure? You should really get a haircut now.”
“I will wait,” you reply as you take out cream for your coffee.
“Your calorie intake has increased and your face looks a bit chubby. You should go see your doctor. Should I make an appointment?”
“No, thank you.”
“Should I send a list of food that you are eating to your doctor?”
“No!”
“Should I email that list to your wife?”
“What the heck! No!!”
“Please throw away eggs. They have expired.”
“Thanks,” you throw away the expired eggs.
“You are really running low on groceries. Should I make a grocery list?”
“Sure.”
“Should I try to find places that have best deal for items on your list?”
“No, I will pick them up from the local Giants,” you reply. You don’t want to go to three stores at the end of a busy day to just save few bucks.
“Your wife always gets the best deals list.”
“Ok. Send me the list.” You are afraid that it might send an email to your wife.
“Should I email you coupons too?”
“Why not!” You are embarrassed to use coupons. But you are getting worried that it might now send email to your wife complaining about your wasteful ways.
“My water filter needs to be changed. Should I order it?”
“How much is it?”
“One hundred nineteen dollars.”
“Can you find me a better deal?”
“This is the best deal.
“Ok. Order it.”
(Later that day, you find a better deal on the filter at Amazon. But your refrigerator company wants to make money by selling you water filters. So it certainly is not interested in finding you a better deal.)
“Your wife’s birthday is next week. Should I order Gucci shoes for your wife? She really likes shoes.”
“How do you know that?”
“I checked her Facebook profile.”
“I will take care of the gift myself.” You are tempted to tell your refrigerator to order the shoes. But if your wife finds out about it, you will be in big trouble.
“My compressor might start making sounds like [...strange clanking sounds…]. Don’t be alarmed by that. I have already scheduled a service call and added it your calendar.”
“How do you know about that sound?”
“I follow tweets from my manufacturer.”
“I hope that you are not tweeting about us to grocery stores. Do you?”
“Please hurry up and try to leave early. Weather is expected to be bad and traffic is looking really bad.”
“Shouldn’t you have told me this before all the chit chat!! What should I do?”
“Please wait. Software update is in progress.…”
Here is what might happen that evening. You arrive at Wegmans, ten miles away from your home to get a good deal on the organic milk and tomatoes. You check email on your smart phone to download the grocery list. You have another email from your refrigerator! It is informing you that Costco has advertised a better sale. So you should go there to pick up tomatoes. The only problem is that the nearest Costco is eight miles away from where you are. You are worried that your refrigerator is “genius” enough to distinguish between the tomatoes from Wegmans and Costco and it might yell at you for wasting money when you try to put them in the refrigerator.
I did not say that you would necessarily enjoy having this conversation and getting emails from your refrigerator. I just said that technology is making it feasible.
By the way, the above scenario is not that far-fetched. Most homes in the United States have a good Internet connection. So, at least in theory, appliances at home can be connected to the Internet. Computing hardware is really inexpensive. So every appliance can have multiple CPUs without increasing its cost appreciably. If it needs more computing power, it can always connect to the cloud (e.g., Amazon Web Services) and do number crunching there.
The sensor cost has come down significantly as well. Every appliance can now have cameras, microphones, inertial measurement units, GPS, and bar code scanners. Significant progress is being made in chemical sensors, RFID readers, and thermo-mechanical sensors. These sensors should soon become pervasive as well.
There have been significant advances in human-machine interfaces as well. Voice recognition, face recognition, gesture recognition, natural language processing, and touch screens are expected to change the way we interact with household appliances. Very soon we should have technology to recognize human emotions as well.
Data mining, data fusion, and search techniques are able to analyze and integrate vast amount of data from different sources and turn it into useful information for human use.
In my opinion, future advances in appliances will come not only from improvements in component technologies, but also from the integration of a diverse set of technologies to better understand the context and provide value-added services to users.
Advances in technologies will enable many new features in products. Obviously, not all of them will be useful to all of us. The “genius” refrigerator described above had many useful features with the exception of few annoying quirks. But this is a problem with every new technology. I cannot imagine living without a computer and the Internet, but I have done enough screaming at my computer due to its counter-intuitive interface and its ability to freeze whenever I have an important deadline.
I am really excited about the possibility of the completely interconnected world. But I am a bit concerned how this will impact my life. Will “genius” appliances control my life, or will they make my life easier? Perhaps a bit of both!
I hope that my friends working in the human-robot interaction area can figure out a better mode of human-refrigerator interaction, before I buy a “genius” refrigerator.
In summary, I am looking forward to plugging my refrigerator to the Internet. I am also ready to receive emails from it, as long as I get only one email per week. I am having tough time staying in touch with human friends on Facebook. So I certainly do not want my refrigerator to become my Facebook friend.
Are you ready to receive emails from your refrigerator?
Terms “smart” and “intelligent” have been widely used for the last three decades to market really “dumb” devices. So we will need a new term to characterize the next generation of appliances. I am not creative enough to come up with a good buzzword. So I will just use the term “genius” to refer to the next generation appliances in this post.
From a purely technological perspective, it is feasible to create a “genius” refrigerator and have the following conversation with it in the morning as you open it to get your breakfast.
“Good morning!” your refrigerator greets you in a perky voice.
“Good morning,” you reply in your before-coffee half-sleepy voice.
“By the way, you need a haircut. Should I make an appointment for you?”
“I want to wait for couple of weeks.”
“Are you sure? You should really get a haircut now.”
“I will wait,” you reply as you take out cream for your coffee.
“Your calorie intake has increased and your face looks a bit chubby. You should go see your doctor. Should I make an appointment?”
“No, thank you.”
“Should I send a list of food that you are eating to your doctor?”
“No!”
“Should I email that list to your wife?”
“What the heck! No!!”
“Please throw away eggs. They have expired.”
“Thanks,” you throw away the expired eggs.
“You are really running low on groceries. Should I make a grocery list?”
“Sure.”
“Should I try to find places that have best deal for items on your list?”
“No, I will pick them up from the local Giants,” you reply. You don’t want to go to three stores at the end of a busy day to just save few bucks.
“Your wife always gets the best deals list.”
“Ok. Send me the list.” You are afraid that it might send an email to your wife.
“Should I email you coupons too?”
“Why not!” You are embarrassed to use coupons. But you are getting worried that it might now send email to your wife complaining about your wasteful ways.
“My water filter needs to be changed. Should I order it?”
“How much is it?”
“One hundred nineteen dollars.”
“Can you find me a better deal?”
“This is the best deal.
“Ok. Order it.”
(Later that day, you find a better deal on the filter at Amazon. But your refrigerator company wants to make money by selling you water filters. So it certainly is not interested in finding you a better deal.)
“Your wife’s birthday is next week. Should I order Gucci shoes for your wife? She really likes shoes.”
“How do you know that?”
“I checked her Facebook profile.”
“I will take care of the gift myself.” You are tempted to tell your refrigerator to order the shoes. But if your wife finds out about it, you will be in big trouble.
“My compressor might start making sounds like [...strange clanking sounds…]. Don’t be alarmed by that. I have already scheduled a service call and added it your calendar.”
“How do you know about that sound?”
“I follow tweets from my manufacturer.”
“I hope that you are not tweeting about us to grocery stores. Do you?”
“Please hurry up and try to leave early. Weather is expected to be bad and traffic is looking really bad.”
“Shouldn’t you have told me this before all the chit chat!! What should I do?”
“Please wait. Software update is in progress.…”
Here is what might happen that evening. You arrive at Wegmans, ten miles away from your home to get a good deal on the organic milk and tomatoes. You check email on your smart phone to download the grocery list. You have another email from your refrigerator! It is informing you that Costco has advertised a better sale. So you should go there to pick up tomatoes. The only problem is that the nearest Costco is eight miles away from where you are. You are worried that your refrigerator is “genius” enough to distinguish between the tomatoes from Wegmans and Costco and it might yell at you for wasting money when you try to put them in the refrigerator.
I did not say that you would necessarily enjoy having this conversation and getting emails from your refrigerator. I just said that technology is making it feasible.
By the way, the above scenario is not that far-fetched. Most homes in the United States have a good Internet connection. So, at least in theory, appliances at home can be connected to the Internet. Computing hardware is really inexpensive. So every appliance can have multiple CPUs without increasing its cost appreciably. If it needs more computing power, it can always connect to the cloud (e.g., Amazon Web Services) and do number crunching there.
The sensor cost has come down significantly as well. Every appliance can now have cameras, microphones, inertial measurement units, GPS, and bar code scanners. Significant progress is being made in chemical sensors, RFID readers, and thermo-mechanical sensors. These sensors should soon become pervasive as well.
There have been significant advances in human-machine interfaces as well. Voice recognition, face recognition, gesture recognition, natural language processing, and touch screens are expected to change the way we interact with household appliances. Very soon we should have technology to recognize human emotions as well.
Data mining, data fusion, and search techniques are able to analyze and integrate vast amount of data from different sources and turn it into useful information for human use.
In my opinion, future advances in appliances will come not only from improvements in component technologies, but also from the integration of a diverse set of technologies to better understand the context and provide value-added services to users.
Advances in technologies will enable many new features in products. Obviously, not all of them will be useful to all of us. The “genius” refrigerator described above had many useful features with the exception of few annoying quirks. But this is a problem with every new technology. I cannot imagine living without a computer and the Internet, but I have done enough screaming at my computer due to its counter-intuitive interface and its ability to freeze whenever I have an important deadline.
I am really excited about the possibility of the completely interconnected world. But I am a bit concerned how this will impact my life. Will “genius” appliances control my life, or will they make my life easier? Perhaps a bit of both!
I hope that my friends working in the human-robot interaction area can figure out a better mode of human-refrigerator interaction, before I buy a “genius” refrigerator.
In summary, I am looking forward to plugging my refrigerator to the Internet. I am also ready to receive emails from it, as long as I get only one email per week. I am having tough time staying in touch with human friends on Facebook. So I certainly do not want my refrigerator to become my Facebook friend.
Are you ready to receive emails from your refrigerator?
For the most part, the above dialog with a "genius" fridge is still a ways off in that it requires general purpose intelligent agents based on "organic computing" with "ambient intelligence" capabilities, so no worries about the virtual friends on FB soon. However, we can build equally useful and annoying tools to monitor the content of the fridge with RFID tags, collect their frequency of use etc, and send messages to notify or order with ease now. Much of this is already doable and if you want to spend the money, you can buy fridges with an Internet connection and a touch screen interface and display, as well as a text-to-speech interace, giving status, recommendations, etc. So, while us old folks may not see a "genius" fridge anytime soon that will carry on a conversation with us, our kids certainly will, and will think nothing of depending on it. As for me being ready to receive emails from my fridge goes, that's fine, I can handle that. But when the fridge, the bathroom scale, my smart house, and my mobile phone that knows everywhere I've been and with whom conspire against me to tell me what I need to do, I'm pulling the fuses on the whole system. :)
ReplyDeleteI might be ready to receive feedback from my fridge, but probably not via email or some other internet-connected system. More so than the busybody AI (or malicious sci-fi AI) scenario, I'd be concerned about the prospect of installing even more sensors in my home that could potentially transmit private details to malicious third parties or the public at large.
ReplyDeleteAlready, mobile devices collect GPS and other data which can be used to reconstruct a very detailed record of a person's life. In the past, this data was only available to law enforcement and national security via rigorous legal requirements; now any app we install on our phones can subtly gain access to this data and transmit it to some third-party server. The potential for malicious use is high.
I'm not sure what information our fridges could possibly gather that might threaten us (well, with the exception of Patrick Bateman's fridge), but here is one example: You and your family are out of town on vacation. Your fridge notices that it hasn't been opened in several days, and decides to announce this on your public Facebook wall. A data-savvy burglar, noticing this, deduces that you are out-of-town and that your home is unoccupied. By extracting GPS data from photos you've posted, the burglar can also determine your home address. This particular scenario might be unlikely (you have a housesitter, or you don't have a public FB wall, or you set your fridge to keep quiet on FB), but there are hundreds of other possible scenarios where your privacy and security could be compromised. As advanced sensing technologies, artificial intelligence, and networking become increasingly integrated, I would imagine the range of possible scenarios like the one above would grow exponentially.
This isn't to ignore the value of having intelligent agents like these in our homes - I'm sure they could make our lives more convenient and efficient. However, privacy and security issues should be addressed in tandem with these genius agents, not as an afterthought - as has been the case with Facebook, Android, and so many others.
Hello Dr. Gupta, nice to see your blog site and read your posts. The conversation with the "genius" refrigerator was entertaining :) and I guess some of it, if not all of it, is not too far-fetched.
ReplyDeleteI like the idea of the "genius refrigerator." I think when this milestone is reached the "genius" theme will increasingly manifest itself in virtually all the products that humans use.
ReplyDeleteVery entertaining post on something that would most probably become a reality pretty soon. I would like to point out though that the interaction or collaboration would not be only among humans and machines, but also among the machines themselves.
ReplyDeleteFor instance, all the workstations in a manufacturing plant might coordinate among themselves on how to go about producing a set of finished goods as efficiently as possible while preserving the quality specifications. This would give a big boost to low-cost manufacturing in the US.
Another pertinent example would be in the health care sector. Imagine that all the diagnostic devices can coordinate the sequence of tests that need to be conducted on patients to minimize wait times and maximize equipment utilization. Or, the devices can figure out the proper set of tests that are required along with the test conditions. Wouldn't our hospitals be able to provide better care at lower costs by aiding physicians with better decision making?
Thus, the possibilities are immense and go beyond data-driven human-machine interaction, though would also be a major component, particularly for our daily appliances (as the post hilariously explains) and assistive care facilities. There are both technical challenges and legitimate social and ethical concerns for sure. But the anticipated impact outweighs them, and I expect engineers and business leaders from various disciplines to work together on bringing "genius" devices to fruition.
I agree with Ashis. It may revolutionize the health care sector. However, that will require us to take a fresh look at the security of digital healthcare data storage as some researchers already raised concern about their vulnerability in this article: http://www.ragibhasan.com/publications/papers/vldb-sdm2007-v3.pdf
DeleteVery interesting and entertaining post. Al-tough, this technologies will solve many issues but it eventually chewing up thinking and working capability of human. In the future, our mind will have to depend on computers to think; for example, we feel handicap these days without internet or smart phone.
ReplyDeleteIt depends ... Obviously sensors, rules, communications, predictive science, are all converging. If they converge to make life more interesting and meaningful ... I am all for it. If they converge to take meaning away ... I am NOT for it. I can appreciate that talking refrigerators and other appliances, devices, etc can be of great utility to the aging (and memory declining) population. So there will be some bad for sure ... but opportunities for lots of good things. Still automated systems are not going to replace gourmet chefs :-)
ReplyDeleteElan