The dangers in smart technologies

Many smart technologies are heading in a disturbing direction. A number of thinkers in Silicon Valley see these technologies as a way not just to give consumers new products that they want but to push them to behave better. Sometimes this will be a nudge; sometimes it will be a shove. But the central idea is clear: social engineering is disguised as product engineering.

But Morozov says there is reason to worry about this approaching revolution. As smart technologies become more intrusive, they risk undermining our autonomy by suppressing behaviors that someone somewhere has deemed undesirable. Smart forks inform us that we are eating too fast. Smart toothbrushes urge us to spend more time brushing our teeth. Smart sensors in our cars can tell if we drive too fast or brake too suddenly.

True these devices can give us useful feedback, but they can also share everything they know about our habits with institutions whose interests are not identical with our own. Insurance companies already offer significant discounts to drivers who agree to install smart sensors in order to monitor their driving habits. How long will it be before customers can't get auto insurance without surrendering to such surveillance? And how long will it be before the self-tracking of our health (weight, diet, steps taken in a day) graduates from being a recreational novelty to a virtual requirement?

How can we avoid completely surrendering to the new technology?

The key is learning to differentiate between "good smart" and "bad smart."

Devices that are "good smart" leave us in complete control of the situation and seek to enhance our decision-making by providing more information. For example: An Internet-jacked kettle that alerts us when the national power grid is overloaded doesn't prevent us from boiling yet another cup of tea, but it does add an extra ethical dimension to that choice. Likewise, a grocery cart that can scan the bar codes of products we put into it, informing us of their nutritional benefits and country of origin, enhances—rather than impoverishes—our autonomy.

Technologies that are "bad smart," by contrast, make certain choices and behaviors impossible. Smart gadgets in the latest generation of cars—breathalyzers that can check if we are sober, steering sensors that verify if we are drowsy, facial recognition technologies that confirm we are who we say we are—seek to limit, not to expand, what we can do. This may be an acceptable price to pay in situations where lives are at stake, such as driving, but we must resist any attempt to universalize this logic.

The most worrisome smart-technology projects start from the assumption that designers know precisely how we should behave, so the only problem is finding the right incentive. A truly smart trash bin would thus make us reflect on our recycling habits and contribute to conscious deliberation—say, by letting us benchmark our usual recycling behavior against other people in our demographic. There are many other contexts in which smart technologies are unambiguously useful and even lifesaving. Smart belts that monitor the balance of the elderly and smart carpets that detect falls seem to fall in this category.

But the problem with many smart technologies is that their designers, in the quest to root out the imperfections of the human condition, seldom stop to ask how much frustration, failure and regret is required for happiness and achievement to retain any meaning.

It's great when the things around us run smoothly, but it's even better when they don't do so by default. That, after all, is how we gain the space to make decisions—many of them undoubtedly wrongheaded—and, through trial and error, to mature into responsible adults, tolerant of compromise and complexity.

To grasp the intellectual poverty that awaits us in a smart world, look no further than recent blueprints for a "smart kitchen". Once we step into this magic space, we are surrounded by video cameras that recognize whatever ingredients we hold in our hands. Tiny countertop robots inform us that, say, arugula doesn't go with boiled carrots or that lemon grass tastes awful with chocolate milk. This kitchen might be smart, but it's also a place where every mistake, every deviation from the master plan, is frowned upon. It's a world that looks more like a Taylorist factory than a place for culinary innovation. Rest assured that lasagna and sushi weren't invented by a committee armed with formulas or with "big data" about recent consumer wants.

The fact is that it is creative experimentation that propels our culture forward. Our stories of innovation tend to glorify the breakthroughs and edit out all the experimental mistakes doesn't mean that mistakes play a trivial role. As any artist or scientist knows, without some protected, even sacred space for mistakes, innovation would cease. An inventor’s path is chorused with groans, riddled with fist-banging and punctuated by head scratches. Stumbling upon the next great invention in an “ah-ha!” moment is a myth. It is only by learning from mistakes that progress is made.

A case in point is the story of James Dyson, the inventor of the famous vacuum cleaner. From cardboard and duct tape to ABS polycarbonate, it took 5,127 prototypes and 15 years to get it right. And, even then there was more work to be done. His first vacuum, DC01, went to market in 1993. They are now up to DC35 now, having improved with each iteration. It’s a never-ending process that is enormously rewarding, and endlessly frustrating. Or look at Edison who famously said, “I have not failed. I’ve just found 10,000 ways that won’t work.” Those 10,000 detours resulted in the Dictaphone, mimeograph, stock ticker, storage battery, carbon transmitter and his joint invention of the light bulb. In the end, 10,000 flops fade into insignificance alongside Edison’s 1,093 patents.

The ability to learn from mistakes — trial and error — is a valuable skill we learn early on. Recent studies show that encouraging children to learn new things on their own fosters creativity. Direct instruction leads to children being less curious and less likely to discover new things. Punishing mistakes doesn’t lead to better solutions or faster results. It stifles invention. By fostering an environment where failure is embraced, even those of us far from our student days have the freedom to make mistakes — and learn from them still. No one is going to get it right the first time. Instead of being punished for mistakes along the way, we need to learn from them. 

With "smart" technology in the ascendant, it will be even harder to resist the allure of a frictionless, problem-free future. When Eric Schmidt, Google's executive chairman, says that "people will spend less time trying to get technology to work…because it will just be seamless," he is not wrong: This is the future we're headed toward. But not all of us will want to go there. As James Dyson and Edison prove there are many a detour in the journey of innovation which may be even more important than a friction less path of discovery.

A better smart-design paradigm would happily acknowledge that the task of technology is not to liberate us from problem-solving. Rather, we need to enroll smart technology in helping us with problem-solving. What we want is not a life where friction and frustrations have been carefully designed out, but a life where we can overcome the frictions and frustrations that stand in our way.

Designers of smart technologies in the future need to take stock of the complexity and richness of the lived human experience—with its gaps, challenges and conflicts—so that their inventions will be destined for changing history. They should embrace failures on the path rather than seek to eliminate it.