by Rose Eveleth
Eveleth is a writer and producer who explores how humans
tangle with science and technology. She’s the creator
and host of
Flash Forward, a
podcast about possible (and not so possible) futures.
by Zoë van Dijk
They see facial
and surveillance devices
Imagine you're taking an
online business class - the kind where you watch video lectures and
then answer questions at the end.
But this isn't a normal class, and you're not just watching the
watching you back...
Every time the facial
recognition system decides that you look bored, distracted, or
tuned out, it makes a note.
And after each lecture, it only asks you about content from those
This isn't a hypothetical system; it's a real one deployed by a
Nestor. And if you don't like the
sound of it, you're not alone. Neither do the actual students.
asked the man behind the system,
Marcel Saucet, how the students
in these classes feel about being watched, he admitted that they
didn't like it.
They felt violated and surveilled, he said, but he shrugged off any
implication that it was his fault.
"Everybody is doing
this," he told me. "It's really early and shocking, but we
cannot go against natural laws of evolution."
As a reporter who covers
technology and the future, I constantly hear variations of this line
as technologists attempt to apply the theory
Charles Darwin made famous in
biology to their own work.
I'm told that there is a progression of technology, a movement that
is bigger than any individual inventor or CEO. They say they are
simply caught in a tide, swept along in a current they cannot fight.
They say it inevitably leads them to,
They say we can't blame
these companies for the erosion of privacy or democracy or trust in
public institutions - that was all going to happen sooner or later.
"When have we ever
been able to keep the genie in the bottle?" they ask.
Besides, they argue,
people buy this stuff so they must want it.
Companies are simply
responding to "natural selection" by consumers. There is nobody to
blame for this, they say. It's as natural as gravity.
is creating a "smart" hairbrush...
Perhaps no one states this belief more clearly than inventor and
futurist Ray Kurzweil in his 2005 book
The Singularity Is Near:
acceleration of technology is the implication and inevitable
result of what I call the law of accelerating returns, which
describes the acceleration of the pace of and the exponential
growth of the products of an evolutionary process."
In fact, our world is
shaped by humans who make decisions, and technology companies are no
To claim that these devices are the result of some kind of
ever-improving natural process not only misunderstands how evolution
works, but it also suggests that everything from biological weapons
to fraudulent startups like
Juicero (the $400 machine that
squeezed juice out of packets) are necessary and natural.
While these "innovations" range from the dangerous to the silly,
they share a common thread: Nothing about them is "natural."
No natural process is
or a "smart" flip
or a "smart"
cryptocurrency from a photography company
internet-connected air freshener...
Evolution is a
terrible metaphor for technology
Technologists' desire to make a parallel to evolution is flawed at
its very foundation.
Evolution is driven by random mutation - mistakes, not plans. (And
while some inventions may indeed be the result of mishaps, the
decision of a company to patent, produce, and market those
inventions is not.)
Evolution doesn't have meetings about,
the customer base...
Evolution doesn't patent things or do focus
groups. Evolution doesn't spend
millions of dollars lobbying Congress
to ensure that its plans go unfettered.
In some situations, even if we can't literally put a technological
genie back in a bottle, we can artificially intervene to make sure
the genie plays by specific rules.
is driven by mistakes,
There are clear laws about what companies can and can't do in the
The FDA 'ensures' drugs are
tested for 'efficacy and safety' before they can be sold.
The USDA 'ensures'
new food research is done with 'care.'
We don't let anybody
frack or drill for oil or build nuclear power plants wherever
We don't let just
anybody make and sell cars or airplanes or guns.
So the assertion that
technology companies can't possibly be shaped or restrained with the
public's interest in mind is to argue that they are fundamentally
different from any other industry.
persists in part because Americans cannot resist the allure of
In many ways, the American tech mogul blaming some invisible arc of
innovation for his choices is simply an echo of the very ethos that
progress at all
The men and women who
colonized America largely saw progress as a march, a continual
straight path, forward at all times, never to be questioned or
Benjamin Franklin (himself an inventor) once
"The rapid progress
true science now makes occasions my regretting sometimes that I
was born so soon. It is impossible to imagine the height to
which may be carried, in a thousand years, the power of man over
So to suggest a slowing -
or perhaps even (god forbid) a reverse - seems antithetical to the
fabric of America.
Uber engineer Anthony
happened doesn't really matter. You don't need to know that
history to build on what they made. In technology, all that
matters is tomorrow," he is echoing the sentiments of our
"These arguments for corporate supremacy and technological
progress are so deeply connected in the American mind,"
says Luke Stark, a researcher
who studies behavior and computing at Microsoft Research
"It's really tied to the whole American manifest destiny and the
broader history of American settler colonialism."
The attitude is that we
must march on, forward, westward, no matter what stands in our way.
Going back is simply not an option.
Even those afraid of all-seeing corporations argue that instead of
pressing pause, we, the consumers, must simply move forward more
"What I try to focus
on is not to try to stop the march of technological progress,"
Yuval Noah Harari, a historian and the author of
told Time magazine.
"Instead, I try to
tied to the whole
American manifest destiny"...
This endless, punishing race in the name of "progress" is often what
drives consumer behavior, too.
Despite the "American dream" - security, safety, prosperity - being
more and more out of reach for everyday Americans, the idea that
it's just around the corner drives people to purchase these
If you have the newest app, people think their lives will be easier,
you'll have more free time, more quality time. Commercials promise
more backyard barbecues under sparklers and birthday surprise
parties facilitated by internet-connected light bulbs.
And when we buy the products, tech companies take that as a green
light to continue on their "inevitable" path, inching ever toward a
world where Amazon knows exactly what you're doing, thinking,
feeling - perhaps even before you do.
"It's all a loop,"
"It's weird. That's what puts people in this bind. They think
they should be able to have it all. They can't, and technology
is a kind of prophylactic to cope with this stuff."
Perhaps the most telling
place this kind of loop shows up is in the high-tech baby device
internet-connected baby monitors, an anklet that keeps track of
your infant's every move, heartbeat, and temperature
In a country with an
ever-eroding social safety net, parents are sold these surveillance
devices under the guise of care and love, with a healthy dash of
As Luke Stark
writes in his paper on consumers
who surveil their loved ones,
particular mothers) have long been reminded by advertisers that
their children are at risk - from external threats, poor
nutrition, social exclusion, the mother's own failures as a
parent - and that their surest route to protecting their
children, and ameliorating their (newly provoked) anxiety about
these dangers, is consumption."
Buying this device, even
if the details are a bit creepy, shows that you care, that you're a
And because parents are shamed and nudged into buying these tracking
devices, more and more of them pop up on the market.
It's these purchases that technologists equate to "natural
selection," but it's nearly impossible for most people to opt out of
a lot of these arrangements.
power comes great responsibility
Often consumers don't have much power of selection at all.
Those who run small
businesses find it
nearly impossible to walk away from
Facebook, Instagram, Yelp, Etsy, even Amazon.
Employers often mandate that their workers use certain apps or
systems like Zoom, Slack, and Google Docs.
"It is only the
hyper-privileged who are now saying, 'I'm not going to give my
kids this,' or, 'I'm not on social media'," says
Rumman Chowdhury, a data
scientist at Accenture.
"You actually have to be so comfortable in your privilege that
you can opt out of things."
And so we're left with a
tech world claiming to be driven by our desires when those decisions
aren't ones that most consumers feel good about.
There's a growing chasm between how everyday users feel about the
technology around them and how companies decide what to make. And
yet, these companies say they have our best interests in mind.
We can't go back, they say. We can't stop the "natural evolution of
But the "natural evolution of technology" was never a thing to begin
with, and it's time to question what "progress" actually means...