with a Rightwing Bias,” The Guardian, published December 16, 2016, www.theguardian.
com/technology/2016/dec/16/google-autocomplete-rightwing-bias-
algorithm-political-propaganda.
xv
Introduction: From Scarcity to Abundance
forms. This cognitive hacking impacts not only how we define things but also
what we associate them with and how positively or negatively we react to
them. And just as the alt-right Google hackers were a minority of Googlers
who were still able to impact the platforms, even small groups of cognitive
hackers can have a noticeable impact on our “mental map” (and the actions it
influences) if they can manipulate a large enough portion of the media we
consume around a particular issue.
Now before you start thinking “The Russians! The Russians!”, it’s worth not-
ing that a lot of cognitive “hacking” is inadvertent. (We will get to Russian,
state-sponsored influence operations later in this book, but they are but a
part of a much bigger problem.) That is, the structure of the media platforms
we use on a daily basis has an impact on the way we perceive the world.
For example, we know that users engage more with media that elicits strong
emotions, particularly if that emotion is anger.7 We also know that people are
increasingly consuming news via social media platforms,8 which deliver con-
tent based on an algorithm that seeks to maximize “engagement” (clicks, likes,
favorites, shares, angry emoji, etc.), and attempts to maximize engagement
often translate into maximizing strong emotions, especially anger. It’s no sur-
prise, then, that those who study politics and digital media observe an
increased polarization around political issues—strong emotions, diminished
nuance, and ideological positions clumped further from the center.
This natural draw of our attention to things that elicit strong emotions is
exacerbated by the competition for our attention, as the amount of data, and
data producers, is growing at mind-boggling rates. In 2013, scientists esti-
mated that 90% of the data in the entire world had been created in the previ-
ous two years alone. 9 And by 2015, Facebook alone had more users than the
entire internet did in 2008.10 Add to that the proliferation of TV channels in many households, and the portable prevalence of other digital screens, and
we have a society marinated in visual and aural media that our evolutionary
history simply did not prepare us for. As every bit of commercial media
competes for our attention, each one does so by trying to outdo the other.
7 Bryan Gardiner, “You’ll Be Outraged at How Easy It Was to Get You to Click on This
Headline,” Wired, published December 18, 2015, www.wired.com/2015/12/psychology-
of-clickbait/.
8 Jordan Crook, “62 Percent of U.S. Adults Get Their News from Social Media, Says
Report,” Techcrunch, published May 26, 2016, https://techcrunch.com/2016/05/26/
most-people-get-their-news-from-social-media-says-report/.
9 SINTEF, “Big Data, for Better or Worse: 90% of World’s Data Generated over Last Two
Years,” Science Daily, published May 22, 2013, www.sciencedaily.com/releases/
2013/05/130522085217.htm.
10 Paul Mason, Postcapitalism: A Guide to Our Future, New York: Farrar, Straus and Giroux
(2015), p. 7.
xvi
Introduction: From Scarcity to Abundance
The result is a digital arms race of cognitive manipulation, amplifying the
already dangerous, polarizing effects of attention-grabbing media.
This polarization is the result of more than just emotion, though. People tend
to engage with things that reinforce their existing point of view, what scien-
tists call confirmation bias—if I agree with it already, there must be something
good about it. This includes things that seem to confirm its accuracy—after
all, it feels good to discover new information that proves you were right all
along!—or to confirm the inaccuracy of the people who hold a different posi-
tion. We have an evolved (and important!) predisposition to dislike, and avoid,
circumstances that surprise us with our own faults. In a past era of regular
predator-prey interactions, such situations were often deadly. But when it
comes to political debate online, that evolved predisposition encourages us to
avoid situations where we can grow intellectually, ethically, and morally, both
as individuals and as a society. By avoiding nuance and succumbing to confir-
mation bias, we again move further from the center—and from those we
disagree with. We build our own personal echo chambers, all the while believ-
ing that we are basing opinions on a wider base of information, thanks to the
wonders of the world-wide web. Under this delusion, and without interven-
tion, society becomes more fragmented, and it is harder for people from dif-
fering ideologies to find common ground.
Though we live in the most information-rich, most connected era in human
history, the way our minds naturally deal with such information abundance
means our generation is probably the most susceptible to propaganda in
human history. But I stress naturally. Our genetic predispositions, hard won by
our ancestors as they fought off threats from every side, are in many ways a
tremendous gift. But in an era where most of us are not regularly threatened
by predators, but by people who would seek to use media to manipulate our
attention to influence our minds and our behavior, we are naturally equipped
with the wrong tools for dealing with these threats. We have a great instinct
for what to do when a predator surprises us from behind, or when we are low
on calories and confronted with a bounty of food to eat. But we have no
instinct for digital disinformation.
However, we have evolved the capacity to learn, to analyze, to reason, to
communicate, to persuade with logic. These are not our automatic instincts.
But if we direct our minds appropriately—if we master our own attention in
the right ways at the right times—we can resist this manipulation and build a
better society. But it takes work. And before that, it takes an understanding
of the problem.
That’s what we’ll explore in this book.
xvi
Introduction: From Scarcity to Abundance
Where We’re Headed
We’ll begin by unpacking the “propaganda problem.” Why is our conscious
attention so limited? How limited is it? And what are the implications for how
we engage information in the digital age? We’ll also explore the algorithmic
basis of social media and other content recommendation systems (like
Pandora, Netflix, and Amazon)—how they work technically, how they inter-
act with the strengths and limitations of our cognitive system, and how they
can be “hacked” to influence our thoughts and actions.
Then we’ll explore several case studies in how social media–based operations
have significantly impacted public consciousness—even public action. From
the Ferguson protests, organized and proliferated on Twitter and Vine, which
led to significant growth of the Black Lives Matter movement, to the coordi-
nated harassment of female game designe
rs and critics on Twitter known as
GamerGate, to the influence operations—both domestic and international—
at work in recent elections and political uprisings in Ukraine, Sweden, the
United States, Turkey, Brazil, the Philippines, Myanmar, Mexico, and Colombia.
We live in an era different from any we’ve ever seen before. We have the
tools, the information, and the interpersonal connections to accomplish
amazing things. But like the discovery of fire or the harnessing of nuclear
energy, our newfound digital capabilities also leave us vulnerable to human-
made disaster, the likes of which we’ve only begun to dream up. This power
can be used for good, used for evil, or treated as a toy. By too often making
digital technology a mere plaything, we open ourselves to those who may use
it for our undoing. But with a little education, a little regulation, and a lot of
care, we can turn this digital technology back into a beacon of hope.
It is my aim in this book to bring that education, foster that care, and, yes,
encourage that regulation. Because if we work together, and we get it right,
we can do some amazing things.
So let’s get started.
P A R T
I
The Propaganda
Problem
C H A P T E R
1
Pay Attention
How Information Abundance Affects the Way We
Consume Media
This chapter will explain the shift from an information economy to an attention
economy and lay out the implications for how information is created, shared,
and consumed on the internet. Having transitioned from a time of information
scarcity to information abundance, information is no longer a sufficiently
monetizable commodity to drive an economy. The focus of human attention
as the monetizable commodity in limited supply gives content recommendation
algorithms a pivotal place in our information landscape and our economy. This
chapter lays out the general economic, cognitive, and technological backdrop
for the emergence of those algorithms.
How Taste Is Made
What’s your favorite dessert?
Mine is probably a nice flourless chocolate cake. With raspberries, blackberries,
maybe some ice cream, … and, of course, something like an espresso,
Americano, or a full-bodied stout to drink—something that can cut through
that richness and cleanse the palate so that each bite is as wonderful as the
first.
I feel like I gained a few pounds just writing that paragraph. Why is it that
everything that tastes so good is so bad for you?!
© Kris Shaffer 2019
K. Shaf fer, Data versus Democracy,
https://doi.org/10.1007/978-1-4842-4540-8_1
4
Chapter 1 | Pay Attention
Or is it?
When I first studied evolutionary psychology in college, I remember learning
that humans generally evolved preferences for things that are good for us. 1
Things that are safe and help us live at least to the age of reproduction, so we
can pass on our genes. Humans who associate something with a positive
emotion will do that thing more often. Things that we associate with negative
emotions we tend to avoid. We seek out pleasure more than pain, and our
ancestors who took pleasure in things that helped them survive and reproduce,
well, they survived and reproduced. Those who took pleasure in things that
put them in danger, well, less of those genes were passed on to future
generations. The result is that the genes we twenty-first-century humans have
inherited generally make us feel pleasure around things that are good for the
species, but produce pain to warn us away from things that would jeopardize
our collective survival.
Of course, this emotional preference, this taste, for things that are biologically
and socially good for the species also applies to food. Foods that are good for
us tend to give us pleasure, while things that are bad for us tend to taste bad,
or trigger our gag reflex.
Wait a second. That seems backward. Didn’t I just lament about how
everything that tastes good is actually bad for us?!
There’s another piece to this evolutionary puzzle. Those evolved preferences,
called “adaptations,” helped our ancestors adapt to their environment. To
their environment. Let’s go back to that flourless chocolate cake. What are
the ingredients? Cocoa (of course), sugar, salt, butter, and lots and lots of
eggs. Nutritionally speaking, there’s not a lot of vitamins, minerals, or fiber—
things we hear a lot about today. Instead there are carbohydrates—“simple”
sugars, to be precise—fat, and protein. We hear a lot about these things from
modern nutritionists, too. They lead to overweight, obesity, heart disease,
diabetes, kidney disease, even gum disease and tooth decay. These ingredients
sit nearer the top of the food pyramid, things we should consume sparingly.
If we should consume them sparingly, then why did we evolve such a strong
taste for them? For those of us in the affluent West, 2000 calories a day are
easy to come by. But the genes that determine our culinary tastes today were
not selected in an era of such abundance. Those genes have changed little
since the Pleistocene period, tens to hundreds of thousands of years ago. 2
Our metaphorical palates were developed in a time before grocery stores,
fast food restaurants, fine dining, potlucks, refrigerators, freezers, even a time
pp. 524–25.
2For a detailed discussion, see Jerome H. Barkow, Leda Cosmides, and John Tooby (eds),
The Adapted Mind: Evolutionary Psychology and the Generation of Culture, New York, NY:
Oxford University Press (1992).
Data versus Democracy
5
before agriculture. Long before humans built a tractor, a plow, a saddle, even
a bow and arrow, our modern food preferences were being written into our
genetic code.
Carbohydrates, fat, protein. While today we highlight the dangers of consuming
them in excess, all of them are essential for our bodies to produce energy,
build muscle, and keep in good working order. All of them are relatively easy
to come by in a society with the technology to farm and hunt efficiently, not
to mention the technology to preserve food and all its nutritional value for
long stretches of time. But 150,000 years ago on the African savannah, our
ancestors had no such technology. Those biological essentials—carbs, fat,
protein—were scarce. Incredibly scarce. The human who took pleasure in
consuming them would seek them out and, presumably, be more likely to find
them and survive. And in an era when many humans lived on the brink of
starvation, those who did not seek out those precious ingredients often did
not survive to pass along their genes to us. It was the survivors who authored
our genetic code, and it is their taste for what was scarce but essential aeons
ago that governs our culinary predilections today.
And so we love things like flourless chocolate cake. Not because our ancestors
adapted a taste for chocolate cake, but because it’s t
he perfect combination
of all the essential, but scarce, things they evolved a taste for.
I mentioned before that our genetic code hasn’t changed much since the
Pleistocene era. That’s because humans living in an era of abundance don’t get
to write the genetic code! If most of humanity is no longer living on the cusp
of starvation, there is no food-related natural selection going on. Death drives
evolution, and genetic flaws get selected out, not the other way around. Take
allergies as an example: my spouse and I have severe enough allergies that had
we lived 100 years earlier, we both would have died from anaphylaxis before
we reproduced. Thanks to modern medicine, we survived! And we both
passed our allergy genes onto our children, leading to some severe medical
situations in just the first few years of their lives. Once we’re no longer on the
brink of extinction, those health advantages aren’t evolutionary advantages.
And so, what was advantageous in our species’ distant past still governs our
preferences today.
Supply and Demand: Why an Information
Economy Is No Longer Sustainable
So I hear some of you asking, “What does all of this have to do with big data,
‘fake news,’ and propaganda?”
Everything.
6
Chapter 1 | Pay Attention
Our ancestors’ taste for food was determined in a time of food scarcity. The
ingredients that were essential but scarce became the ingredients that were
most preferred and sought out. However, in an era of abundance, those
preferences don’t fit, especially when the same ingredients that are essential
in small to moderate amounts become harmful in large amounts. Our natural
instincts for food have become dangerous in an era of relative abundance.
The same can be said of how we consume information. The way we deal with
information is based on an evolutionary history—and an educational
experience—in which information was scarce or at the very least expensive.
Most of our cognitive system, which governs how our senses connect to our
brain, evolved before Homo sapiens even existed. 3 The bulk of our genetic
code was pretty well fixed long before the invention of writing, let alone the
printing press, mass media, radio, television, or the algorithmic news feed. 4