MORE ABOUT THIS BOOK
Our First Language
I am a face reader. And so are you.
This skill is so critical to our survival that we have dedicated brain circuits for processing facial information. Imaging can pinpoint the exact place in the brain where our face-reading network lies. Scientists are still exploring how these neural circuits develop, but most believe that we’re born face readers, at least to some extent. From the moment of birth—before its eyes can even focus—a baby is wired to seek out faces. Within hours, a baby can tell its mother’s face from a female stranger’s, and in just a few months it grows expert at identifying sex, race, emotions, and other basic traits.
This face-to-face connection is the child’s first language. It is so potent that newborns are drawn even to facelike configurations, a row of two (or three or more) objects above a single one. But they do not respond to the same configuration—or even a human face—upside down.
I once had a funny experience at a corporate retreat, where I’d been invited to address some two hundred software engineers. I had to laugh when the executive director warned that they’d introduce me as a “life coach,” since “face reader” sounded too esoteric. But in fact I was planning to talk about life—about what our faces (and those of others) reveal about the way we live.
I began my speech by drawing a large oval on the whiteboard, with a short vertical line in the middle and a horizontal one below it. Then I stood back and invited comments.
Minutes passed, with everyone silent. I was getting nervous, but then, at last, a hand shot up.
“What do you see?” I asked.
“The OFF button on the remote control,” the guy said.
What I had drawn, of course, was not an OFF button but the outline of a face, with the vertical line representing the nose and the horizontal one the mouth. But since my drawing didn’t have eyes, no one could guess what it represented. That’s how deeply we’re imprinted with the two-objects-above-a-single-one model of the human countenance. Our instant interpretation of that image as a face is the reason that emojis—just a couple of dots and a line—can convey a world of meaning.
* * *
In time, when a newborn can distinguish features, it will literally see the love in its mother’s eyes, her pupils enlarged with emotion. That’s why teddy bears have black button eyes—they’re all pupil—and probably why, as studies show, men are attracted to women with big pupils. Centuries ago, to be alluring, Italian women would dilate their pupils with the extract of a plant that got the name “beautiful lady,” or belladonna.
This attraction to large pupils is primal and unconscious. As we grow up, we consciously gather data on others from multiple sources: their tone of voice, words, body language, hairstyles and clothing, and even the context in which we see them. We may be less aware of our face-reading powers, but the brain circuits producing them continue to fire, sparking gut feelings and intuitions.
In fact, research shows that fusiform gyrus, the specific area of the brain triggered by faces, continues to develop from infancy on.1 Other complex visual-processing systems, like place recognition, are more static. As we mature, we grow to recognize the much wider range of faces necessary to navigate our broadening social networks. We also get increasingly better at distinguishing similar faces. By adulthood, we range from limited to gifted in our ability to identify faces, but on average, we remember and recognize some 20 percent of the people we’ve seen.
Roughly 2.5 percent of Americans fall at the low end of the identification spectrum because of “prosopagnosia,” or face blindness, which may be inborn or acquired though trauma or diseases like Alzheimer’s.2 People with prosopagnosia have trouble distinguishing familiar faces, though they can readily identify other objects. The title character in the famous book by neurologist Oliver Sacks, The Man Who Mistook His Wife for a Hat, has a form of prosopagnosia. He identifies his wife by her voice and other people in his life, like his brother, by such specific features as big teeth. In middle age, Sacks himself, who’d never recognized his own reflection in a mirror, finally confronted his own face blindness.
The opposite of face blindness is “super-recognition,” an extraordinary gift for identifying faces. The estimated 2 percent of people who are super-recognizers may memorize and recognize as many as 80 percent of the faces they see.3 New Scotland Yard employs an elite super-recognizer squad to catch criminals by studying footage from London’s million-plus surveillance cameras. Finding a needle in the haystack—a person filmed at a crime scene who has, say, a mug shot in the system—requires the uncanny ability to pick out a face on grainy, low-resolution video and match it with that of the known offender. Detective Chief Inspector Mick Neville, who created the unit, calls face recognition the “third revolution” of forensics, after fingerprints and DNA.
His squad has had surprising success. After civil unrest in 2011, computer face-recognition software could spot only one rioter from 200,000 hours of footage, while a single super-recognizer identified 190. According to published statistics, 73 percent of the squad’s identifications have led to criminal charges. But what troubles rights advocates is that 13 percent, even after independent review by a second super-recognizer, have led to false arrests.4
* * *
An inevitable problem in facial analysis—whether human or electronic—is that it’s only as accurate as its database. Humans, from birth onward, can best identify and read faces of the ethnicity we see most. White people tend to be better at recognizing white faces, black people at recognizing black faces, Asians at recognizing Asians, and so on.
Face-recognition technology, though vastly improved since 2011, shows the same bias. In a 2018 study of the capacity to read gender from a photo, three leading systems—those of Microsoft, IBM, and Megvii of China—detected faces of white or light-skinned men with an error rate of less than 1 percent. But when it came to spotting darker female faces, Microsoft’s system had an error rate of 21 percent, while IBM’s and Megvii’s rates approached 35 percent.5
Also in 2018, the American Civil Liberties Union (ACLU) tested Amazon’s system, used by some police departments, by running photos of all members of Congress against 25,000 published mug shots. No less than twenty-eight lawmakers were identified as criminals. Most were African American or Hispanic, including such well-known figures as congressmen John Lewis of Georgia and Bobby L. Rush of Illinois. The finding led the ACLU to condemn face-recognition technology as “flawed, biased, and dangerous.”6
Even so, at this point face-recognition technology is largely unregulated. An estimated 117 million Americans, disproportionately nonwhite, appear in law enforcement databases.7 The FBI has its own database of faces, which is reportedly smaller and less sophisticated than Facebook’s cache of more than 2 billion images.8 As government agencies and private companies—without restriction and oversight—amass data from millions of unsuspecting people’s faces, the vast potential for abuse of the technology is obvious.
Fear of such abuse prompted two Stanford University scientists to test whether a facial-analysis program could guess people’s sexual orientation. From public dating sites, they collected 35,000 photos of self-described gay and heterosexual people, all white, and had a widely used algorithm assess them for subtle differences. They then plugged in images of random faces and asked the computer to judge their sexual orientation. The results were chilling: the algorithm picked the correct sexual orientation 71 percent of the time for women and 81 percent of the time for men. When the computer was shown five images of a person, instead of one, the accuracy rose to 83 percent for women and 91 percent for men.9
Scientists at Shanghai Jiao Tong University applied the same method, using 1,856 images of men aged eighteen to fifty-five, to distinguish criminals from law-abiding citizens. Their algorithm detected the criminals among the randomly chosen faces with an accuracy of 89.5 percent and—especially relevant for our purposes—pinpointed the specific features that made them different. These traits included a curvature of the upper lip that was, on average, 23 percent greater than in the law-abiding group, a 6 percent shorter distance between the inner corners of the eyes, and an angle that was 20 percent narrower between two lines drawn from the tip of the nose to the corners of the mouth.10
The Chinese study was small and based on an algorithm created for Asian faces. Like the Stanford one, it raises frightening issues of invasion of privacy that are beyond the scope of this book. But because they have so many applications—airport security, law enforcement, medical diagnosis, and more—facial recognition and analysis are among the hottest fields in technology, growing faster than we can easily control or even imagine.
* * *
These new technologies have sparked a resurgence of interest in our human capacity for face reading. Dismissed in the West as a pseudoscience for most of the twentieth century, it is now studied at such prominent universities as the University of California at Berkeley, New York University, Stanford, and Princeton, to name a few. Some of these researchers still reject face reading as physiognomy, the practice of judging character from faces that dates back thousands of years, as we’ll discuss in chapter 3, “Face Reading in History.”
Critics of face reading attribute traits that humans, as opposed to machines, read into faces as stereotyping. But even the most skeptical acknowledge that we’re hardwired to assess people instantly from their appearance. As scientist Alexander Todorov of Princeton, an authority on first impressions, writes, “Seeing a face for less than one-tenth of a second is sufficient to make up our minds.…”11
Over the past few decades, dozens of studies have demonstrated that, when shown the image of a face—whether human or computer generated—most subjects will define its personality the same way. To make their experiments uniform, psychologists usually have subjects rate personalities on what they call the “big five” traits: extraversion, agreeableness, conscientiousness, emotional stability/neuroticism, and openness to experience. Consensus on the personalities that faces express shows up even in research on children.
For example, in a pair of 2014 studies, scientists discovered that three- to four-year-olds could consistently distinguish between faces that were “nice” and “mean,” “strong” and “not very strong,” and “smart” and “not very smart.” Averaging the responses across categories, 72 percent of the three- to four-year-olds agreed on the traits the faces revealed. Among five- and six-year-olds, the degree of agreement was higher (81 percent); and among seven- to ten-year-olds, it rose to 88 percent. Among adults, the degree of agreement was nearly the same as among the seven- to ten-year-olds (89 percent).12
Still, many scientists argue that personality can’t be gleaned from the face alone. For one thing, they say, faces change from moment to moment, as we’ll discuss in chapter 7, “The World of Expression.” For another, they warn of “overgeneralization,” or making too much of detectable cues. One confusing cue they cite is attractiveness, which can have a halo effect, making the person seem lovable, kind, and so on simply because of being beautiful. While we are all drawn to attractive people (see “The Perfect Face,” which lists traits that research shows make people attractive), in face reading the concept of beauty is not especially relevant.
Instead, what we look for is an overall picture. We know from research on face blindness that the human brain and a computer algorithm process faces differently. Rather than break out features, the brain considers the face holistically, as one image. That’s a major reason why this book is not a how-to guide, offering lists of facial characteristics with assigned meanings.
A computer might have a database of images of, say, a thousand criminals’ faces and sift through them to see what they have in common. But humans confront each new face in real time, taking in its fixed features, like the eyes, nose, and mouth; watch it shift through a range of microexpressions; and draw a conclusion, usually unconsciously, through intuition.
* * *
The Perfect Face
Is there such a thing as the perfect face? Many studies have been done on what makes faces attractive. University of Toronto psychologists Daniel E. Re and Nicholas O. Rule, surveying the research, highlighted certain aspects that appeal across cultures, as well as probable reasons:
Even skin tone and texture, likely because they suggest youth and health. Study subjects shown a face with color variations smoothed out judged the person to be five years younger, and one with wrinkles smoothed out to be as much as fifteen years younger.Skin color, with red tones (but not too much red) suggesting well-oxygenated blood and cardiovascular fitness and yellow tones (in Causcasians, Asians, and Africans) implying good immune function. Yellow skin tones come from carotenoid compounds in fruits and vegetables, which are depleted by illness. One 2012 study found that eating three to four servings of fruits and vegetables per day for six weeks increased carotenoids enough to make people look more attractive.Averageness, an observation first made in 1878 (and more recently tested using computer-generated composites), possibly because faces deviating wildly from the norm might indicate poor health. But attractiveness is not necessarily the same as beauty. The most attractive faces have some special eye-catching feature.Symmetry, but not two sides of the face that are mirror-images, which would look weird and unnatural. People across cultures prefer faces with their basic structures aligned, if not perfectly, though the link to underlying health is not as certain.Fleshiness (adiposity) of the face reflects a person’s BMI or Body Mass Index. People tend to prefer faces reflecting a BMI of approximately 20, which is right in the middle of the normal range, to faces that look underweight or overweight.Femininity and masculinity have been much studied as factors in attractiveness, with uncertain results. Scientists believe that the hormone estrogen, governing the development of women’s faces, gives them the features usually defined as feminine: large eyes, full lips, a small and pointed chin, and high cheekbones. More feminine features imply higher levels of estrogen, suggesting fertility, the theory holds. Subjects of studies, when asked to make a neutral face more attractive, usually enhance the femininity of its features.Masculine features, established by testosterone, include a prominent brow ridge, high cheekbones, and a square jaw. But men’s faces—unlike women’s, in which femininity is correlated with attractiveness—are not necessarily most appealing when they appear most masculine. Some studies show that women prefer more feminized features in both Asian and Caucasian men, others have shown no preference, and still others have shown that women’s preferences may vary by context. Researchers speculate that a masculine appearance, signaling high testosterone, may be linked to undesirable personality traits, including aggressive behavior, infidelity, and lack of interest in parenting. Attraction to masculine looks was highest in areas where the standard of living was lowest. There may be other, as yet undetermined, factors that lend allure to a male face.13
For a face reader, the perfect face, male or female, is one that conveys authenticity. In chapter 12, “Perception,” I’ll describe my own vision of the perfect face: the “enlightened” visage.
* * *
All this new scientific interest is simply, if powerfully, validating what human beings have known for thousands of years. As psychology professors Ran Hassin and Yaacov Trope write, in an account of their studies of faces published in the Journal of Personality and Social Psychology, “It seems only reasonable to assume that if the mind devotes special brain resources to processing faces, it will try to extract as much information from the face as it can.”14
I’ve spent my adult life, as I’ll discuss in chapter 2, studying the ancient systems that have tried to “extract information” by making our innate, automatic assessments of faces more conscious. Every face is a complex map of life, with inborn traits, characteristics developed through experience, talents, reflections of health, signs of shifting emotional weather, and more. By telling the stories I’ve observed in faces and explaining my interpretations, I hope to help you recognize your own intuitive powers and begin, more consciously, to develop them.
By awakening your instinctive capacity to read faces, you can achieve a deeper understanding of others—and, even more, yourself.
Copyright © 2019 by Eric Standop