Computer maps 21 distinct emotional expressions -- even 'happily disgusted'

Study more than triples the number of facial expressions researchers can use to track the origins of emotions in the brain

2014-03-31

(Press-News.org) COLUMBUS, Ohio—Researchers at The Ohio State University have found a way for computers to recognize 21 distinct facial expressions—even expressions for complex or seemingly contradictory emotions such as "happily disgusted" or "sadly angry."

In the current issue of the Proceedings of the National Academy of Sciences, they report that they were able to more than triple the number of documented facial expressions that researchers can now use for cognitive analysis.

"We've gone beyond facial expressions for simple emotions like 'happy' or 'sad.' We found a strong consistency in how people move their facial muscles to express 21 categories of emotions," said Aleix Martinez, a cognitive scientist and associate professor of electrical and computer engineering at Ohio State. "That is simply stunning. That tells us that these 21 emotions are expressed in the same way by nearly everyone, at least in our culture."

The resulting computational model will help map emotion in the brain with greater precision than ever before, and perhaps even aid the diagnosis and treatment of mental conditions such as autism and post-traumatic stress disorder (PTSD).

Since at least the time of Aristotle, scholars have tried to understand how and why our faces betray our feelings—from happy to sad, and the whole range of emotions beyond. Today, the question has been taken up by cognitive scientists who want to link facial expressions to emotions in order to track the genes, chemicals, and neural pathways that govern emotion in the brain.

Until now, cognitive scientists have confined their studies to six basic emotions—happy, sad, fearful, angry, surprised and disgusted—mostly because the facial expressions for them were thought to be self-evident, Martinez explained.

But deciphering a person's brain functioning with only six categories is like painting a portrait with only primary colors, Martinez said: it can provide an abstracted image of the person, but not a true-to-life one.

What Martinez and his team have done is more than triple the color palette—with a suite of emotional categories that can be measured by the proposed computational model and applied in rigorous scientific study.

"In cognitive science, we have this basic assumption that the brain is a computer. So we want to find the algorithm implemented in our brain that allows us to recognize emotion in facial expressions," he said. "In the past, when we were trying to decode that algorithm using only those six basic emotion categories, we were having tremendous difficulty. Hopefully with the addition of more categories, we'll now have a better way of decoding and analyzing the algorithm in the brain."

They photographed 230 volunteers—130 female, 100 male, and mostly college students—making faces in response to verbal cues such as "you just got some great unexpected news" ("happily surprised"), or "you smell a bad odor" ("disgusted"). In the resulting 5,000 images, they painstakingly tagged prominent landmarks for facial muscles, such as the corners of the mouth or the outer edge of the eyebrow. They used the same method used by psychologist Paul Ekman, the scientific consultant for the television show "Lie to Me." Ekman's Facial Action Coding System, or FACS, is a standard tool in body language analysis.

They searched the FACS data for similarities and differences in the expressions, and found 21 emotions—the six basic emotions, as well as emotions that exist as combinations of those emotions, such as "happily surprised" or "sadly angry."

The researchers referred to these combinations as "compound emotions." While "happily surprised" can be thought of as an expression for receiving unexpected good news, "sadly angry" could be the face we make when someone we care about makes us angry.

The model was able to determine the degree to which the basic emotions and compound emotions were characterized by a particular expression.

For example, the expression for happy is nearly universal: 99 percent of the time, study participants expressed happiness by drawing up the cheeks and stretching the mouth in a smile. Surprise was also easily detected: 92 percent of the time, surprised participants opened their eyes wide and dropped their mouth open.

"Happily surprised" turned out to be a compound of the expressions for "happy" and "surprised." About 93 percent of the time, the participants expressed it the same way: with the wide-open eyes of surprise and the raised cheeks of happiness—and a mouth that was a hybrid of the two—both open and stretched into a smile.

The computer model also gives researchers a tool to understand seemingly contradictory emotions. "Happily disgusted," for instance, creates an expression that combines the scrunched-up eyes and nose of "disgusted" with the smile of "happy."

Martinez explained this emotion as "how you feel when you watch one of those funny 'gross-out' movies and something happens that's really disgusting, but you just have to laugh because it's so incredibly funny."

While the model is meant to be a tool for basic research in cognition, Martinez can foresee potential applications in the treatment of disorders that involve emotional triggers, such as PTSD, or a lack of recognition of other people's emotions, such as autism.

"For example, if in PTSD people are more attuned to anger and fear, can we speculate that they will be tuned to all the compound emotions that involve anger or fear, and perhaps be super-tuned to something like 'angrily fearful'? What are the pathways, the chemicals in the brain that activate those emotions? We can make more hypotheses now, and test them," he said. "Then eventually we can begin to understand these disorders much better, and develop therapies or medicine to alleviate them."

Coauthors on the study included doctoral students Shichuan Du and Yong Tao. The work was funded in part by the National Institutes of Health.

INFORMATION:

Contact: Aleix Martinez, (614) 688-8225; Martinez.158@osu.edu

Written by Pam Frost Gorder, (614) 292-9475; Gorder.1@osu.edu

Editor's note: Images to accompany the story are available from Pam Frost Gorder.

[Attachments] See images for this press release:

ELSE PRESS RELEASES FROM THIS DATE:

Using your loaf to fight brain disease

2014-03-31

A humble ingredient of bread – baker's yeast – has provided scientists with remarkable new insights into understanding basic processes likely involved in diseases such as Parkinson's and cancer. In a new study published today (Monday March 31) in the prestigious journal PNAS (Proceedings of the National Academy of Science), the team from Germany, Leicester, and Portugal detail a new advance – describing for the first time a key feature in cellular development linked to the onset of these devastating diseases. The research team is from the University Medical Center Goettingen, ...

Can antibiotics cause autoimmunity?

2014-03-31

(PHILADELPHIA) -- The code for every gene includes a message at the end of it that signals the translation machinery to stop. Some diseases, such as cystic fibrosis and Duchenne muscular dystrophy, can result from mutations that insert this stop signal into the middle of an essential gene, causing the resulting protein to be truncated. Some antibiotics cause the cell's translation machinery to ignore the stop codons and are therefore being explored as a potential therapy for these diseases. But new research reported online in Proceedings of the National Academy of Sciences ...

Self-healing engineered muscle grown in the laboratory

2014-03-31

VIDEO: After veins grow into the implanted engineered muscle fibers, blood cells can be seen traveling through them, sustaining and nourishing the new tissue. Click here for more information. DURHAM, N.C. -- Biomedical engineers have grown living skeletal muscle that looks a lot like the real thing. It contracts powerfully and rapidly, integrates into mice quickly, and for the first time, demonstrates the ability to heal itself both inside the laboratory and inside an animal. The ...

Ancient whodunit may be solved: The microbes did it!

2014-03-31

Evidence left at the crime scene is abundant and global: Fossil remains show that sometime around 252 million years ago, about 90 percent of all species on Earth were suddenly wiped out — by far the largest of this planet's five known mass extinctions. But pinpointing the culprit has been difficult, and controversial. Now, a team of MIT researchers may have found enough evidence to convict the guilty parties — but you'll need a microscope to see the killers. The perpetrators, this new work suggests, were not asteroids, volcanoes, or raging coal fires, all of which have ...

Experimental cancer drug reverses schizophrenia in adolescent mice

2014-03-31

Johns Hopkins researchers say that an experimental anticancer compound appears to have reversed behaviors associated with schizophrenia and restored some lost brain cell function in adolescent mice with a rodent version of the devastating mental illness. The drug is one of a class of compounds known as PAK inhibitors, which have been shown in animal experiments to confer some protection from brain damage due to Fragile X syndrome, an inherited disease in humans marked by mental retardation. There also is some evidence, experts say, suggesting PAK inhibitors could be used ...

Possible explanation for human diseases caused by defective ribosomes

2014-03-31

Ribosomes are essential for life, generating all of the proteins required for cells to grow. Mutations in some of the proteins that make ribosomes cause disorders characterized by bone marrow failure and anemia early in life, followed by elevated cancer risk in middle age. These disorders are generally called "ribosomopathies." How can ribosomopathies first appear as diseases caused by too few cells, but later turn into diseases caused by too many cells? This paradox has puzzled the scientific community for years. A new study, which uses a genetic approach to examine ...

Oxygen depletion in the Baltic Sea is 10 times worse than a century ago

2014-03-31

This news release is available in German. After several years of discussions, researchers from Aarhus University (Denmark), Lund University (Sweden) and Stockholm University (Sweden) have determined that nutrients from the land are the main cause of widespread areas of oxygen depletion. The results were published on 31 March in the prestigious American journal Proceedings of the National Academy of Sciences. Nutrients are the villain The deepest areas of the Baltic Sea have always had a low oxygen content. The inflow of fresh water is actually limited by low thresholds ...

Scientists understand how E. coli clone has become globally distributed

2014-03-31

Scientists have for the first time come closer to understanding how a clone of E. coli, described as the most important of its kind to cause human infections, has spread across the world in a very short time. E. coli clone ST131 is one of the leading causes of urinary tract and blood stream infections and has crossed the globe at a rapid rate. Worryingly, members of this clone are becoming more resistant to antibiotics. As an indication of scale, more than half of all women will suffer a urinary tract infection at least once in their lives. An international team of scientists, ...

Researchers announce first phononic crystal that can be altered in real time

2014-03-31

Using an acoustic metadevice that can influence the acoustic space and can control any of the ways in which waves travel, engineers have demonstrated, for the first time, that it is possible to dynamically alter the geometry of a three-dimensional colloidal crystal in real time. The colloidal crystals designed in the study, called metamaterials, are artificially structured materials that extend the properties of existing naturally occurring materials and compounds. The research by academics from the University of Bristol's Department of Mechanical Engineering is published ...

Weaker gut instinct makes teens open to risky behavior

2014-03-31

DURHAM, N.C. -- Making a snap decision usually means following your initial reaction -- going with your gut. That intuitive feeling sprouts from the limbic system, the evolutionarily older and simpler part of the brain that affects emotion, behavior and motivation. But during adolescence, the limbic system connects and communicates with the rest of the brain differently than it does during adulthood, leaving many adolescents vulnerable to riskier behaviors, according to Duke University researchers. "We know adolescence is a time of profound social change. It's also ...