Gently does it: Humans outperform a software classifier in recognizing subtle, nonstereotypical facial expressions

Neta Yitzhak, Nir Giladi, Tanya Gurevich, Daniel S. Messinger, Emily B. Prince, Katherine Martin, Hillel Aviezer

Research output: Contribution to journalArticlepeer-review

Abstract

According to dominant theories of affect, humans innately and universally express a set of emotions using specific configurations of prototypical facial activity. Accordingly, thousands of studies have tested emotion recognition using sets of highly intense and stereotypical facial expressions, yet their incidence in real life is virtually unknown. In fact, a commonplace experience is that emotions are expressed in subtle and nonprototypical forms. Such facial expressions are at the focus of the current study. In Experiment 1, we present the development and validation of a novel stimulus set consisting of dynamic and subtle emotional facial displays conveyed without constraining expressers to using prototypical configurations. Although these subtle expressions were more challenging to recognize than prototypical dynamic expressions, they were still well recognized by human raters, and perhaps most importantly, they were rated as more ecological and naturalistic than the prototypical expressions. In Experiment 2, we examined the characteristics of subtle versus prototypical expressions by subjecting them to a software classifier, which used prototypical basic emotion criteria. Although the software was highly successful at classifying prototypical expressions, it performed very poorly at classifying the subtle expressions. Further validation was obtained from human expert face coders: Subtle stimuli did not contain many of the key facial movements present in prototypical expressions. Together, these findings suggest that emotions may be successfully conveyed to human viewers using subtle nonprototypical expressions. Although classic prototypical facial expressions are well recognized, they appear less naturalistic and may not capture the richness of everyday emotional communication.

Original languageEnglish
Pages (from-to)1187-1198
Number of pages12
JournalEmotion
Volume17
Issue number8
DOIs
StatePublished - Dec 2017

Keywords

  • Facial expressions
  • Nonprototypical
  • Software classifier
  • Subtle

All Science Journal Classification (ASJC) codes

  • General Psychology

Fingerprint

Dive into the research topics of 'Gently does it: Humans outperform a software classifier in recognizing subtle, nonstereotypical facial expressions'. Together they form a unique fingerprint.

Cite this