Classifying Facial Expressions of Emotion

There is a large body of research regarding classifying facial expressions of emotion. I will be presenting one popular theory relevant to our collaboration here.

The Facial Action Coding System (also known as FACS) is a system used to classify human facial movements based on a system developed by Swedish anatomist Carl-Herman Hjortsjo and later adopted by Paul Ekman and Wallace Friesen, pioneers in the research of facial expressions of emotion. It works by coding the movements of individual face muscles and as you may have guessed, has been used to systematically categorize physical, facial expressions of emotion based on these specific muscle movements. It  first deconstructs the facial expressions into muscle movements, and then further encodes them as Action Units (AUs), representing the contraction or relaxation of one or more muscles.

Therefore, FACS can be used by man and machine alike to correctly label expressions. Given that FACS categorizes facial muscle movement, its use can extent beyond humans to several non-human primates, including chimpanzees, rhesus macaques, gibbons, and orangutans, with a recent adaption for dogs as well.

Commonly considered the 6 basic emotions, Happiness, Sadness, Surprise, Fear, Anger, and Disgust are represented by the following AUs:

Happiness	6+12
Sadness	1+4+15
Surprise	1+2+5B+26
Fear	1+2+4+5+7+20+26
Anger	4+5+7+23
Disgust	9+15+16

However, using AU and the FACS system, researchers from Ohio State University have identified what they consider to be 21 distinct emotion categories derived from compound facial expressions of emotion, each with their own unique AU combination (Du, Tao, & Martinez, 2014).

Given that emotional expressions can be broken down into action units that can be observed and measured,  this opens the door for the development of computer programs that can detect expressions in real time using live video or image capture. I will describe some of these programs in my next post. 

Some content from this post has been taken from the FACS Wikipedia page. Images from this post acquired using Google Images.