Institut des Systèmes Intelligents
et de Robotique


Sorbonne Universite



Tremplin CARNOT Interfaces



Thesis Title : What the face reveals : Action Units analysis for emotional interpretation


Keywords : Facial expression, Action Unit, emotional dynamic, LGBP, SVM, multi-kernel learning, multilinear analysis.


Abstract  :

A current challenge in designing computerized environments is to place the human user at the core of the system.  There is a need for the computer to be able to interact naturally with the user, similar to the way human-human interaction takes place.

This thesis addresses the problem of facial expressions recognition from images or sequences. For that, three applications are proposed: the localization and tracking of facial feature points to detect the temporal segment of the emotion, the analysis of facial micro-movements, named Action Units (AU), and the detection of the emotion.

From a methodological point of view, various techniques have been proposed. To represent the visage in a way that separate identity and emotion, we compute histograms differences of Local Gabor Binary Patterns  (LGBP) and a new kernel function, the function HDI. We also propose an adaptation of the multi-linear analysis in the case of missing observations. For the classifier, we use recent progress in multi-kernel learning for Support Vector Machine (SVM) to combine different types of features, as static features with dynamic features or geometric features with appearance features. Rigorous comparisons with the state of the art and the first place at the first international campaign for AU recognition, FERA'11, show the progress achieved in facial expression recognition.

In addition, various issues specific to the facial expression recognition have been discussed, as the use of information about the subject's identity, the temporal analysis of the emotion and the prior detection of AU to recognize emotions.