Human micro-gesture recognition by adversarial training

Abstract

Human communication is based on primarily on verbal language, that is complex enough to deliver deep concepts, but alongside that, non-verbal language delivers other kinds of details, like intentional and non-intentional messages about ourselves. Thus, it is interesting to combine the growing ability to process big amount of data of nowadays machine learning algorithms with machine vision techniques aiming to investigate more this less explored communication channel. The focus of this Master’s Thesis work are Micro-Gestures: fast and short in time movements of the body, mainly of the face and the hands, that are often unconscious and express people true feelings, regardless of the message people are consciously trying to communicate. For this particular type of gestures, iMiGUE is a new dataset that classifies 32 types of these Micro-Gestures. The dataset is composed of videos taken after tennis matches, and collects about 18,000 clips of Micro-Gestures. In this work, a non-traditional machine learning technique is explored, to check if it can be applied to this task and improve the classification of this Micro-Gestures, that has very subtle differences between them. Such technique is adversarial learning, that involves training neural networks also with samples that are imperceptibly modified specifically to fool the classifier. Adversarial learning is proved to be able to enhance robustness of networks by the training with these adversarial samples. The work investigates the effects of adversarial learning on two different implementations, to check whether it is a valuable tool to capture the characteristics of the Micro-Gestures.