Facial Emotion Recognition Using Residual Neural Networks

Abstract

Facial emotion recognition (FER) has been an emerging research topic in recent years. Recent automatic FER systems generally apply deep learning methods and focus on two important issues: lack of sufficient labeled training data and variations in images such as illumination, pose, or expression-related variations among different cultures. Although Convolutional Neural Networks (CNNs) are widely used in automatic FER, they cannot be used when the number of layers is large. Therefore, a residual technique is applied to CNNs and this architecture is named residual neural network. In this paper, an automatic facial emotion recognition method using residual networks with random data augmentation is proposed on a merged FER dataset consisting of 41,598 facial images of size 48 × 48 pixels from seven basic emotion classes. Experimental results show that ResNet34 with data augmentation performs better than CNN with a classification accuracy of 81%.