Speech-based emotion recognition and next reaction prediction

Abstract

Communication through voice is one of the main components of affective computing in human-computer interaction. In this type of interaction, properly comprehending the meanings of the words or the linguistic category and recognizing the emotion included in the speech is essential for enhancing the performance. In order to model the emotional state, the speech waves are utilized, which bear signals standing for emotions such as boredom, fear, joy and sadness. In the first step of the emotional reaction prediction system proposed in this paper, different emotions are recognized by means of different types of classifiers. The second step is the prediction of a sequence of the next emotional reactions using neural networks. The sequence is extracted based on the speech signals being digitized at tenths of a second, after concatenating the different speech signals of each subject. The prediction problem is solved as a nonlinear auto-regression time-series neural network with the assumption that the variables are defined as data-feedback time-series. the best average recognition rate is 86.25%, which is achieved by the Random Forest classifier, and the average prediction rate of reactions by using neural networks is 60.30%. © 2017 IEEE.