Abstract: Facial expression recognition and learning sentiment analysis based on classroom video image understanding have become research hotspots in smart education. However, these applications often face great challenges in real-world scenarios with low-quality image and video acquisition, and serious multi-target occlusion in complex environments. In this paper, a multitask recognition model for classifying student expressions is proposed. Firstly, this study constructs a multitask classroom expression dataset and effectively alleviates the imbalance of class label distribution in the dataset. Secondly, a classroom expression classification model based on multitask learning is proposed. By introducing knowledge distillation and designing a dual-channel fusion mechanism, the model effectively integrates the three tasks of discrete expression recognition, facial action unit detection and valence-arousal estimation. This integration leverages the relationship between multitasks to further enhance the performance of discrete expression classification. Finally, the proposed method is compared with the existing advanced methods across multiple datasets. Results show that the proposed model effectively improves the accuracy of expression classification, and demonstrates superior performance in the multitask recognition of classroom expressions, which provides technical support for multi-dimensional evaluation and analysis of classroom emotions.

Key words: deep learning, expression recognition, affective computing, classroom expression