Lightweight Multi-Task Emotion Recognition for Human-Robot Interaction

Alexey Dmitrievich Yakushev Национальный исследовательский университет ИТМО
Alexey Michailovich Kashevnik St. Petersburg Federal Research Center of the Russian Academy of Sciences (SPC RAS),

Keywords: Human-Robot Interaction, Emotion Recognition, Computer Vision, Multi-Task Learning, MobileNetV3, Asymmetric Loss, EMOTIC

Abstract. We address the problem of real-time emotion perception on resource-

limited robotic platforms. Instead of pursuing marginal accuracy gains with

computationally heavy architectures, we investigate the performance limits of a

streamlined multi-task setup. The proposed method performs 26-way emotion

classification alongside continuous Valence, Arousal, and Dominance (VAD)

regression using a shared MobileNetV3-Large backbone. To prevent the

classification head from being biased by the heavily skewed label distribution of

the EMOTIC dataset, we apply Asymmetric Loss (ASL). Furthermore,

decoupling the feature projections for discrete and continuous outputs mitigates

gradient interference between the two tasks. On the EMOTIC test split, the model

achieved 25.33% mean average precision for classification and a mean error rate

of 0.9146 for VAD prediction. Given the fine-grained, context-dependent nature

of the labels, this provides a robust baseline for edge-oriented models. On the

deployment side, after ONNX conversion and OpenVINO optimization, CPU

inference requires only 3.29 ms per frame (over 300 FPS). This leaves substantial

computational headroom for concurrent robot control loops. The results

demonstrate that massive models are not strictly necessary to extract reliable

affective signals on edge devices.