A major challenge in automatic human emotion recognition is that of categorizing the very broad and complex spectrum of human emotions. In this regard, a critical bottleneck is represented by the difficulty in obtaining annotated data to build such models. Indeed, all the publicly available datasets collected to this aim are either annotated with (i) the six prototypical emotions, or (ii) continuous valence/arousal (VA) values. On the one hand, the six basic emotions represent a coarse approximation of the vast spectrum of human emotions, and are of limited utility to understand a person's emotional state. Oppositely, performing dimensional emotion recognition using VA can cover the full range of human emotions, yet it lacks a clear interpretation. Moreover, data annotation with VA is challenging as it requires expert annotators, and there is no guarantee that annotations are consistent with the six prototypical emotions. In this paper, we present an investigation aiming to bridge the gap between the two modalities. We propose to leverage VA values to obtain a fine-grained taxonomy of emotions, interpreting emotional states as probability distributions over the VA space. This has the potential for enabling automatic annotation of existing datasets with this new taxonomy, avoiding the need for expensive data collection and labeling. However, our preliminary results disclose two major problems: first, continuous VA values and the six standard emotion labels are often inconsistent, raising concerns about the validity of existing datasets; second, datasets claimed to be balanced in terms of emotion labels become instead severely unbalanced if provided with a fine-grained emotion annotation. We conclude that efforts are needed in terms of data collection to further push forward the research in this field.

Towards a Better Understanding of Human Emotions: Challenges of Dataset Labeling / Guerdelli, H.; Ferrari, C.; Cardia Neto, J. B.; Berretti, S.; Barhoumi, W.; Del Bimbo, A.. - 14365 LNCS:(2024), pp. 242-254. [10.1007/978-3-031-51023-6_21]

Towards a Better Understanding of Human Emotions: Challenges of Dataset Labeling

Ferrari C.;
2024-01-01

Abstract

A major challenge in automatic human emotion recognition is that of categorizing the very broad and complex spectrum of human emotions. In this regard, a critical bottleneck is represented by the difficulty in obtaining annotated data to build such models. Indeed, all the publicly available datasets collected to this aim are either annotated with (i) the six prototypical emotions, or (ii) continuous valence/arousal (VA) values. On the one hand, the six basic emotions represent a coarse approximation of the vast spectrum of human emotions, and are of limited utility to understand a person's emotional state. Oppositely, performing dimensional emotion recognition using VA can cover the full range of human emotions, yet it lacks a clear interpretation. Moreover, data annotation with VA is challenging as it requires expert annotators, and there is no guarantee that annotations are consistent with the six prototypical emotions. In this paper, we present an investigation aiming to bridge the gap between the two modalities. We propose to leverage VA values to obtain a fine-grained taxonomy of emotions, interpreting emotional states as probability distributions over the VA space. This has the potential for enabling automatic annotation of existing datasets with this new taxonomy, avoiding the need for expensive data collection and labeling. However, our preliminary results disclose two major problems: first, continuous VA values and the six standard emotion labels are often inconsistent, raising concerns about the validity of existing datasets; second, datasets claimed to be balanced in terms of emotion labels become instead severely unbalanced if provided with a fine-grained emotion annotation. We conclude that efforts are needed in terms of data collection to further push forward the research in this field.
2024
9783031510229
9783031510236
Towards a Better Understanding of Human Emotions: Challenges of Dataset Labeling / Guerdelli, H.; Ferrari, C.; Cardia Neto, J. B.; Berretti, S.; Barhoumi, W.; Del Bimbo, A.. - 14365 LNCS:(2024), pp. 242-254. [10.1007/978-3-031-51023-6_21]
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11381/2988113
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? 0
social impact