Introduction

The NTHU-NTUA Chinese Interactive Multimodal Emotion Corpus is designed with three major elements:

1) Adopt the use of dyadic interactions for natural elicitation of affective behaviors.

2) Collect both external behaviors and internal physiology of the dyad simultaneously.

3) Annotate rich emotion attributes of the interacting dyads from different perspectives.

 

This publicly-available large- scale Chinese affective database opens up potential oppor- tunities for researchers to advance research in directions, such as cross-cultural (language) emotion recognition, inter- action dynamics modeling, joint external-internal behavior- physiology modeling, and perceptual analyses on diverse emotion evaluations.

 

The collection protocol is designed based on situating the dyadic interaction in daily-life scenario settings, where each interaction lasts approximately 3-minute long with an overall pre-defined target affective atmosphere (i.e., angry, happy, sad, neutral, surprise, and frustration).

 

Overview

This database includes recordings of 44 Chinese speaking subjects engaged in spontaneous dyadic spoken interactions. The multimodal data includes approximately 11-hour worth of audio, video, and electrocardiogram data recorded continuously and synchronously. The database is also completed with a rich set of emotion annotations on discrete and continuous-in-time annotation by a total of 49 annotators. These emotion annotations include a diverse perspectives: peer-report, director-report, self-report, and observer-report.

 

To our best knowledge, the NNIME is one of the few large-scale Chinese affective dyadic interaction database that have been systematically collected, organized, and to be publicly-released to the research community.