Novel bidirectional multimodal system for affective human-robot engagement
Loading...
Author (Corporation)
Publication date
2021
Typ of student thesis
Course of study
Collections
Type
04B - Conference paper
Editors
Editor (Corporation)
Supervisor
Parent work
2021 IEEE Symposium Series on Computational Intelligence (SSCI) Proceedings
Special issue
DOI of the original publication
Link
Series
Series number
Volume
Issue / Number
Pages / Duration
Patent number
Publisher / Publishing institution
IEEE
Place of publication / Event location
New York
Edition
Version
Programming language
Assignee
Practice partner / Client
Abstract
Multimodal interaction is an essential prerequisite for affective human-robot engagement. Research on bidirectional, affective multimodal interaction systems investigates systems that recognize a user's affect and generate emotional response based on this user's affect. The presented work investigates a novel bidirectional, affective multimodal interaction system using a social robot and an open-source dialogue system framework, developing a prototype based here on Pepper and Rasa. Compared to special lab robotics systems, the proposed system is more attainable, while incorporating, alongside speech and facial expression, eye gaze as one of the major features to convey emotions as input channels. The system generates and emulates emotional output behaviors based on a user's affect using speech, gestures and emojis. This paper describes the concrete implementation and evaluation of the proposed system. Results of the evaluation show that, although the recognition accuracy of the input channels perform differently well, the system can derive well-defined rule-based emotional output behaviors with a high multimodal accuracy rate in the given test scenarios.
Keywords
Subject (DDC)
Event
IEEE Symposium Series on Computational Intelligence (IEEE SSCI 2021)
Exhibition start date
Exhibition end date
Conference start date
05.12.2021
Conference end date
07.12.2021
Date of the last check
ISBN
978-1-7281-9048-8
ISSN
Language
English
Created during FHNW affiliation
Yes
Strategic action fields FHNW
Publication status
Published
Review
Peer review of the complete publication
Open access category
Closed
License
Citation
Applewhite, T., Zhong, J., & Dornberger, R. (2021). Novel bidirectional multimodal system for affective human-robot engagement. 2021 IEEE Symposium Series on Computational Intelligence (SSCI) Proceedings. IEEE Symposium Series on Computational Intelligence (IEEE SSCI 2021). https://doi.org/10.1109/SSCI50451.2021.9659935