Non-intrusive Measurement of Player Engagement and Emotions: Real-Time Deep Neural Network Analysis of Facial Expressions during Game Play

Dines Rae Selvig, Henrik Schoenau-Fog

Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

Abstract

Prior research suggests and reveals that there is a correlation between human emotional responses and the subjective qualities of digital interactive experiences. Using facial analysis done by deep neural networks presents a true non-intrusive way of measuring emotional responses and engagement assessed as the desire to continue playing. This paper proposes a tool to measure emotional responses across eight different emotions and in real time of any game. The emotional recognition system achieves an accuracy of 98% and the continuation desire system achieves 93.3% accuracy in a pilot test with a two player game and 78.5% accuracy in a single player game. This forms a strong tool that shows a correlation between emotions and the continuation desire of a player, which can be used to evaluate engagement in games and digital interactive experiences, e.g. in critical stages of development of said content.
Original languageEnglish
Title of host publicationHCI in Games : Second International Conference, HCI-Games 2020, Held as Part of the 22nd HCI International Conference, HCII 2020, Copenhagen, Denmark, July 19–24, 2020, Proceedings
EditorsXiaowen Fang
Number of pages20
Volume12211
Place of PublicationSwitzerland
PublisherSpringer
Publication date19 Jul 2020
Pages330-349
ISBN (Print)978-3-030-50163-1
ISBN (Electronic)978-3-030-50164-8
DOIs
Publication statusPublished - 19 Jul 2020
Event22nd International Conference on Human Computer Interaction - Copenhagen, Denmark
Duration: 19 Jul 202024 Jul 2020

Conference

Conference22nd International Conference on Human Computer Interaction
CountryDenmark
CityCopenhagen
Period19/07/202024/07/2020
SeriesLecture Notes in Computer Science
Volume12211
ISSN0302-9743

Keywords

  • Continuation desire
  • Conation
  • Emotion
  • Machine learning
  • Facial expressions
  • Design tools/technologies
  • Game and flow
  • Game immersion
  • Player engagement assessment
  • Facial capture

Fingerprint

Dive into the research topics of 'Non-intrusive Measurement of Player Engagement and Emotions: Real-Time Deep Neural Network Analysis of Facial Expressions during Game Play'. Together they form a unique fingerprint.

Cite this