Assessing individual contributions to Collaborative Problem Solving: A network analysis approach

Zachari Swiecki*, A. R. Ruis, Cayley Farrell, David Williamson Shaffer

*Corresponding author

Research output: Contribution to journalJournal articleResearchpeer-review

Abstract

Collaborative Problem Solving (CPS) is an interactive, interdependent, and temporal process. However, current methods for measuring the CPS processes of individuals, such as coding and counting, treat these processes as sets of isolated and independent events. In contrast, Epistemic Network Analysis (ENA) models how the contributions of a given individual relate to the contributions of others. This article examines the communications of air defense warfare teams from an experiment comparing two different computer-based decision support systems, using this data to ask whether ENA provides a more ecologically valid quantitative model of CPS than coding and counting. Qualitative analysis showed that commanders using one system asked questions to understand the tactical situation, while commanders using an experimental system focused more on actions in response to the tactical situation. Neither of the coding and counting approaches we tested corroborated these findings with statistically significant results. In contrast, ENA created models of the individual contributions of commanders that (a) showed statistical differences between commanders using the two systems to corroborate the qualitative analysis, and (b) revealed differences in individual performance. This suggests that ENA is a more powerful tool for CPS assessment than coding and counting approaches.
Original languageEnglish
Article number105876
JournalComputers in Human Behavior
Volume104
ISSN0747-5632
DOIs
Publication statusPublished - Mar 2020

    Fingerprint

Keywords

  • Assessment
  • Coding and counting
  • Collaborative Problem Solving
  • Epistemic Network Analysis
  • Measurement

Cite this