Evaluating Learning Analytics of Adaptive Learning Systems: A Work in Progress Systematic Review

Tobias Alexander Bang Tretow-Fish; Md Saifuddin Khalid

doi:10.1007/978-3-031-06675-7_3

Evaluating Learning Analytics of Adaptive Learning Systems: A Work in Progress Systematic Review

Tobias Alexander Bang Tretow-Fish^*, Md Saifuddin Khalid

^*Corresponding author for this work

Research output: Contribution to book/anthology/report/conference proceeding › Article in proceeding › Research › peer-review

5 Citations (Scopus)

Abstract

There is currently no systematic overview of methods for evaluating Learning Analytics (LA) and Learning Analytics Dashboards (LAD) of Adaptive Learning Platforms (ALPs). 10 articles and 2 reviews are analyzed and synthesized. Focusing on the purposes of evaluation, methods used in the studies are grouped into five categories (C1-5): C1) evaluation of LA and LAD design and framework, C2) evaluation of performance with LA and LAD, C3) evaluation of adaptivity functions of the system, C4) evaluation of perceived value, and C5) Evaluation of pedagogical and didactic theory/context. While there is a relative high representation of evaluations in the C1-C4 categories of methods, which contribute to the design and development of the interaction and interface design features, the C5 category is not represented. The presence of pedagogical and didactical theory in the LA, LAD, and ALPs is lacking. Though traces of pedagogical theory is present none of the studies evaluates on its impact.

Original language	English
Title of host publication	Design, Learning, and Innovation - 6th EAI International Conference, DLI 2021, Proceedings
Editors	Eva Brooks, Jeanette Sjöberg, Anders Kalsgaard Møller
Number of pages	16
Publisher	Springer
Publication date	2022
Pages	37-52
ISBN (Print)	9783031066740
DOIs	https://doi.org/10.1007/978-3-031-06675-7_3
Publication status	Published - 2022
Externally published	Yes
Event	6th EAI International Conference on Design, Learning, and Innovation, DLI 2021 - Virtual, Online Duration: 10 Dec 2021 → 11 Dec 2021

Conference

Conference	6th EAI International Conference on Design, Learning, and Innovation, DLI 2021
City	Virtual, Online
Period	10/12/2021 → 11/12/2021

Series	Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST
Volume	435 LNICST
ISSN	1867-8211

Bibliographical note

Publisher Copyright:
© 2022, ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering.

Keywords

Adaptive learning platforms
Evaluation
Learning analytics

Access to Document

10.1007/978-3-031-06675-7_3

AUB Link

Search for the material in Aalborg University Library's search engine

Cite this

Tretow-Fish, Tobias Alexander Bang ; Khalid, Md Saifuddin. / Evaluating Learning Analytics of Adaptive Learning Systems : A Work in Progress Systematic Review. Design, Learning, and Innovation - 6th EAI International Conference, DLI 2021, Proceedings. editor / Eva Brooks ; Jeanette Sjöberg ; Anders Kalsgaard Møller. Springer, 2022. pp. 37-52 (Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST, Vol. 435 LNICST).

@inproceedings{1f5c9decd4194a62a8b53f10734e329d,

title = "Evaluating Learning Analytics of Adaptive Learning Systems: A Work in Progress Systematic Review",

abstract = "There is currently no systematic overview of methods for evaluating Learning Analytics (LA) and Learning Analytics Dashboards (LAD) of Adaptive Learning Platforms (ALPs). 10 articles and 2 reviews are analyzed and synthesized. Focusing on the purposes of evaluation, methods used in the studies are grouped into five categories (C1-5): C1) evaluation of LA and LAD design and framework, C2) evaluation of performance with LA and LAD, C3) evaluation of adaptivity functions of the system, C4) evaluation of perceived value, and C5) Evaluation of pedagogical and didactic theory/context. While there is a relative high representation of evaluations in the C1-C4 categories of methods, which contribute to the design and development of the interaction and interface design features, the C5 category is not represented. The presence of pedagogical and didactical theory in the LA, LAD, and ALPs is lacking. Though traces of pedagogical theory is present none of the studies evaluates on its impact.",

keywords = "Adaptive learning platforms, Evaluation, Learning analytics",

author = "Tretow-Fish, \{Tobias Alexander Bang\} and Khalid, \{Md Saifuddin\}",

note = "Publisher Copyright: {\textcopyright} 2022, ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering.; 6th EAI International Conference on Design, Learning, and Innovation, DLI 2021 ; Conference date: 10-12-2021 Through 11-12-2021",

year = "2022",

doi = "10.1007/978-3-031-06675-7\_3",

language = "English",

isbn = "9783031066740",

series = "Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST",

publisher = "Springer",

pages = "37--52",

editor = "Eva Brooks and Jeanette Sj{\"o}berg and M{\o}ller, \{Anders Kalsgaard\}",

booktitle = "Design, Learning, and Innovation - 6th EAI International Conference, DLI 2021, Proceedings",

address = "Germany",

}

Tretow-Fish, TAB & Khalid, MS 2022, Evaluating Learning Analytics of Adaptive Learning Systems: A Work in Progress Systematic Review. in E Brooks, J Sjöberg & AK Møller (eds), Design, Learning, and Innovation - 6th EAI International Conference, DLI 2021, Proceedings. Springer, Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST, vol. 435 LNICST, pp. 37-52, 6th EAI International Conference on Design, Learning, and Innovation, DLI 2021, Virtual, Online, 10/12/2021. https://doi.org/10.1007/978-3-031-06675-7_3

Evaluating Learning Analytics of Adaptive Learning Systems: A Work in Progress Systematic Review. / Tretow-Fish, Tobias Alexander Bang; Khalid, Md Saifuddin.
Design, Learning, and Innovation - 6th EAI International Conference, DLI 2021, Proceedings. ed. / Eva Brooks; Jeanette Sjöberg; Anders Kalsgaard Møller. Springer, 2022. p. 37-52 (Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST, Vol. 435 LNICST).

Research output: Contribution to book/anthology/report/conference proceeding › Article in proceeding › Research › peer-review

TY - GEN

T1 - Evaluating Learning Analytics of Adaptive Learning Systems

T2 - 6th EAI International Conference on Design, Learning, and Innovation, DLI 2021

AU - Tretow-Fish, Tobias Alexander Bang

AU - Khalid, Md Saifuddin

PY - 2022

Y1 - 2022

N2 - There is currently no systematic overview of methods for evaluating Learning Analytics (LA) and Learning Analytics Dashboards (LAD) of Adaptive Learning Platforms (ALPs). 10 articles and 2 reviews are analyzed and synthesized. Focusing on the purposes of evaluation, methods used in the studies are grouped into five categories (C1-5): C1) evaluation of LA and LAD design and framework, C2) evaluation of performance with LA and LAD, C3) evaluation of adaptivity functions of the system, C4) evaluation of perceived value, and C5) Evaluation of pedagogical and didactic theory/context. While there is a relative high representation of evaluations in the C1-C4 categories of methods, which contribute to the design and development of the interaction and interface design features, the C5 category is not represented. The presence of pedagogical and didactical theory in the LA, LAD, and ALPs is lacking. Though traces of pedagogical theory is present none of the studies evaluates on its impact.

AB - There is currently no systematic overview of methods for evaluating Learning Analytics (LA) and Learning Analytics Dashboards (LAD) of Adaptive Learning Platforms (ALPs). 10 articles and 2 reviews are analyzed and synthesized. Focusing on the purposes of evaluation, methods used in the studies are grouped into five categories (C1-5): C1) evaluation of LA and LAD design and framework, C2) evaluation of performance with LA and LAD, C3) evaluation of adaptivity functions of the system, C4) evaluation of perceived value, and C5) Evaluation of pedagogical and didactic theory/context. While there is a relative high representation of evaluations in the C1-C4 categories of methods, which contribute to the design and development of the interaction and interface design features, the C5 category is not represented. The presence of pedagogical and didactical theory in the LA, LAD, and ALPs is lacking. Though traces of pedagogical theory is present none of the studies evaluates on its impact.

KW - Adaptive learning platforms

KW - Evaluation

KW - Learning analytics

UR - https://www.scopus.com/pages/publications/85131941380

U2 - 10.1007/978-3-031-06675-7_3

DO - 10.1007/978-3-031-06675-7_3

M3 - Article in proceeding

AN - SCOPUS:85131941380

SN - 9783031066740

T3 - Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST

SP - 37

EP - 52

BT - Design, Learning, and Innovation - 6th EAI International Conference, DLI 2021, Proceedings

A2 - Brooks, Eva

A2 - Sjöberg, Jeanette

A2 - Møller, Anders Kalsgaard

PB - Springer

Y2 - 10 December 2021 through 11 December 2021

ER -

Tretow-Fish TAB, Khalid MS. Evaluating Learning Analytics of Adaptive Learning Systems: A Work in Progress Systematic Review. In Brooks E, Sjöberg J, Møller AK, editors, Design, Learning, and Innovation - 6th EAI International Conference, DLI 2021, Proceedings. Springer. 2022. p. 37-52. (Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST, Vol. 435 LNICST). doi: 10.1007/978-3-031-06675-7_3