The effectiveness of e-learning: An explorative and integrative review of the definitions, methodologies and factors that promote e-Learning effectiveness

A structured search of library databases revealed that research examining the effectiveness of e-Learning has heavily increased within the last five years. After taking a closer look at the search results, the authors discovered that previous researchers defined and investigated effectiveness in multiple ways. At the same time, learning and development professionals within public and private organisations are increasingly being asked to prove the effectiveness of their learning and development initiatives. This paper investigates the effectiveness of e-Learning through an integrative review. The paper answers the following research questions: How is the effectiveness of e-Learning defined? How is the effectiveness of e-Learning measured? What makes e-Learning solutions effective? The authors discovered 19 distinct ways to define effectiveness, the most common of which is ‘learning outcome’, appearing in 41 % of the articles examined in the literature review. Moreover, the most common way to measure effectiveness is quantitatively with pre- and post-tests. This paper includes an empirical study of an e-Learning solution for science teachers (K–12) which serves as a valuable addition to the findings of the literature study. The study suggests that it is difficult to use e-Learning to improve teaching performance, as participating teachers can apply several strategies to avoid substantially changing their work-related practices. Furthermore, the study shows that only using the fulfilment of pre-defined learning objectives as an effectiveness parameter does not allow developers and researchers to see unexpected and unintended changes in practice that occur as a result of the e-Learning program. Finally, the research provides insight into the validity of self-assessments, suggesting that participants are able to successfully report their own practices, provided certain qualitative survey approaches are used. In this paper, a model for understanding the relationships of the key factors that influence effectiveness is developed. The model categorises these factors from three perspectives: the context in which the e-Learning solution is used, the artefact (the e-Learning solution it self) and the individuals that use the artefact. It was found that support and resources, the individuals’ motivation and prior experience and interaction between the artefact and the individuals that use it all influence effectiveness. Finally, this paper discusses whether e-Learning and traditional face-to-face learning should be measured according to the same definitions of and approaches to effectiveness, ending with a call for learning designers and researchers to target their measurement efforts to counting what counts for them and their stakeholders.