DOES QUALITY IMPROVEMENT IMPROVE PATIENTS’ HEALTH? A SYSTEMATIC REVIEW OF MEASURES OF EFFECT USED IN PDSA PROJECTS

Abstract

Introduction Quality improvement is an inherent part of modern healthcare systems worldwide, used for the continuous advancement in effectiveness and safety. Amongst other approaches, the plan-do-study-act (PDSA) method, a four-step iterative method, is widely used for testing and implementation of quality-improving interventions in health care despite a poor evidence of its effectiveness. Accepting the premise that the underlying purpose of improvements in health care is to improve patients’ health as suggested by, amongst others, the American National Academy of Medicine, the data measurements used to assess the impact of quality improvement projects, consequently, ought to reflect how they eventually impact patients’ health.

Objectives The purpose of the present paper is to review recently published PDSA projects to investigate how the effect of quality improvements are being measured, and if the authors provide scientific evidence that their choice of effect measure, i.e. the quality indicator, is associated with patients’ health.

Methods The basis of the present study was a systematic review of studies on PDSA quality improvement projects published in 2015-2017. For all identified papers, the primary quality indicators were categorized in accordance with A. Donabedian’s three definitions of dimensions of quality, i.e., structure, process, and outcome indicators. Secondly, it was assessed if the authors of the studies reported the applied quality indicator as being evidence-based, i.e. whether they were associated with patient-relevant outcomes. In addition, adherence to the SQUIRE guidelines as reported in the studies was observed. The assessment was performed individually by NKS and ASV. Any disagreements were resolved through discussion.

Results In all, 54 studies were included for assessment. One study was excluded, as reporting of any quality indicators was absent. The result of the systematic review showed that one out of 53 studies applied a structure indicator, in 83 percent (44/53) of the studies process indicators were applied, and outcome indicators were applied in 15 percent of the studies (8/53). In general, the quality indicators were very heterogeneous causing incomparability of results. The results from the assessment of whether authors identified the applied quality indicator as being an evidence-based indicator showed that only in four studies, the authors reported them as such.

Conclusion Process indicators, rather than health-related outcome measures, appear to be used most often in quality improvement projects applying the PDSA method. Evidence-based indicators were only applied in four studies. Overall, this challenges the ability to show if, and how, interventions actually affect patients’ health. This is problematic for a number of reasons; if the applied quality indicator does, in fact, not signify final impact patients’ health, the value of the interventions might be questioned. If we cannot confirm an association between applied quality indicators and final impact on health, we may potentially harm patients indirectly by pursuing ineffective solutions. This would be a direct antithesis to the main purpose of the health care system. This study is based on assessments of self-reported information in the studies and, consequently, the results presented here is a reflection of the information reported in the studies. However, systematic reporting guidelines were only applied in two studies. Enhancement of the validity of the present results calls for an increase in the consistent and systematic reporting of quality improvement interventions.