DOES QUALITY IMPROVEMENT IMPROVE PATIENTS’ HEALTH? A SYSTEMATIC REVIEW OF MEASURES OF EFFECT USED IN PDSA PROJECTS

Research output: Contribution to conference without publisher/journalConference abstract for conferenceResearchpeer-review

Abstract

Abstract

Introduction Quality improvement is an inherent part of modern healthcare systems worldwide, used for the continuous advancement in effectiveness and safety. Amongst other approaches, the plan-do-study-act (PDSA) method, a four-step iterative method, is widely used for testing and implementation of quality-improving interventions in health care despite a poor evidence of its effectiveness. Accepting the premise that the underlying purpose of improvements in health care is to improve patients’ health as suggested by, amongst others, the American National Academy of Medicine, the data measurements used to assess the impact of quality improvement projects, consequently, ought to reflect how they eventually impact patients’ health.

Objectives The purpose of the present paper is to review recently published PDSA projects to investigate how the effect of quality improvements are being measured, and if the authors provide scientific evidence that their choice of effect measure, i.e. the quality indicator, is associated with patients’ health.

Methods The basis of the present study was a systematic review of studies on PDSA quality improvement projects published in 2015-2017. For all identified papers, the primary quality indicators were categorized in accordance with A. Donabedian’s three definitions of dimensions of quality, i.e., structure, process, and outcome indicators. Secondly, it was assessed if the authors of the studies reported the applied quality indicator as being evidence-based, i.e. whether they were associated with patient-relevant outcomes. In addition, adherence to the SQUIRE guidelines as reported in the studies was observed. The assessment was performed individually by NKS and ASV. Any disagreements were resolved through discussion.

Results In all, 54 studies were included for assessment. One study was excluded, as reporting of any quality indicators was absent. The result of the systematic review showed that one out of 53 studies applied a structure indicator, in 83 percent (44/53) of the studies process indicators were applied, and outcome indicators were applied in 15 percent of the studies (8/53). In general, the quality indicators were very heterogeneous causing incomparability of results. The results from the assessment of whether authors identified the applied quality indicator as being an evidence-based indicator showed that only in four studies, the authors reported them as such.

Conclusion Process indicators, rather than health-related outcome measures, appear to be used most often in quality improvement projects applying the PDSA method. Evidence-based indicators were only applied in four studies. Overall, this challenges the ability to show if, and how, interventions actually affect patients’ health. This is problematic for a number of reasons; if the applied quality indicator does, in fact, not signify final impact patients’ health, the value of the interventions might be questioned. If we cannot confirm an association between applied quality indicators and final impact on health, we may potentially harm patients indirectly by pursuing ineffective solutions. This would be a direct antithesis to the main purpose of the health care system. This study is based on assessments of self-reported information in the studies and, consequently, the results presented here is a reflection of the information reported in the studies. However, systematic reporting guidelines were only applied in two studies. Enhancement of the validity of the present results calls for an increase in the consistent and systematic reporting of quality improvement interventions.
Original languageEnglish
Publication date2018
Number of pages2
Publication statusPublished - 2018
Event5th Nordic Conference on Research in Patient Safety and Quality in Healthcare - DGI byen, København, Denmark
Duration: 30 Aug 201831 Aug 2018
Conference number: 5
http://www.nsqh.org/2018

Conference

Conference5th Nordic Conference on Research in Patient Safety and Quality in Healthcare
Number5
LocationDGI byen
CountryDenmark
CityKøbenhavn
Period30/08/201831/08/2018
Internet address

Keywords

    Cite this

    Hermansen, N. K., Vestergaard, A. S., & Ehlers, L. H. (2018). DOES QUALITY IMPROVEMENT IMPROVE PATIENTS’ HEALTH? A SYSTEMATIC REVIEW OF MEASURES OF EFFECT USED IN PDSA PROJECTS. Abstract from 5th Nordic Conference on Research in Patient Safety and Quality in Healthcare, København, Denmark.
    @conference{285a002139034bc08531c7a57bcdd42e,
    title = "DOES QUALITY IMPROVEMENT IMPROVE PATIENTS’ HEALTH? A SYSTEMATIC REVIEW OF MEASURES OF EFFECT USED IN PDSA PROJECTS",
    abstract = "AbstractIntroduction Quality improvement is an inherent part of modern healthcare systems worldwide, used for the continuous advancement in effectiveness and safety. Amongst other approaches, the plan-do-study-act (PDSA) method, a four-step iterative method, is widely used for testing and implementation of quality-improving interventions in health care despite a poor evidence of its effectiveness. Accepting the premise that the underlying purpose of improvements in health care is to improve patients’ health as suggested by, amongst others, the American National Academy of Medicine, the data measurements used to assess the impact of quality improvement projects, consequently, ought to reflect how they eventually impact patients’ health.Objectives The purpose of the present paper is to review recently published PDSA projects to investigate how the effect of quality improvements are being measured, and if the authors provide scientific evidence that their choice of effect measure, i.e. the quality indicator, is associated with patients’ health. Methods The basis of the present study was a systematic review of studies on PDSA quality improvement projects published in 2015-2017. For all identified papers, the primary quality indicators were categorized in accordance with A. Donabedian’s three definitions of dimensions of quality, i.e., structure, process, and outcome indicators. Secondly, it was assessed if the authors of the studies reported the applied quality indicator as being evidence-based, i.e. whether they were associated with patient-relevant outcomes. In addition, adherence to the SQUIRE guidelines as reported in the studies was observed. The assessment was performed individually by NKS and ASV. Any disagreements were resolved through discussion.Results In all, 54 studies were included for assessment. One study was excluded, as reporting of any quality indicators was absent. The result of the systematic review showed that one out of 53 studies applied a structure indicator, in 83 percent (44/53) of the studies process indicators were applied, and outcome indicators were applied in 15 percent of the studies (8/53). In general, the quality indicators were very heterogeneous causing incomparability of results. The results from the assessment of whether authors identified the applied quality indicator as being an evidence-based indicator showed that only in four studies, the authors reported them as such. Conclusion Process indicators, rather than health-related outcome measures, appear to be used most often in quality improvement projects applying the PDSA method. Evidence-based indicators were only applied in four studies. Overall, this challenges the ability to show if, and how, interventions actually affect patients’ health. This is problematic for a number of reasons; if the applied quality indicator does, in fact, not signify final impact patients’ health, the value of the interventions might be questioned. If we cannot confirm an association between applied quality indicators and final impact on health, we may potentially harm patients indirectly by pursuing ineffective solutions. This would be a direct antithesis to the main purpose of the health care system. This study is based on assessments of self-reported information in the studies and, consequently, the results presented here is a reflection of the information reported in the studies. However, systematic reporting guidelines were only applied in two studies. Enhancement of the validity of the present results calls for an increase in the consistent and systematic reporting of quality improvement interventions.",
    keywords = "PDSA method, effectiveness, quality indicators, evidence based, patient outcomes",
    author = "Hermansen, {Nanna Kastrup} and Vestergaard, {Anne Sig} and Ehlers, {Lars Holger}",
    year = "2018",
    language = "English",
    note = "null ; Conference date: 30-08-2018 Through 31-08-2018",
    url = "http://www.nsqh.org/2018",

    }

    DOES QUALITY IMPROVEMENT IMPROVE PATIENTS’ HEALTH? A SYSTEMATIC REVIEW OF MEASURES OF EFFECT USED IN PDSA PROJECTS. / Hermansen, Nanna Kastrup; Vestergaard, Anne Sig; Ehlers, Lars Holger.

    2018. Abstract from 5th Nordic Conference on Research in Patient Safety and Quality in Healthcare, København, Denmark.

    Research output: Contribution to conference without publisher/journalConference abstract for conferenceResearchpeer-review

    TY - ABST

    T1 - DOES QUALITY IMPROVEMENT IMPROVE PATIENTS’ HEALTH? A SYSTEMATIC REVIEW OF MEASURES OF EFFECT USED IN PDSA PROJECTS

    AU - Hermansen, Nanna Kastrup

    AU - Vestergaard, Anne Sig

    AU - Ehlers, Lars Holger

    PY - 2018

    Y1 - 2018

    N2 - AbstractIntroduction Quality improvement is an inherent part of modern healthcare systems worldwide, used for the continuous advancement in effectiveness and safety. Amongst other approaches, the plan-do-study-act (PDSA) method, a four-step iterative method, is widely used for testing and implementation of quality-improving interventions in health care despite a poor evidence of its effectiveness. Accepting the premise that the underlying purpose of improvements in health care is to improve patients’ health as suggested by, amongst others, the American National Academy of Medicine, the data measurements used to assess the impact of quality improvement projects, consequently, ought to reflect how they eventually impact patients’ health.Objectives The purpose of the present paper is to review recently published PDSA projects to investigate how the effect of quality improvements are being measured, and if the authors provide scientific evidence that their choice of effect measure, i.e. the quality indicator, is associated with patients’ health. Methods The basis of the present study was a systematic review of studies on PDSA quality improvement projects published in 2015-2017. For all identified papers, the primary quality indicators were categorized in accordance with A. Donabedian’s three definitions of dimensions of quality, i.e., structure, process, and outcome indicators. Secondly, it was assessed if the authors of the studies reported the applied quality indicator as being evidence-based, i.e. whether they were associated with patient-relevant outcomes. In addition, adherence to the SQUIRE guidelines as reported in the studies was observed. The assessment was performed individually by NKS and ASV. Any disagreements were resolved through discussion.Results In all, 54 studies were included for assessment. One study was excluded, as reporting of any quality indicators was absent. The result of the systematic review showed that one out of 53 studies applied a structure indicator, in 83 percent (44/53) of the studies process indicators were applied, and outcome indicators were applied in 15 percent of the studies (8/53). In general, the quality indicators were very heterogeneous causing incomparability of results. The results from the assessment of whether authors identified the applied quality indicator as being an evidence-based indicator showed that only in four studies, the authors reported them as such. Conclusion Process indicators, rather than health-related outcome measures, appear to be used most often in quality improvement projects applying the PDSA method. Evidence-based indicators were only applied in four studies. Overall, this challenges the ability to show if, and how, interventions actually affect patients’ health. This is problematic for a number of reasons; if the applied quality indicator does, in fact, not signify final impact patients’ health, the value of the interventions might be questioned. If we cannot confirm an association between applied quality indicators and final impact on health, we may potentially harm patients indirectly by pursuing ineffective solutions. This would be a direct antithesis to the main purpose of the health care system. This study is based on assessments of self-reported information in the studies and, consequently, the results presented here is a reflection of the information reported in the studies. However, systematic reporting guidelines were only applied in two studies. Enhancement of the validity of the present results calls for an increase in the consistent and systematic reporting of quality improvement interventions.

    AB - AbstractIntroduction Quality improvement is an inherent part of modern healthcare systems worldwide, used for the continuous advancement in effectiveness and safety. Amongst other approaches, the plan-do-study-act (PDSA) method, a four-step iterative method, is widely used for testing and implementation of quality-improving interventions in health care despite a poor evidence of its effectiveness. Accepting the premise that the underlying purpose of improvements in health care is to improve patients’ health as suggested by, amongst others, the American National Academy of Medicine, the data measurements used to assess the impact of quality improvement projects, consequently, ought to reflect how they eventually impact patients’ health.Objectives The purpose of the present paper is to review recently published PDSA projects to investigate how the effect of quality improvements are being measured, and if the authors provide scientific evidence that their choice of effect measure, i.e. the quality indicator, is associated with patients’ health. Methods The basis of the present study was a systematic review of studies on PDSA quality improvement projects published in 2015-2017. For all identified papers, the primary quality indicators were categorized in accordance with A. Donabedian’s three definitions of dimensions of quality, i.e., structure, process, and outcome indicators. Secondly, it was assessed if the authors of the studies reported the applied quality indicator as being evidence-based, i.e. whether they were associated with patient-relevant outcomes. In addition, adherence to the SQUIRE guidelines as reported in the studies was observed. The assessment was performed individually by NKS and ASV. Any disagreements were resolved through discussion.Results In all, 54 studies were included for assessment. One study was excluded, as reporting of any quality indicators was absent. The result of the systematic review showed that one out of 53 studies applied a structure indicator, in 83 percent (44/53) of the studies process indicators were applied, and outcome indicators were applied in 15 percent of the studies (8/53). In general, the quality indicators were very heterogeneous causing incomparability of results. The results from the assessment of whether authors identified the applied quality indicator as being an evidence-based indicator showed that only in four studies, the authors reported them as such. Conclusion Process indicators, rather than health-related outcome measures, appear to be used most often in quality improvement projects applying the PDSA method. Evidence-based indicators were only applied in four studies. Overall, this challenges the ability to show if, and how, interventions actually affect patients’ health. This is problematic for a number of reasons; if the applied quality indicator does, in fact, not signify final impact patients’ health, the value of the interventions might be questioned. If we cannot confirm an association between applied quality indicators and final impact on health, we may potentially harm patients indirectly by pursuing ineffective solutions. This would be a direct antithesis to the main purpose of the health care system. This study is based on assessments of self-reported information in the studies and, consequently, the results presented here is a reflection of the information reported in the studies. However, systematic reporting guidelines were only applied in two studies. Enhancement of the validity of the present results calls for an increase in the consistent and systematic reporting of quality improvement interventions.

    KW - PDSA method, effectiveness, quality indicators, evidence based, patient outcomes

    M3 - Conference abstract for conference

    ER -

    Hermansen NK, Vestergaard AS, Ehlers LH. DOES QUALITY IMPROVEMENT IMPROVE PATIENTS’ HEALTH? A SYSTEMATIC REVIEW OF MEASURES OF EFFECT USED IN PDSA PROJECTS. 2018. Abstract from 5th Nordic Conference on Research in Patient Safety and Quality in Healthcare, København, Denmark.