Observer agreement of treatment responses on planar bone scintigraphy in prostate cancer patients: importance of the lesion assessment method

PURPOSE: The aim of this study was to assess observer agreement on the evaluation of treatment responses of bone metastases by bone scintigraphy (BS) using different scoring methods in prostate cancer patients.

PATIENTS AND METHODS: Sixty-three paired BS from 55 patients were included. BS was performed before and after more than 12 weeks of anticancer treatment. A panel of experienced nuclear medicine physicians from several institutions evaluated treatment response using three different methods: (a) standard clinical assessment, (b) MD Anderson criteria, and (c) Prostate Cancer Working Group 2 (PCWG-2) criteria. All methods were based on the evaluation of paired before-after bone scans.

RESULTS: Readers were able to classify the presence of bone metastases at baseline with a high level of agreement [Cohen's κ=0.94, 95% confidence interval (CI) 0.82-1.00]. Observer agreement on bone response by PCWG-2 criteria showed considerable agreement (Cohen's κ=0.84, 95% CI: 0.69-0.99). Evaluation using standard clinical assessment and MD Anderson criteria showed moderate agreement (0.52, 95% CI: 0.36-0.69 and 0.64, 95% CI: 0.48-0.79, respectively). There was considerable variation among readers for regional lesion count on individual scans, with limits of agreement of -10 to 10 lesions or more for the majority of anatomical regions, including the thorax, spine, and pelvis.

CONCLUSION: Observer agreement on treatment response by BS varied notably across methods. Optimal agreement was achieved by the PCWG-2 criteria. Variation in the classification of treatment response of bone metastases may have a significant impact on clinical decision-making, emphasizing the need for a uniform approach, including during clinical practice. Response assessment by lesion counting on repeated BS without access to previous scans cannot be recommended.