A national drug related problems database: Evaluation of use in practice, reliability and reproducibility

Background: A drug related problems database (DRP-database) was developed on request by clinical pharmacists. The information from the DRP-database has only been used locally e.g. to identify focus areas and to communicate identified DRPs to the hospital wards. Hence the quality of the data at the national level is unknown, which may compromise national analyses for benchmarking and identification of national focus areas. Objective: The aim of the study was to evaluate the use in practice, reliability and reproducibility of the DRPs documented in the Danish drug related problems database. Setting: Danish hospital pharmacies. Methods: Practice use of the DRP-database was explored by an electronic questionnaire distributed to hospital pharmacies, and consisted of questions regarding current and previous use of the DRP-database. The reliability was evaluated by comparing the categorization of 24 cases by clinical pharmacists with categorization performed by the project group. Reproducibility was explored by re-categorization of a sample of existing records in the DRP-database by two project group members individually. Main outcome measures: Observed proportion of agreement and Fleiss' kappa as measures of inter-rater reliability and reproducibility. Results: The practice use study of 12 hospital pharmacy locations revealed that when implementing the DRP-database, the majority of identified DRPs are documented in the DRP-database, however, some variations throughout the country exist. The interrater reliability study of 34 clinical pharmacists showed high inter-rater reliability with the project group (Fleiss' kappa = 0.79 with 95 % CI (0.70; 0.88)), and the reproducibility study also documented high inter-rater reliability of a sample of 379 records from the DRP-database re-categorized by two project group members (Fleiss' kappa = 0.81 with 95 % CI (0.78; 0.85)). Conclusion: The study showed high reliability and reproducibility of the DRP-database, however, some local variation in the use of the DRP-database throughout the country existed affecting the overall quality. These findings indicate that data in the DRP-database may be pooled, and national analyses may be conducted to explore development areas for common interest.