Right to Contest AI Diagnostics: Defining Transparency and Explainability Requirements from a Patient’s Perspective

The problem of the transparency and explainability of AI decision-making has attracted considerable attention in recent years. In this chapter, we argue that patients have a right to contest AI medical decisions and that the transparency requirements of AI decision-making in health care should be guided by this right. We define the right to contest AI medical decisions both formally and substantially. Formally, the right to contest AI medical decisions must be a right (i) that is grounded in moral values, (ii) that is effective in protecting patients’ rights, (iii) that is proportional to the potential costs to others, and (iv) that is an application of a more general right to contest medical decisions. Substantially, the right to contest AI medical decisions should enable patients to contest (I) the AI system’s use of personal and sensitive data, (II) the system’s potential biases, (III) the system performance, and (IV) the division of labor between the system and healthcare professionals. We justify and define 14 specific informational requirements - i.e., transparency requirements - that follow from the substantial notion of the right to contest AI medical decisions. Finally, we briefly discuss the patient-centered approach taken in this chapter against alternative approaches grounding transparency requirements in considerations of democracy and the interests of science.