4 Citations (Scopus)
412 Downloads (Pure)


The recently emerged field of explainable artificial intelligence (XAI) attempts to shed lights on 'black box' Machine Learning (ML) models in understandable terms for human. As several explanation methods are developed alongside different applications for a black box model, the need for expert-level evaluation in inspecting their effectiveness becomes inevitable. This is significantly important for sensitive domains such as medical applications where evaluation of experts is essential to better understand how accurate the results of complex ML are and debug the models if necessary. The aim of this study is to experimentally show how the expert-level evaluation of XAI methods in a medical application can be utilized and aligned with the actual explanations generated by the clinician. To this end, we collect annotations from expert subjects equipped with an eye-tracker while they classify medical images and devise an approach for comparing the results with those obtained from XAI methods. We demonstrate the effectiveness of our approach in several experiments.
Original languageEnglish
Title of host publicationPattern Recognition. ICPR International Workshops and Challenges
EditorsAlberto Del Bimbo, Rita Cucchiara, Stan Sclaroff, Giovanni Maria Farinella, Tao Mei, Marco Bertini, Hugo Jair Escalante, Roberto Vezzani
Number of pages12
PublisherSpringer Publishing Company
Publication dateFeb 2021
Pages 35-46
ISBN (Print)978-3-030-68795-3
ISBN (Electronic)978-3-030-68796-0
Publication statusPublished - Feb 2021
EventPattern Recognition. ICPR International Workshops and Challenges - Online
Duration: 10 Jan 202115 Jan 2021


ConferencePattern Recognition. ICPR International Workshops and Challenges
SeriesLecture Notes in Computer Science


  • Explainable AI (XAI), Deep learning, Expert-level explanation, XAI evaluation, Retinal Images, Eye-tracker


Dive into the research topics of 'Expert level evaluations for explainable AI (XAI) methods in the medical domain'. Together they form a unique fingerprint.

Cite this