Core Concepts
Allowing humans to interactively guide machine attention does not consistently improve the accuracy of human-AI teams in fine-grained image classification tasks.
Abstract
The paper introduces an interactive interface called CHM-Corr++ that enables users to guide the attention of an image classification model (CHM-Corr) by selecting image patches of interest. The goal is to explore whether this interactive approach can enhance users' understanding of the model and improve their decision-making accuracy compared to static explanations.
The key findings are:
Participants struggled to reject incorrect model predictions, regardless of the type of explanation provided.
Contrary to expectations, the interactive dynamic explanations did not improve participants' decision accuracy compared to static explanations.
The usefulness of interactivity depended on the interaction outcomes - when the model maintained its initial (correct) prediction, interactivity was helpful, but when it maintained an incorrect prediction, interactivity was less effective.
The authors hypothesize that the limited effectiveness of interactivity may be due to the nature of the task (fine-grained bird classification) where the AI attention is already sufficient, as well as the inherent shortcomings of the base CHM-Corr classifier.
The findings challenge the common assumption that interactivity inherently boosts the effectiveness of explainable AI (XAI) systems. The work contributes an interactive tool for manipulating model attention and lays the groundwork for future research on enabling effective human-AI collaboration in computer vision.
Stats
The model (CHM-Corr) initially correctly classified 300 samples and misclassified 300 samples from the CUB-200 test set.
Quotes
"Allowing humans to interactively guide machines where to look does not always improve a human-AI team's classification accuracy"
"Our user study with 18 machine learning researchers who performed ∼1,400 decisions shows that our interactive approach does not improve user accuracy on CUB-200 bird image classification over static explanations."