Keskeiset käsitteet
Conformal prediction sets can assist in labeling out-of-distribution images, but their utility varies based on image difficulty and set size.
Tiivistelmä
The study evaluates the utility of conformal prediction sets for AI-advised image labeling through a large online experiment. It compares prediction sets to Top-1 and Top-𝑘 displays, finding that prediction sets excel at labeling out-of-distribution images, especially with small set sizes. The study highlights challenges and implications for real-world decision-making.
Tilastot
Conformal prediction sets excel at assisting humans in labeling out-of-distribution images, especially with small set sizes.
Prediction sets lead to reduced labeling accuracy for in-distribution instances compared to Top-𝑘 predictions.
Participants are willing to pay roughly equivalent amounts for each type of display.
Lainaukset
"Prediction sets excel at assisting humans in labeling out-of-distribution images, especially when the set size is small."
"For in-distribution instances, prediction sets lead to reduced labeling accuracy compared to Top-𝑘 predictions."