Sparse Concept Bottleneck Models: Leveraging Gumbel Tricks for Interpretable and Accurate Image Classification

The authors propose a novel framework for building Concept Bottleneck Models (CBMs) from pre-trained multi-modal encoders like CLIP. Their approach leverages Gumbel tricks and contrastive learning to create sparse and interpretable inner representations in the CBM, leading to significant improvements in accuracy compared to prior CBM methods.