Belangrijkste concepten
The authors introduce new families of surrogate losses for the multi-class abstention loss function, including state-of-the-art surrogate losses in the single-stage setting and a novel family of loss functions in the two-stage setting. They prove strong non-asymptotic and hypothesis set-specific consistency guarantees for these surrogate losses, which upper-bound the estimation error of the abstention loss function in terms of the estimation error of the surrogate loss.
Samenvatting
The paper analyzes the score-based formulation of learning with abstention in the multi-class classification setting. The authors introduce new families of surrogate losses for the abstention loss function, which include the state-of-the-art surrogate losses in the single-stage setting and a novel family of loss functions in the two-stage setting.
For the single-stage setting, the authors prove strong non-asymptotic and hypothesis set-specific consistency guarantees for these surrogate losses, which upper-bound the estimation error of the abstention loss function in terms of the estimation error of the surrogate loss. These guarantees are more favorable than the existing asymptotic consistency guarantees.
For the two-stage setting, the authors propose surrogate losses and prove that they benefit from similar strong consistency guarantees. They show that the two-stage formulation is also realizable H-consistent, which addresses an open problem in the literature.
The authors experimentally evaluate their new algorithms on CIFAR-10, CIFAR-100, and SVHN datasets and demonstrate the practical significance of their new surrogate losses and two-stage abstention algorithms. The results also show that the relative performance of the state-of-the-art score-based surrogate losses can vary across datasets.
Statistieken
The authors use the following datasets in their experiments:
CIFAR-10
CIFAR-100
SVHN