Core Concepts
Criteria for detecting uncertainty-based corner cases in object instance segmentation models without relying on ground truth data.
Abstract
The article presents an approach to detect uncertainty-based corner cases in object instance segmentation models. The key aspects are:
The authors use a probabilistic definition of object detectors and instance segmentation networks to model the predictive uncertainty. This allows them to derive criteria based on the uncertainty in classification, bounding box regression, and instance mask prediction.
The proposed corner case criteria cover the uncertainty in class scores, bounding box parameters, and instance masks. The authors also introduce combined criteria that consider the mismatch between bounding box and mask predictions.
The authors evaluate the proposed criteria using the COCO and NuImages datasets. They perform feature selection to identify the most important criteria and use them as input to a decision function that classifies object detections into True Positive, Localization Corner Case, Classification Corner Case, Localization & Classification Corner Case, and False Positive.
The authors demonstrate the application of the corner case criteria in an iterative training cycle, where they selectively add model-detected corner cases to the training dataset to improve the overall model performance.
Stats
The mean class score Dckmax and its standard deviation σckmax for the class with the highest confidence.
The mean class score Dck2nd and its standard deviation σck2nd for the class with the second-highest confidence.
The mean bounding box Db and its standard deviation σb.
The mean IoU ioub between the mean bounding box Db and all other bounding box predictions bj, and its standard deviation σioub.
The mean instance mask Dm and its bounding box Dmbox, along with the standard deviation σmbox of the mask bounding box.
The mean IoU ioum between the mean instance mask Dm and all other mask predictions mj, and its standard deviation σioum.
The mean area Am of the instance masks and its standard deviation σAm.
The IoU ioumis between the mean bounding box Db and the bounding box Dmbox enclosing the mean instance mask Dm.
The Kullback-Leibler Divergence KL(pb|pm) and KL(pm|pb) between the bounding box IoU distribution pb and the mask IoU distribution pm.
The Jensen-Shannon Distance JS(pb|pm) between the bounding box IoU distribution pb and the mask IoU distribution pm.
The Earth Mover's Distance EMD(pb|pm) between the bounding box IoU distribution pb and the mask IoU distribution pm.
Quotes
"Corner Cases [3, 19, 39] are strongly related to anomalies [9, 14, 37], outliers [14, 17], and novelties [9, 14] but also cover samples where the model fails [12, 19, 22, 36] and data relevant for model improvement [6, 39]."
"Corner case detection enables data selection to be guided to identify valuable data and label it more efficiently, offering tremendous cost-saving potential. Besides, there are also other use cases, e.g., active learning [25], novelty detection [14, 37], and dataset construction, i.e., creating a training and testing dataset covering all relevant and therefore crucial situations."