핵심 개념
Joint-Task Regularization (JTR) leverages cross-task relationships to simultaneously regularize all tasks in a single joint-task latent space, improving learning when data is not fully labeled for all tasks.
초록
The content discusses the problem of multi-task learning (MTL) with partially labeled data. Most existing MTL methods require fully labeled datasets, which can be prohibitively expensive and impractical to obtain, especially for dense prediction tasks.
The authors propose a new approach called Joint-Task Regularization (JTR) to address this issue. JTR encodes predictions and labels for multiple tasks into a single joint-task latent space and regularizes the encoded features by a distance loss in this space. This allows information to flow across multiple tasks during regularization, leveraging cross-task relationships. Additionally, JTR scales linearly with the number of tasks, unlike previous pair-wise task regularization methods which scale quadratically.
The authors extensively benchmark JTR on variations of three popular MTL datasets - NYU-v2, Cityscapes, and Taskonomy - under different partially labeled scenarios. JTR outperforms existing methods across these benchmarks, demonstrating its effectiveness in achieving data-efficient multi-task learning.
통계
Annotating a segmentation mask for a single image in the Cityscapes dataset took over 1.5 hours on average.
The NYU-v2 dataset has an additional 47,584 samples labeled only for the depth estimation task, which is approximately 60 times larger than the multi-task training split.