Core Concepts
REBAR method uses retrieval-based reconstruction to identify positive pairs in time-series contrastive learning, achieving state-of-the-art performance.
Abstract
1. Abstract
Self-supervised contrastive learning relies on identifying positive data pairs for downstream tasks.
Classical vision approaches use augmentations, but time-series lack obvious invariances.
REBAR proposes a novel method using a learned measure for identifying positive pairs.
2. Introduction
Self-supervised learning enables rich representations without labels, crucial for health applications.
Contrastive learning constructs embeddings based on positive and negative pairs of unlabeled samples.
Vision applications use augmentations to construct positive pairs, but time-series lack such invariances.
3. Data Extraction
"Our main contributions in this work are:"
"This is the first work to use a similarity measure to select positive and negative pairs in time-series contrastive learning."
"We demonstrate that our learned measure predicts mutual class membership in a nearest neighbor sense."
4. Related Work
Augmentation-based methods are common in time-series research, but the effectiveness varies across different works.
Sampling-based methods like TNC sample nearby subsequences as positives and utilize hyperparameters for negatives.
5. Notation
Dataset designated by A ∈ RN×U×D with N long time-series of U temporal length and D channels.
X(i) ∈ RT ×D is a subsequence of A(i) with length T.
6. REBAR Approach
REBAR cross-attention reconstructs ¯Xanchor by retrieving motifs from Xcand that match the context window.
Design focuses on class-discriminative reconstruction using dilated convolutions for motif comparison.
7. Applying REBAR Measure in Contrastive Learning
Positive instances are selected based on similarity to anchor using REBAR measure.
Within-time-series loss captures how class labels change over time, while between-time-series loss captures differences among patients.
8. REBAR Nearest Neighbor Validation Experiment
Validates that REBAR predicts mutual class membership effectively through nearest neighbor classification experiments.
9. Downstream Experiments and Results
Linear probe classification results show that REBAR consistently outperforms other contrastive learning methods.
Stats
"Through validation experiments, we show that the REBAR error is a predictor of mutual class membership."
"Our main contributions in this work are:"
"This is the first work to use a similarity measure to select positive and negative pairs in time-series contrastive learning."