
Current automatic metrics struggle to accurately capture the nuances of meeting summarization, often masking or rewarding errors, and failing to reflect the severity of issues in generated summaries.


coremsg

evaluating-automatic-metrics-for-meeting-summarization-uncovering-limitations-and-biases


Evaluating Automatic Metrics for Meeting Summarization: Uncovering Limitations and Biases


title_rewrite


CREAM, a novel framework, addresses the unique challenges of evaluating meeting summaries by leveraging comparison-based metrics and an Elo ranking system to assess conciseness and completeness without requiring reference texts.


comparison-based-reference-free-elo-ranked-automatic-evaluation-for-meeting-summarization


Comparison-Based Reference-Free Elo-Ranked Automatic Evaluation for Meeting Summarization
