This paper examines the tensions between invariance and adaptation in natural language generation (NLG) systems. The authors conduct five case studies that perturb different types of identity-related language features, such as names, roles, locations, dialect, and style, in NLG system inputs. Through these case studies, they observe actual system behaviors and examine people's expectations of system behaviors.
The key findings are:
Observed system behaviors can differ in terms of coherence, sentiment and affect, formality, textual complexity, and identity-related assumptions. Some systems also exhibit blocking behaviors in response to certain identity-related features.
People's expectations of system behaviors vary widely. Some favor adaptation to identity-related features to account for social norms, cultural differences, and feature-specific information. Others prefer invariance, citing reasons like prescriptivism, perceived unnecessary complexity of adaptation, and concerns about false assumptions.
The authors highlight the open challenges around specifying "fair" or "good" NLG system behaviors, as people have diverging views on whether systems should be invariant or adapt to identity-related language features.
إلى لغة أخرى
من محتوى المصدر
arxiv.org
الرؤى الأساسية المستخلصة من
by Li Lucy,Su L... في arxiv.org 04-04-2024
https://arxiv.org/pdf/2310.15398.pdfاستفسارات أعمق