Core Concepts
Audiovisual display idioms that systematically integrate sonification and visualization offer potential for enhanced data exploration and presentation compared to unimodal approaches.
Abstract
This state-of-the-art report provides a comprehensive overview of academic contributions that blend sonification and visualization within the context of data exploration and data presentation. The authors first provide background on the field of sonification and its relation to visualization. They then describe their systematic literature search and filtering process, which resulted in a corpus of 57 relevant publications from 2011 to 2023.
The authors classify the literature along various dimensions, including:
- Purpose (exploration, presentation, or both)
- Visualization and sonification techniques used
- Reading level, search level, and data types
- Mapping redundancy between visual and auditory channels
- Evaluation methods
- Target platforms and interaction possibilities
- User groups and design goals
Through this classification, the authors identify key trends, design patterns, and research gaps in the field of integrated audiovisual displays. They also discuss three adjacent topics - accessibility, monitoring, and data art - that are related to but not the primary focus of this review.
The findings of this report can help researchers from both the visualization and sonification communities understand the potentials and challenges of such integrated designs, while hopefully inspiring them to collaborate across disciplines.
Stats
"Audiovisual display idioms that systematically integrate sonification and visualization offer potential for enhanced data exploration and presentation compared to unimodal approaches."
"The authors classify the literature along various dimensions, including: Purpose (exploration, presentation, or both), Visualization and sonification techniques used, Reading level, search level, and data types, Mapping redundancy between visual and auditory channels, Evaluation methods, Target platforms and interaction possibilities, User groups and design goals."
Quotes
"Both the theoretical cross-pollination and the practical integration and combination of sonification and visualization offer potential for interesting research outcomes."
"We believe that a well-designed audiovisual representation can be more than the mere sum of a visual and an auditory representation."
"The findings of this report may be used by researchers from both fields to understand the potentials and challenges of such integrated designs while hopefully inspiring them to collaborate with experts from the respective other field."