IEEE VIS 2024 Content: Integrating Annotations into the Design Process for Sonifications and Physicalizations

Integrating Annotations into the Design Process for Sonifications and Physicalizations



 Rhys Sorenson-Graff - Whitman College, Walla Walla, United States

 S. Sandra Bae - University of Colorado Boulder, Boulder, United States

Jordan Wirfs-Brock - Whitman College, Walla Walla, United States

 Download Supplemental Material

 Room: Bayshore VI

2024-10-17T15:00:00ZGMT-0600Change your timezone on the schedule page
2024-10-17T15:00:00Z

Exemplar figure, described by caption below — Examples of geometric annotations used in a visualization, sonification, and physicalization. Geometric annotations draw attention to a specific section of the data representation, providing additional context, detail, and clarity to a section if it contains crucial information or is of significant interest to the viewer. Visualizations can integrate geometric annotations with call-out boxes. Sonifications can highlight specific excerpts using sub-clips of audio. Physicalizations can present multiple frames of reference to emphasize different perspectives that zoom in and out of the physicalization (photo credit to Klauss et al.)

Fast forward

Full Video

Keywords

Annotations, physicalization, sonification

Abstract

Annotations are a critical component of visualizations, helping viewers interpret the visual representation and highlighting critical data insights. Despite their significant role, we lack an understand- ing of how annotations can be incorporated into other data representations, such as physicalizations and sonifications. Given the emergent nature of these representations, sonifications, and physicalizations lack formalized conventions (e.g., design space, vocabulary) that can introduce challenges for audiences to interpret the intended data encoding. To address this challenge, this work focuses on how annotations can be more tightly integrated into the design process of creating sonifications and physicalizations. In an exploratory study with 13 designers, we explore how visualization annotation techniques can be adapted to sonic and physical modalities. Our work highlights how annotations for sonification and physicalizations are inseparable from their data encodings.