VISAP - Reimagining Archives: AI, Performance, and Cultural Memorybookmark_border

This paper presents The Ocular Harpsichord 2.0, an audiovisual instrument for exploring small media archives through MIDI-triggered latent space navigation. The aesthetic and formal tendencies in audiovisual collections tend to be hard to grasp and remain underexplored; the instrument makes the interior logic of a bounded corpus legible by letting a performer play it. The system trains on a specific collection and learns how sound and image are tied together in that particular archive. These relations are specific to one collection, shaped by its own textures and rhythms. The performer navigates the archive through gesture and MIDI input, and the audiovisual output follows the collection’s own structure. Using visualization of the latent space through dimensionality reduction, the performer is invited to examine the learned terrain. The result is framed here as an epistemological instrument: a playful, interactive way of exploring and amplifying the inherent aesthetics of an audiovisual data set.

Visualization research often begins from stable, pre-existing datasets. Yet contemporary AI systems do not merely display data; they actively produce it, transforming bodies, materials, and language into machine-readable signals and generated outputs. Peeling Cycle (2026) is a live audiovisual performance that makes this production visible. Rather than visualizing a fixed dataset, it visualizes the live transformation of bodily, material, semantic, and synthetic data through AI perception. The work integrates endoscopic imagery, breath classification through spectral analysis, DIY laser projection, live object detection, word-vector association, language-model interpretation, customized voice synthesis, and real-time image generation through TouchDesigner and Ableton Live, staging a continuous pipeline through which breath, cloth, laser traces, and gesture become synthetic voice, symbolic objects, and audiovisual rupture. We read this process through the lens of amplification. Amplification here is ambivalent: it renders hidden bodily and material signals perceptible, but in doing so it also intensifies exposure, classification, gendered semantic association, and misrecognition. This paper contributes: first, a conceptual framework that understands synthetic datafication as a performative visualization process; second, a live AI system that transforms bodily, material, semantic, and synthetic data into audiovisual form; and third, a critical analysis of how amplification produces gendered visibility through sensing, scanning, semantic steering, synthetic rendering, and rupture, grounded in two performed iterations of the work and the audience feedback they generated.

This research investigates how machine learning and data visualization can reveal the multiple and sometimes divergent ways artistic identity is constructed across descriptions of artworks. Drawing on texts authored by artists, curators, and cataloguers, the project treats identity not as a fixed or singular attribute, but as a mutable and relational concept that emerges through interactions among different sources and perspectives. Through embedding-based analysis and interactive network visualization, the proposed methodology maps relationships between these textual descriptions, making visible patterns of convergence, divergence, and distance across forms of authorship. The resulting visualizations reveal how institutional and self-authored texts can construct distinct representations of artistic identity, while demonstrating how computational systems can make these differences legible as structured relationships rather than isolated or anecdotal contradictions.

This research presents a visualization methodology for creating alternative archival connections within incomplete, fragmented, and historically uneven collections. Meta-Acervos is an interactive meta-archive system that integrates information from dispersed art collections held by public museums in Brazil, using computational analysis and interactive visualization to foreground relationships between objects across institutional boundaries. Rather than functioning solely as an interface for accessing museum artworks, the system explores how visualization and interface design can support alternative discursive, curatorial, and museological approaches to cultural heritage. By challenging archival structures organized around established schools, artistic movements, techniques, and historical periods, Meta-Acervos investigates how computational systems can produce different ways of encountering and interpreting collections. Situated within the context of the Global South, the project also examines the contradictions of archival digitization in Brazil, where preservation, classification, and access are shaped by histories of inequality, institutional priorities, and colonial knowledge structures. The research argues that visualization can function not only as a means of representing archival information, but also as a critical method for interrogating and reconfiguring the structures through which cultural heritage is organized, while making visible the assumptions and biases embedded in archival data and computational technologies.

Recent advances in generative artificial intelligence have enabled the synthesis of complex human motion with unprecedented fidelity. However, current text-to-motion systems rely strictly on linguistic semantics: if an input reads "I put my hands up", the model searches for a pose with raised hands, and every non-semantic property of the text is discarded as noise. In this work, we treat that discarded structure as the signal. We present an embodied visualization instrument that amplifies not what a text means, but how it is built. Our method first quantizes dance kinematics into a motion codebook of 256 stylistic "regions" using Principal Component Analysis and K-Means clustering, and orders those regions along the dominant axis of movement. We then map the raw byte representation of any input text directly onto this codebook, producing a deterministic sequence of regions that we call the text's "choreographic genome". A precomputed plausibility graph and a set of physics smoothing routines turn this genome into fluid, full-body movement, so that the dancing body becomes a display surface for the byte-level structure that semantic systems ignore. Through a series of artistic case studies, including a Shakespeare sonnet, a machine error log, source code, an abolitionist's question, and Indigenous and Devanagari scripts, we show that each text produces a visibly distinct dance, and that scripts marginalized by ASCII-centric computing are amplified into close to three times as much movement per character. We frame this not as a motion-synthesis benchmark, but as a critical and poetic visualization that asks what we choose to count as signal, and what we allow to go unheard.