BELIV: Learning What's True, Doing What's Rightbookmark_border

Provenance is commonly understood as a record of the processes by which an analysis was produced, and provenance research has accordingly focused on faithfully capturing interactions, workflows, and computational steps to support reproducibility and reuse. We argue that this framing overlooks a more fundamental question: what evidence of an analysis should be preserved in the first place? Drawing on archival science, history, and human sensemaking, we reframe provenance capture as an epistemic and archival design problem rather than a purely technical one. We examine it through three questions, concerning the sources of provenance, the selection of observable traces, and the timing of capture, and offer a set of provocations that expose assumptions embedded in current provenance systems, with implications for both human- and AI-assisted analysis. Provenance systems, we argue, do not simply record analyses; they shape the evidence from which analyses are later interpreted, reproduced, and understood.

Contribution statements are an increasingly common way to make research labor visible, reduce academic malfeasance, and provide broader transparency. Despite this potential value, they remain uncommon in visualization and HCI. To explore this gap, we conducted an online study with (N=21) visualization and HCI researchers. We find a range of differing opinions about the utility of contribution statements, which are set against a background of tensions relating to contribution frameworks that inadequately fit contribution types in HCI and especially visualization, power dynamics between authors, bias in authorship perceptions, and the tedium of providing yet another form of documentation. From these factors, we offer a modest recommendation to authors: consider contribution statements. There are contexts when they may usefully explicate work, and others where they can cause author-team conflict or become a burdensome chore. Regardless of whether they are used by readers or not, we suggest that scaffolded mechanisms for reflecting on contribution roles are valuable both for public accountability and internal alignment. To institutions, we recommend that contribution statements be exempt from page or word limits, and that flexible templates and examples be provided to authors, but that they continue to not be required. By surfacing these perspectives, we seek to open a dialogue about what constitutes authorship, how our community might move toward more equitable and transparent attribution practices, and where the visualization and HCI communities might be uniquely equipped to help.

In this work, we present how employing the lens of entanglement helped us examine and reframe epistemic tensions arising in a visualization co-design project. Entanglement theory challenges traditional assumptions in the visualization research community by emphasizing that knowledge is not produced through linear, isolated processes, but is inherently entangled with phenomena and apparatuses. While this perspective offers a compelling critique of conventional research practices, its practical value for visualization research remains underexplored. We apply the entanglement lens to examine and reframe the epistemic tensions that emerged in a longitudinal community-based visualization co-design project. Our experience shows that the entanglement perspective not only provides a richer understanding of these tensions, but also helps transform them into generative opportunities for methodological and theoretical reflection. Applying this lens enabled us to critically interrogate the language used in research, to develop a more nuanced understanding of visualization co-design, and to surface ``dark sides'' of conventional visualization design pipelines. These contributions illustrate the practical value of embracing entanglement as an epistemological lens for visualization research.

In this position paper, we argue for the importance of containerizing visual analytics artifacts to improve the scientific progress of our research community. First, based on our previous experience authoring, reading, and reviewing visual analytics research from the previous decade, we identify risks and impediments to empirical research in visual analytics within the broader reproducibility crisis, including an inability to compare systems and techniques and a slow pace of adjusting to a rapidly changing ecosystem of analysis methods, such as machine learning and artificial intelligence. Next, we analyze the benefits of different artifact sharing and deployment setups across five artifact goals: reproducibility, replicability, portability, performance, and scalability. Then, we argue that containerization addresses the issue of reproducibility in visual analytics research, with downstream benefits in terms of replicability, portability, performance, and scalability. To demonstrate the ease of use and the utility of containerization, we deploy ten visual analytics systems developed over the course of a graduate seminar onto a live server using containerization with the help of agentic AI. We share lessons learned through the containerization process, and describe a suite of potential benefits for our research community as well as research opportunities for leveraging modularity to improve the artifacts themselves.

A common expectation in visualization research is that outcomes recommend how researchers and practitioners take action or make design decisions. We often express these as “guidelines.” Yet, the term “guideline” is both ambiguous and loosely defined, and what one researcher considers a guideline may be too broad, too loose, or too strict for another. We take a closer look at a broader set of terms that can express desirable results around visualization research, and untangle how these words are understood in the community in relation to other similar terms. We base our work on an exploratory study with experts, followed by a crowdsourcing study with a separate mapping phase (n = 30) and rating phase (n = 42) targeting input from the broader visualization community, and an analysis of the use of terminology in 3,877 IEEE VIS papers published from 1990 to 2024. Based on our findings, we call for more nuanced, precise discussions of research outcomes and their communication to the broader community, including practitioners and students.

Graphical perception studies are the visualization community's preferred tool for evaluating visualizations. By measuring how accurately people interpret various arrangements of visual marks and channels, they aim to establish best practices for encoding data visually. We argue this model of assessing visualizations is fundamentally flawed, and no amount of additional empirical studies will fix it. The problem is that visualization frameworks theorize effectiveness at the level of the encoder (i.e., which data-to-visual mappings work best for a given context). Human perception, however, consists of a fundamentally incompatible decoder that operates at a different level, namely, retinal images. This encoder-decoder asymmetry means that experimental results and guidelines are often poor predictors of actual perceptual performance. Moreover, the image that actually reaches the visual system is not determined by the encoding specification alone, but is rather emergent from interactions between encoding rules, the input data, and even micro-design parameters, all of which are invisible to encoding theory. Consequently, small changes in data distributions or seemingly minor design variations can substantially change how a visualization is perceived, even when the nominal encoding specification remains unchanged. The space of such interactions is vast and cannot be covered by running one study after another. We argue that visualizations should be studied as images, and evaluated using computational models of human vision that take pixels as input. Such models capture the perceptual representations the visual system actually constructs, shifting evaluation toward modeling the decoder rather than anchoring on abstract encoding specifications. This approach is scalable, human-grounded, and sensitive to the emergent image properties that encoding theory and graphical perception studies both miss. We first describe methodological and theoretical weaknesses of the current paradigm, and propose a theory of visualization perception grounded in summary-statistical accounts of vision. We then demonstrate how image-based vision models can predict visualization discriminability in scatterplots while reproducing established results. We close by outlining a research agenda for vision-based visualization evaluation.

Understanding and enhancing visual analytics (VA) performance is important for maximizing their impact. Existing studies have successfully applied well-established summative evaluation methods from information visualization to the VA context, yet the recent emphasis on an extra data analysis/modeling stage in the VA pipeline poses an additional challenge. Inspired by the modern concept of visualization literacy, this paper examines model literacy, namely users’ knowledge of the analysis model used in a VA technique, as an additional factor for VA performance. Results from a controlled study on the visual analysis of multidimensional data with two dimensionality-reduction models indicate a positive correlation between model-task accuracy and VA-task accuracy. The study involves two common dimensionality-reduction models, PCA and t-SNE. The correlation is stronger for PCA than for t-SNE in the current task design, a pattern consistent with the possibility that VA effectiveness is more closely associated with model literacy when model outputs are less directly readable from the visualization. Completion-time evidence does not show a stable efficiency gain, suggesting that differences in model intuitiveness may help explain when model knowledge shortens task completion and when it involves additional interpretive effort. The findings of this study suggest ways to further enrich VA evaluation methods and provide directions for developing more rigorous model-literacy assessment instruments.

We analyzed the ethics reporting in 255 IEEE VIS papers from 2024 and 2025, as published in TVCG. This analysis arose from our experience as readers and reviewers of IEEE VIS papers that such reporting is frequently incomplete or missing, as well as from investigations in which we ourselves had to answer challenges regarding ethics approval in our own work. Visualization research naturally often involves human participants, yet ethics approval and informed-consent procedures are not always explicitly reported. In our corpus, 189 papers (74.1%) reported on work involving human participants. Only 6 of them (3.2%) reported to have obtained ethics approval, an approval identifier, and having received informed consent from the participants, while 26 (13.8%) reported at least ethics approval and informed consent. These omissions do not imply that the empirical work was unethical or lacked approval. They rather show that current reporting practices make ethical approval difficult to assess. Beyond our alarming—yet true—paper title, we wish to raise awareness on how authors themselves may eventually be at risk for not properly reporting ethics. We argue that VIS should adopt clearer and more standardized ethics-reporting practices to protect participants, authors, reviewers, and editors.

How we review papers impacts the intellectual progress of our research field. A collective set of reviews decides which research is accepted and influences which work is potentially abandoned or how it is changed. In this opinion paper, we highlight ten specific issues we see with peer review at the IEEE Visualization and Visual Analytics conference (IEEE VIS) and briefly discuss options for improvement. Our focus is not on presenting solutions but on starting a broader discussion about what we value and what we need to change in how we review at IEEE VIS.

We report on three years of shared reflection among nine colleagues with different but complementary backgrounds, roles, and career stages who share a common interest in and commitment to visualization research, teaching, and learning. In our international community of practice, we reflect on our visualization belief systems through critical visualization activities that help us consider, develop and question our values in the context of our distinct personal and professional lives. To ground our research, we review the history of critical and reflective visualization. We suggest that reflective visualization practices offer opportunities for (1) deepening the role that value systems play in visualization teaching, research, and practice (including ethical, cultural, and personal values), (2) developing pedagogical activities in which diverse approaches to data visualization are used to acknowledge the role of values, and (3) nurturing critical reflective practices across communities of students, teachers, practitioners, and researchers. We anticipate that these opportunities are widely applicable to data visualization. These reflective practices may benefit from approaches that encourage surfacing the diverse values of visualization. We hope our sharing and reflection can contribute to a vibrant, ongoing discussion of the role of values in visualization research, teaching, and practice; drive the creation and sharing of new activities and approaches; and encourage the formation of new reflective and supportive groups within our larger visualization community.

In design disciplines, particularly human-computer interaction (HCI) and information visualization (VIS), researchers often need to study qualitative material that is visual in nature, like screenshots, sketches, photographs, videos, or images of physical objects and spaces. Despite an existing literature on how to analyze visual artifacts in social science and the arts, few papers reflect on the value and the means to support such analysis in the context of information visualization or graphical user interface design. This is particularly intriguing, as visual design is often the primary channel for communicating data, and visual perception is one of humans’ main epistemic sources in the world. Over the past 25 years, we have analyzed such visual artifacts across dozens of our own qualitative studies and surveys. We present six cases, reflect on four values and practical strategies for using visual methods for analyzing visual artifacts, and introduce an open archive of visual methods for HCI and VIS at visualmethods.pages.dev.

Visual analytics began with a cognitive ambition rather than a graphical one: it was defined as the science of analytical reasoning carried out through a principled combination of interactivity and visual representations of data. That ambition, while still relevant, needs to be reoriented in the modern AI era, where human-AI roles, systems, and tools will continue to evolve. The goal of this paper is to recalibrate what analytical reasoning looks like in human-AI collaboration, because that recalibration is what tells us how to evaluate and design these systems: a chart, a caption, or an interaction log can look thoroughly reasoned while a person did little more than approve what a model produced with minimal agency, and the visible trace gives no sign of the difference. An evaluation task only traces the system's reasoning, and a designer working from such evaluations has no signal about which parts of the reasoning the human should still be doing. What it cannot show is who actually paid the cost of the reasoning, and that is the question we think worth asking. The central position is this: instead of judging critical thinking by the coherence of the final artifact, we judge it by how the cost of reasoning is borne, passed on, or lost at each handoff. The handoffs are not chosen ad hoc; they are the encode-decode transitions that already structure a human-AI visualization pipeline, where intent becomes encoding, encoding becomes artifact, and artifact becomes interpretation. As it crosses a handoff, a reasoning element can sit in one of four states: transmitted, when the human bore the cost and the trace says so; offloaded, when the model bore it in the open; misattributed, when the model bore it, but the trace credits the human; and evaporated, when no one bore it, yet the artifact carries it anyway. Three of these four states can leave behind the same visible trace, so an evaluator who reads only the finished output cannot tell the reasoning an analyst genuinely did from the convincing appearance of reasoning that no one did. Asking the analyst does not rescue the judgment either: whoever is best placed to report on a handoff usually took part in it, and no one can honestly account for a cost they never paid. In high-consequence real-world settings, a clinician acting on a model's cohort or an analyst briefing a decision, tracking who paid at each handoff is what turns a trace into a diagnosis of collaboration gone wrong.

In this position paper, we argue that answer modality choices can affect the validity of visualization evaluations. Visualization evaluation studies frequently rely on specific answer modalities such as multiple-choice questions, open-ended numerical estimates, verbal explanations, or spatial interactions. Yet, answer modality is often treated as a neutral reporting mechanism rather than as a factor that may influence reasoning strategies, representational transformations, and ultimately measured performance and the conclusions researchers can draw from their observations. We discuss two broad classes of modality effects on participant performance: changes in participants' task-solving strategies and changes induced by representational transformations between the visual stimulus and the expected response format. We further argue that answer modality determines what evidence about participants' reasoning is preserved for analysis. We ground our reflections in observations from prior empirical work and propose practical guidance for reasoning about answer modality as part of visualization evaluation design.