IEEE VIS 2025 Content: NLI4VolVis: Natural Language Interaction for Volume Visualization via LLM Multi-Agents and Editable 3D Gaussian Splatting

NLI4VolVis: Natural Language Interaction for Volume Visualization via LLM Multi-Agents and Editable 3D Gaussian Splatting



 Kuangshi Ai -

 Kaiyuan Tang -

 Chaoli Wang -

 Screen-reader Accessible PDF

 Download Supplemental Material

Image not found — This work will interest practitioners in scientific visualization, LLM multi-agent system, and human-computer interaction—especially those working with complex 3D volumetric data. Practitioners can use NLI4VolVis to explore and edit volumetric datasets using natural language, simplifying tasks like object selection, view manipulation, lighting management, and stylization. The system lowers the barrier for non-experts, enhances accessibility, and provides a flexible framework for integrating LLM multi-agents into interactive visualization workflows.

Keywords

Volume visualization, novel view synthesis, natural language interaction, open-vocabulary querying, editable 3D Gaussian splatting, multi-agent collaboration

Abstract

Traditional volume visualization (VolVis) methods, like direct volume rendering, suffer from rigid transfer function designs and high computational costs. Although novel view synthesis approaches enhance rendering efficiency, they require additional learning effort for non-experts and lack support for semantic-level interaction. To bridge this gap, we propose NLI4VolVis, an interactive system that enables users to explore, query, and edit volumetric scenes using natural language. NLI4VolVis integrates multi-view semantic segmentation and vision-language models to extract and understand semantic components in a scene. We introduce a multi-agent large language model architecture equipped with extensive function-calling tools to interpret user intents and execute visualization tasks. The agents leverage external tools and declarative VolVis commands to interact with the VolVis engine powered by 3D editable Gaussians, enabling open-vocabulary object querying, real-time scene editing, best-view selection, and 2D stylization. We validate our system through case studies and a user study, highlighting its improved accessibility and usability in volumetric data exploration. We strongly recommend readers check out our case studies, demo video, and source code at https://nli4volvis.github.io/.