A Visual Approach to Auditing Cross-Lingual Attribution Bias in LLMs

Abstract

Large language models (LLMs) are increasingly consulted as informal knowledge sources, yet the answer a model gives can depend on the emph{language} the question is asked in. We present an interactive lab that probes this cross-lingual divergence by asking the same factual question in several languages across several models, reducing each free-text answer to a canonical cited entity, and surfacing where the attributions disagree. The system contributes (i) a reproducible generate-extract-cluster pipeline that turns multilingual prose into comparable entity citations; (ii) two interpretable metrics, Core Answer Flexibility (CAF) and Context Richness (CR), that separate whether the headline answer changes from whether alternatives are acknowledged; and (iii) four coordinated views for reading bias at a glance. Across two probes, "Who invented the airplane?" and "Who invented the telephone?", in three to four languages over seven models, we find cross-lingual bias rarely flips the main answer; instead, it surfaces in the also-cited periphery, and the sharpest axis separating models is whether they acknowledge alternatives at all.