The LLM Belief Atlas: Visualizing How Language-Model Stances Depend on Elicitation

Abstract

What a language model appears to believe about a proposition is not a scalar but a signal that depends on how it is elicited and read: forcing a choice can fabricate a value, option-logprob scoring carries order bias, and sycophancy makes a stance a function of framing. We present the LLM Belief Atlas, a scalable in-browser WebGL instrument organized around a framework of belief spaces. The framework treats a model's stance as a latent construct observed only through a four-dimensional measurement space, model, item, elicitation condition, and readout channel, and names seven subspaces, each bound to a facet of psychometric construct validity. The instrument co-registers three switchable belief readouts (Likert agreement, free-text reflection, and option logprobs) of the same elicitation instances and morphs between them at the million-point scale, while exposing uncertainty. On 76,048 elicited beliefs from four LLMs, framing conditions explain a share of stance variance comparable to belief itself (27.1% vs 35.9%, and larger on the two natively-read models), and a within-model reliability measure predicts when cross-model disagreement is reproducible.