Visual Computing

University of Konstanz
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations),

Revealing the Unwritten: Visual Investigation of Beam Search Trees to Address Language Model Prompting Challenges

T. Spinner, R. Sevastjanova, R. Kehlbeck, T. Stähle, D. A. Keim, O. Deussen, A. Spitz, M. El-Assady
Teaser of Revealing the Unwritten: Visual Investigation of Beam Search Trees to Address Language Model Prompting Challenges

The BSTs for the prompt women like to with different quantifiers used in the place of the token. The user can select wordlists for exploration; the tree is collapsed showing only interesting nodes for the analysis.

Material

Paper (.pdf, 2.6MB)

Abstract

The present popularity of generative language models has amplified interest in interactive methods to guide model outputs. Prompt refinement is considered one of the most effective means to influence output among these meth ods. We identify several challenges associated with prompting large language models, categorized into data- and model-specific, linguistic, and socio-linguistic challenges. A comprehensive examination of model outputs, including runner-up candidates and their corresponding probabilities, is needed to address these issues. The beam search tree, the prevalent algorithm to sample model outputs, can inherently supply this information. Consequently, we leverage an interactive visual method for investigating the beam search tree, facilitating analysis of the decisions made by the model during generation. Our explorative approach validates existing results and offers additional insights.

BibTeX

@inproceedings{Spinner2025RevealingUnwrittenVisual,
  abstract  = {The present popularity of generative language models has amplified interest in interactive methods to guide model outputs. Prompt refinement is considered one of the most effective means to influence output among these methods. We identify several challenges associated with prompting large language models, categorized into data- and model-specific, linguistic, and socio-linguistic challenges. A comprehensive examination of model outputs, including runner-up candidates and their corresponding probabilities, is needed to address these issues. The beam search tree, the prevalent algorithm to sample model outputs, can inherently supply this information. Consequently, we leverage an interactive visual method for investigating the beam search tree, facilitating analysis of the decisions made by the model during generation. Our explorative approach validates existing results and offers additional insights.},
  address   = {Vienna, Austria},
  author    = {T. Spinner, R. Sevastjanova, R. Kehlbeck, T. Stähle, D. A. Keim, O. Deussen, A. Spitz, M. El-Assady},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)},
  doi       = {10.18653/v1/2025.acl-demo.29},
  editor    = {Mishra, Pushkar and Muresan, Smaranda and Yu, Tao},
  isbn      = {979-8-89176-253-4},
  month     = {July},
  pages     = {295--306},
  publisher = {Association for Computational Linguistics},
  title     = {Revealing the Unwritten: Visual Investigation of Beam Search Trees to Address Language Model Prompting Challenges},
  url       = {https://aclanthology.org/2025.acl-demo.29/},
  year      = {2025}
}