Large Language Models are Easily Confused: A Quantitative Metric, Security Implications and Typological Analysis

Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

Abstract

Language Confusion is a phenomenon where Large Language Models (LLMs) generate text that is neither in the desired language, nor in a contextually appropriate language. This phenomenon presents a critical challenge in text generation by LLMs, often appearing as erratic and unpredictable behavior. We hypothesize that there are linguistic regularities to this inherent vulnerability in LLMs and shed light on patterns of language confusion across LLMs. We introduce a novel metric, Language Confusion Entropy, designed to directly measure and quantify this confusion, based on language distributions informed by linguistic typology and lexical variation. Comprehensive comparisons with the Language Confusion Benchmark (Marchisio et al., 2024) confirm the effectiveness of our metric, revealing patterns of language confusion across LLMs. We further link language confusion to LLM security, and find patterns in the case of multilingual embedding inversion attacks. Our analysis demonstrates that linguistic typology offers theoretically grounded interpretation, and valuable insights into leveraging language similarities as a prior for LLM alignment and security.
Original languageEnglish
Title of host publicationFindings of the Association for Computational Linguistics: NAACL 2025
Number of pages17
PublisherAssociation for Computational Linguistics
Publication date29 Apr 2025
DOIs
Publication statusAccepted/In press - Jan 2025
EventThe 2025 Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics - Albuquerque, United States
Duration: 29 Apr 20254 May 2025
https://2025.naacl.org/

Conference

ConferenceThe 2025 Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics
Country/TerritoryUnited States
City Albuquerque
Period29/04/202504/05/2025
Internet address

Keywords

  • Large Language Models
  • Cybersecurity
  • Linguistics

Fingerprint

Dive into the research topics of 'Large Language Models are Easily Confused: A Quantitative Metric, Security Implications and Typological Analysis'. Together they form a unique fingerprint.

Cite this