A Call for Consistency in Reporting Typological Diversity

Wessel Poelman, Esther Ploeger, Miryam de Lhoneux, Johannes Bjerva

Publikation: Bidrag til bog/antologi/rapport/konference proceedingKonferenceartikel i proceedingForskningpeer review


In order to draw generalizable conclusions about the performance of multilingual models across languages, it is important to evaluate on a set of languages that captures linguistic diversity.Linguistic typology is increasingly used to justify language selection, inspired by language sampling in linguistics. However, justifications for ‘typological diversity’ exhibit great variation, as there seems to be no set definition, methodology or consistent link to linguistic typology.In this work, we provide a systematic insight into how previous work in the ACL Anthology uses the term ‘typological diversity’.Our two main findings are: 1) what is meant by typologically diverse language selection is not consistent and 2) the actual typological diversity of the language sets in these papers varies greatly.We argue that, when making claims about ‘typological diversity’, an operationalization of this should be included.A systematic approach that quantifies this claim, also with respect to the number of languages used, would be even better.
TitelProceedings of the 6th Workshop on Research in Computational Linguistic Typology and Multilingual NLP
ForlagAssociation for Computational Linguistics
Publikationsdato17 mar. 2024
ISBN (Trykt)979-8-89176-071-4
StatusUdgivet - 17 mar. 2024
BegivenhedThe 18th Conference of the European Chapter of the Association for Computational Linguistics - Radisson Blu, St. Julian's, Malta
Varighed: 17 mar. 202422 mar. 2024


KonferenceThe 18th Conference of the European Chapter of the Association for Computational Linguistics
LokationRadisson Blu
BySt. Julian's


Dyk ned i forskningsemnerne om 'A Call for Consistency in Reporting Typological Diversity'. Sammen danner de et unikt fingeraftryk.