Search results

  • 2025

    All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languages

    Vayani, A., Dissanayake, D., Watawana, H., Ahsan, N., Sasikumar, N., Thawakar, O., Ademtew, H. B., Hmaiti, Y., Kumar, A., Kuckreja, K., Maslych, M., Ghallabi, W. A., Qin, C., Shaker, A. M., Zhang, M., Ihsani, M. K., Esplana, A., Gokani, M., Mirkin, S. & Singh, H. & 47 others, Srivastava, A., Hamerlik, E., Izzati, F. A., Maani, F. A., Cavada, S., Chim, J., Gupta, R., Manjunath, S., Zhumakhanova, K., Rabevohitra, F. H., Amirudin, A., Ridzuan, M., Kareem, D., More, K., Li, K., Shakya, P., Saad, M., Ghasemaghaei, A., Djanibekov, A., Azizov, D., Jankovic, B., Bhatia, N., Obando-Ceron, J., Otieno, O., Farestam, F., Rabbani, M., Baliah, S., Sanjeev, S., Shtanchaev, A., Fatima, M., Nguyen, T., Kareem, A., Aremu, T., Xavier, N., Bhatkal, A., Toyin, H., Chadha, A., Cholakkal, H., Anwer, R. M., Felsberg, M., Laaksonen, J., Solorio, T., Choudhury, M., Laptev, I., Shah, M., Khan, S. & Khan, F., 10 Jun 2025, The IEEE/CVF Conference on Computer Vision and Pattern Recognition 2025. IEEE (Institute of Electrical and Electronics Engineers)

    Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

    File
    99 Downloads (Pure)
  • DaKultur: Evaluating the Cultural Awareness of Language Models for Danish with Native Speakers

    Müller-Eberstein, M., Zhang, M., Bassignana, E., Brunsgaard Trolle, P. & van der Goot, R., 29 Apr 2025, 3rd Workshop on Cross-Cultural Considerations in NLP (C3NLP). Association for Computational Linguistics, ACL Anthology

    Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

    Open Access
    File
    7 Downloads (Pure)
  • HiFi-KPI: A Dataset for Hierarchical KPI Extraction from Earnings Filings

    Aavang, R. T., Rizzi, G., Bøggild, R., Iolov, A., Zhang, M. & Bjerva, J., Feb 2025, (Submitted).

    Research output: Working paper/PreprintPreprint

  • How Do Hackathons Foster Creativity? Towards AI Collaborative Evaluation of Creativity at Scale

    Falk, J., Chen, Y., Rafner, J., Zhang, M., Bjerva, J. & Nolte, A., Apr 2025, CHI '25: Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery (ACM), 34 p.

    Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

    Open Access
  • Humanity's Last Exam

    Center for AI Safety, 24 Jan 2025, (In preparation).

    Research output: Working paper/PreprintPreprint

    File
    76 Downloads (Pure)
  • INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledge

    Romanou, A., Foroutan, N., Sotnikova, A., Chen, Z., Nelaturu, S. H., Singh, S., Maheshwary, R., Altomare, M., Haggag, M. A., Snegha, A., Amayuelas, A., Amirudin, A. H., Aryabumi, V., Boiko, D., Chang, M., Chim, J., Cohen, G., Dalmia, A. K., Diress, A. & Duwal, S. & 39 others, Dzenhaliou, D., Florez, D. F. E., Farestam, F., Imperial, J. M., Islam, S. B., Isotalo, P., Jabbarishiviari, M., Karlsson, B. F., Khalilov, E., Klamm, C., Koto, F., Krzemiński, D., de Melo, G. A., Montariol, S., Nan, Y., Niklaus, J., Novikova, J., Ceron, J. S. O., Paul, D., Ploeger, E., Purbey, J., Rajwal, S., Ravi, S. S., Rydell, S., Santhosh, R., Sharma, D., Skenduli, M. P., Moakhar, A. S., Moakhar, B. S., Tamir, R., Tarun, A. K., Wasi, A. T., Weerasinghe, T. O., Yilmaz, S., Zhang, M., Schlag, I., Fadaee, M., Hooker, S. & Bosselut, A., 24 Apr 2025, The Thirteenth International Conference on Learning Representations. Singapore: International Conference on Learning Representations, Vol. The Thirteenth International Conference on Learning Representations.

    Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

    File
    35 Downloads (Pure)
  • Kaleidoscope: In-language Exams for Massively Multilingual Vision Evaluation

    Salazar, I., Burda, M. F., Islam, S. B., Moakhar, A. S., Singh, S., Farestam, F., Romanou, A., Boiko, D., Khullar, D., Zhang, M., Krzemiński, D., Novikova, J., Shimabucoro, L., Imperial, J. M., Maheshwary, R., Duwal, S., Amayuelas, A., Rajwal, S., Purbey, J. & Ruby, A. & 24 others, Popovič, N., Suppa, M., Wasi, A. T., Kadiyala, R. M. R., Tsymboi, O., Kostritsya, M., Moakhar, B. S., Merlin, G. D. C., Coletti, O. F., Shiviari, M. J., fard, M. F., Fernandez, S., Grandury, M., Abulkhanov, D., Sharma, D., Mitri, A. G. D., Marchezi, L. B., Obando-Ceron, J., Kohut, N., Ermis, B., Elliott, D., Ferrante, E., Hooker, S. & Fadaee, M., 9 Apr 2025, (In preparation).

    Research output: Working paper/PreprintPreprint

    File
    32 Downloads (Pure)
  • MorSeD: Morphological Segmentation of Danish and its Effect on Language Modeling

    van der Goot, R., Jensen, A., Allerslev Schledermann, E., Wildner Kildeberg, M., Larsen, N., Zhang, M. & Bassignana, E., Mar 2025, Proceedings of the 25th Nordic Conference on Computational Lingustics and 11th Baltic Conference on Human Language Technologies : NoDaLiDa/Baltic-HLT 2025. Northern European Association for Language Technology

    Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

  • On-Device LLMs for Home Assistant: Dual Role in Intent Detection and Response Generation

    Birkmose, R., Mørkeberg Reece, N., Hofsted Norvin, E., Bjerva, J. & Zhang, M., 29 Apr 2025, Proceedings of the Tenth Workshop on Noisy and User-generated Text (W-NUT 2025). Association for Computational Linguistics

    Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

    Open Access
  • Sailor2: Sailing in South-East Asia with Inclusive Multilingual LLMs

    Dou, L., Liu, Q., Chen, C., Wang, Z., Jin, Z., Liu, Z., Zhu, T., Du, C., Yang, P., Wang, H., Liu, J., Zhao, Y., Feng, X., Mao, X., Yeung, M. T., Pipatanakul, K., Koto, F., Thu, M. S., Kydlíček, H. & Liu, Z. & 20 others, Lin, Q., Sripaisarnmongkol, S., Sae-Khow, K., Thongchim, N., Konkaew, T., Borijindargoon, N., Dao, A., Maneegard, M., Artkaew, P., Yong, Z.-X., Nguyen, Q., Phatthiyaphaibun, W., Tran, H. H., Zhang, M., Chen, S., Pang, T., Du, C., Wan, X., Lu, W. & Lin, M., 18 Feb 2025, (In preparation).

    Research output: Working paper/PreprintPreprint

    File
    44 Downloads (Pure)
  • File
    32 Downloads (Pure)
  • Scaling Reasoning can Improve Factuality in Large Language Models

    Zhang, M., Bjerva, J. & Biswas, R., May 2025, (Submitted).

    Research output: Working paper/PreprintPreprint

  • SEFL: Harnessing Large Language Model Agents to Improve Educational Feedback Systems

    Zhang, M., Dilling, A. P., Gondelman, L., Lyngdorf, N. E., Lindsay, E. & Bjerva, J., Feb 2025, (Submitted).

    Research output: Working paper/PreprintPreprint

  • SHADES: Towards a Multilingual Assessment of Stereotypes in Large Language Models

    BigScience, Ploeger, E. & Zhang, M., 1 May 2025, Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, p. 11995-12041

    Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

    Open Access
  • SnakModel: Lessons Learned from Training an Open Danish Large Language Model

    Zhang, M., Müller-Eberstein, M., Bassignana, E. & van der Goot, R., Mar 2025, Proceedings of the 25th Nordic Conference on Computational Lingustics and 11th Baltic Conference on Human Language Technologies : NoDaLiDa/Baltic-HLT 2025. Northern European Association for Language Technology

    Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

  • The Responsible Development of Automated Student Feedback with Generative AI

    Lindsay, E., Zhang, M., Johri, A. & Bjerva, J., 22 Apr 2025, Proceedings of the IEEE Global Engineering Education Conference (EDUCON 2025): Special Session: Generative AI and Ethical Integration in Higher Education: Navigating Innovation and Responsibility. IEEE Press, 9 p.

    Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

  • 2024

    Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning

    Singh, S., Vargus, F., D'souza, D., Karlsson, B. F., Mahendiran, A., Ko, W.-Y., Shandilya, H., Patel, J., Mataciunas, D., O'Mahony, L., Zhang, M., Hettiarachchi, R., Wilson, J., Machado, M., Souza Moura, L., Krzemiński, D., Fadaei, H., Ergün, I., Okoh, I. & Alaagib, A. & 13 others, Mudannayake, O., Alyafeai, Z., Chien, V. M., Ruder, S., Guthikonda, S., Alghamdi, E. A., Gehrmann, S., Muennighoff, N., Bartolo, M., Kreutzer, J., Üstün, A., Fadaee, M. & Hooker, S., 2024, The 62nd Annual Meeting of the Association for Computational Linguistics . Ku, L.-W., Martins, A. F. T. & Srikumar, V. (eds.). Association for Computational Linguistics, p. 11521-11567 47 p.

    Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

    Open Access
    File
    11 Citations (Scopus)
    27 Downloads (Pure)
  • Can Humans Identify Domains?

    Barrett, M., Müller-Eberstein, M., Bassignana, E., Pauli, A. B., Zhang, M. & van der Goot, R., May 2024, The 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation. European Language Resources Association, p. 2745–2765 21 p. (Proceedings of International Conference on Computational Linguistics (COLING)).

    Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

    Open Access
    File
    1 Citation (Scopus)
    20 Downloads (Pure)
  • Deep Learning-based Computational Job Market Analysis: A Survey on Skill Extraction and Classification from Job Postings

    Senger, E., Zhang, M., van der Goot, R. & Plank, B., 2024, NLP4HR 2024 - 1st Workshop on Natural Language Processing for Human Resources, Proceedings of the Workshop. Hruschka, E., Lake, T., Otani, N. & Mitchell, T. (eds.). Association for Computational Linguistics, ACL Anthology, p. 1-15 15 p. (NLP4HR 2024 - 1st Workshop on Natural Language Processing for Human Resources, Proceedings of the Workshop).

    Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

    6 Citations (Scopus)
  • Entity Linking in the Job Market Domain

    Zhang, M., van der Goot, R. & Plank, B., 2024, EACL 2024 - 18th Conference of the European Chapter of the Association for Computational Linguistics, Findings of EACL 2024. Graham, Y., Purver, M. & Purver, M. (eds.). Association for Computational Linguistics, ACL Anthology, p. 410-419 10 p. (EACL 2024 - 18th Conference of the European Chapter of the Association for Computational Linguistics, Findings of EACL 2024).

    Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

    2 Citations (Scopus)
  • JOBSKAPE: A Framework for Generating Synthetic Job Postings to Enhance Skill Matching

    Magron, A., Dai, A., Zhang, M., Montariol, S. & Bosselut, A., 2024, NLP4HR 2024 - 1st Workshop on Natural Language Processing for Human Resources, Proceedings of the Workshop. Hruschka, E., Lake, T., Otani, N. & Mitchell, T. (eds.). Association for Computational Linguistics, ACL Anthology, p. 43-58 16 p. (NLP4HR 2024 - 1st Workshop on Natural Language Processing for Human Resources, Proceedings of the Workshop).

    Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

    5 Citations (Scopus)
  • Leveraging Large Language Models for Actionable Course Evaluation Student Feedback to Lecturers

    Zhang, M., Lindsay, E., Thorbensen, F. B., Poulsen, D. B. & Bjerva, J., Sept 2024, Proceedings of the 52nd Annual Conference of the European Society for Engineering Education (SEFI). Société européenne pour la formation des ingénieurs (SEFI), p. 1089-1098

    Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

    Open Access
    File
    15 Downloads (Pure)
  • NNOSE: Nearest Neighbor Occupational Skill Extraction

    Zhang, M., van der Goot, R., Kan, M. Y. & Plank, B., 2024, EACL 2024 - 18th Conference of the European Chapter of the Association for Computational Linguistics, Proceedings of the Conference. Graham, Y., Purver, M. & Purver, M. (eds.). Association for Computational Linguistics, ACL Anthology, p. 589-608 20 p. (EACL 2024 - 18th Conference of the European Chapter of the Association for Computational Linguistics, Proceedings of the Conference, Vol. 1).

    Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

    2 Citations (Scopus)
  • Rethinking Skill Extraction in the Job Market Domain using Large Language Models

    Nguyen, K. C., Zhang, M., Montariol, S. & Bosselut, A., 2024, NLP4HR 2024 - 1st Workshop on Natural Language Processing for Human Resources, Proceedings of the Workshop. Hruschka, E., Lake, T., Otani, N. & Mitchell, T. (eds.). Association for Computational Linguistics, ACL Anthology, p. 27-42 16 p. (NLP4HR 2024 - 1st Workshop on Natural Language Processing for Human Resources, Proceedings of the Workshop).

    Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

    4 Citations (Scopus)
  • 2023

    ESCOXLM-R: Multilingual Taxonomy-driven Pre-training for the Job Market Domain

    Zhang, M., van der Goot, R. & Plank, B., Jul 2023, The 61st Annual Meeting of the Association for Computational Linguistics. Toronto, Canada: Association for Computational Linguistics, Vol. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). p. 11871–11890 20 p.

    Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

    Open Access
    11 Citations (Scopus)
  • 2022

    Evidence > Intuition: Transferability Estimation for Encoder Selection

    Bassignana, E., Müller-Eberstein, M., Zhang, M. & Plank, B., 7 Dec 2022, The 2022 Conference on Empirical Methods in Natural Language Processing: EMNLP 2022. Abu Dhabi, United Arab Emirates: Association for Computational Linguistics, p. 4218–4227

    Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

    Open Access
    6 Citations (Scopus)
  • Experimental Standards for Deep Learning in Natural Language Processing Research

    Ulmer, D., Bassignana, E., Müller-Eberstein, M., Varab, D., Zhang, M., van der Goot, R., Hardmeier, C. & Plank, B., 2022, Findings of 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, p. 2673-2692 20 p.

    Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

    8 Citations (Scopus)
  • Experimental Standards for Deep Learning Research: A Natural Language Processing Perspective

    Ulmer, D. T., Bassignana, E., Müller-Eberstein, M., Varab, D., Zhang, M., Hardmeier, C. & Plank, B., 29 Apr 2022.

    Research output: Contribution to conference without publisher/journalPaper without publisher/journalResearchpeer-review

    Open Access
    File
    150 Downloads (Pure)
  • Kompetencer: Fine-grained Skill Classification in Danish Job Postings via Distant Supervision and Transfer Learning

    Zhang, M., Jensen, K. N. & Plank, B., 16 Jun 2022, 13th International Conference on Language Resources and Evaluation. European Language Resources Association, p. 436-447 11 p.

    Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

    Open Access
    13 Citations (Scopus)
  • Skill Extraction from Job Postings using Weak Supervision

    Zhang, M., Jensen, K. N., van der Goot, R. & Plank, B., 19 Sept 2022, RecSys in HR'22: The 2nd Workshop on Recommender Systems for Human Resources, in conjunction with the 16th ACM Conference on Recommender Systems, September 18--23, 2022, Seattle, USA.. CEUR Workshop Proceedings

    Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

    Open Access
    File
    80 Downloads (Pure)
  • SKILLSPAN: Hard and Soft Skill Extraction from English Job Postings

    Zhang, M., Jensen, K. N., Sonniks, S. D. & Plank, B., 2022, NAACL 2022 - 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference. Association for Computational Linguistics, ACL Anthology, p. 4962-4984 23 p. (NAACL 2022 - 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference).

    Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

    31 Citations (Scopus)
  • 2021

    Cartography Active Learning

    Zhang, M. & Plank, B., 8 Nov 2021, Findings of the Association for Computational Linguistics: EMNLP 2021. Association for Computational Linguistics, p. 395–406

    Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

    Open Access
    File
    27 Citations (Scopus)
    50 Downloads (Pure)
  • De-identification of Privacy-related Entities in Job Postings

    Jensen, K. N., Zhang, M. & Plank, B., 21 May 2021, Proceedings of the 23rd Nordic Conference on Computational Linguistics. Association for Computational Linguistics, p. 210-221 (Linköping Electronic Conference Proceedings; No. 21, Vol. 178).

    Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

    Open Access
    File
    121 Downloads (Pure)