Automatic evaluation of language models for low-resource Finno-Ugric languages
Organization
TartuNLP
Abstract
In spring 2024, we gathered some human feedback for our new Finno-Ugric language models. Since collecting human feedback is both expensive and time-consuming, it would be good to replace it with feedback from stronger language models, such as OpenAI models. The so-called LLM-judges have been shown to correlate well with human evaluations in English and several other high-resource languages. The goal of this thesis is to investigate whether LLM-judges can also be used to automate the evaluation of language models for low-resource Finno-Ugric languages (specifically Estonian, Võro, Livonian, and Komi).
Graduation Theses defence year
2024-2025
Supervisor
Hele-Andra Kuulmets
Spoken language (s)
Estonian, English
Requirements for candidates
Level
Masters
Keywords
Application of contact
Name
Hele-Andra Kuulmets
Phone
E-mail