Establishing a Benchmark for Large Language Models in Business Process Management
Organisatsiooni nimi
Software Engineering and Information Systems
Kokkuvõte
Large Language Models (LLMs) opened up new ways to improve decision support systems in many areas, including Business Process Management (BPM). The Business Process Large Language Model (BPLLM) Framework has recently demonstrated promising results by combining Retrieval-Augmented Generation (RAG) with fine-tuning, enriching process-specific knowledge and facilitating conversations around BPM tasks. However, despite the advances, there remains a lack of standardized benchmarks that evaluate LLMs' effectiveness in BPM tasks through structured validation, grounded in user feedback and empirical performance analysis.
The goal of this thesis is to create a comprehensive benchmark for LLMs applied to BPM tasks. The benchmark will be validated using a combination of empirical experiments, expert opinions, and user-driven research variants, such as utilizing LLMs as judges or assistants. This will enable a better understanding of LLM frameworks and their integration into BPM, while offering a standardized method for future evaluations.
The goal of this thesis is to create a comprehensive benchmark for LLMs applied to BPM tasks. The benchmark will be validated using a combination of empirical experiments, expert opinions, and user-driven research variants, such as utilizing LLMs as judges or assistants. This will enable a better understanding of LLM frameworks and their integration into BPM, while offering a standardized method for future evaluations.
Lõputöö kaitsmise aasta
2024-2025
Juhendaja
Viktoriia Abakumova
Suhtlemiskeel(ed)
inglise keel
Nõuded kandideerijale
Tase
Magister
Märksõnad
Kandideerimise kontakt
Nimi
Viktoriia Abakumova
Tel
E-mail