LLM cross-model alignment measurement and improvement
Organization
TartuNLP
Abstract
It has been shown that independently trained LLMs (GPT, BERT, etc) learn input vector representations that have a certain structure, similar between the different models. The aim of this thesis is to measure this similarity (or "alignment") between pre-trained models, in-training models and possibly integrate alignment into the training process as an optimization objective.
Graduation Theses defence year
2023-2024
Supervisor
Mark Fishel
Spoken language (s)
Estonian, English
Requirements for candidates
Python, pytorch, transformers, vector space transformations and metrics
Level
Masters
Keywords
Application of contact
Name
Mark Fishel
Phone
E-mail