arvutiteaduse instituudi lõputööde teemade register


Hierarchical Transformers for NLP ad Reasoning Tasks
Organisatsiooni nimiGeometric Deep Learning
KokkuvõteTransformer architectures give outstanding results on many NLP tasks, particularly sequential data. The advance of transformer models over LSTM is that they predict long-distance dependencies directly, rather than passing long-distance dependencies down a long recurrence chain. The Transformer models have broken long-distance dependencies from the recurrence mechanism using a direct matrix product. The multi-layers are building hierarchies over what they are "attending" to. We propose new transformer architecture that has an exploits hierarchy implicitly and explicitly. Our application areas of large-scale machine learning models are
Lõputöö kaitsmise aasta2022-2023
JuhendajaKallol Roy
Suhtlemiskeel(ed)inglise keel
Nõuded kandideerijale
Tase Bakalaureus, Magister
Märksõnad #Transformers, Group Theory
Kandideerimise kontakt
Nimi KALLOL ROY
Tel +37256051480
E-mail kallol.roy@ut.ee
Vaata lähemalt https://kallolroy.me/


ati.study@lists.ut.ee