Hierarchical Transformers for NLP ad Reasoning Tasks
Organization
Geometric Deep Learning
Abstract
Transformer architectures give outstanding results on many NLP tasks, particularly sequential data. The advance of transformer models over LSTM is that they predict long-distance dependencies directly, rather than passing long-distance dependencies down a long recurrence chain. The Transformer models have broken long-distance dependencies from the recurrence mechanism using a direct matrix product. The multi-layers are building hierarchies over what they are "attending" to. We propose new transformer architecture that has an exploits hierarchy implicitly and explicitly. Our application areas of large-scale machine learning models are
Graduation Theses defence year
2022-2023
Supervisor
Kallol Roy
Spoken language (s)
English
Requirements for candidates
Level
Bachelor, Masters
Keywords
Application of contact
Name
KALLOL ROY
Phone
+37256051480
E-mail
See more