Generative Dependency Language Modeling Using Recurrent Neural Networks
Name
Liisa Rätsep
Abstract
This thesis proposes an approach to incorporating syntactical data to the task of generative language modeling. We modify the logic of a transition-based dependency parser to generate new words to the buffer using the top items in the stack as input. We hypothesize that the approach provides benefits in modeling long-term dependencies. We implement our system along with a baseline language model and observe that our approach provides an improvement in perplexity scores and that this improvement is more significant in modeling sentences that contain longer dependencies. Additionally, the qualitative analysis of the generated sentences demonstrates that our model is able to generate more cohesive sentences.
Graduation Thesis language
English
Graduation Thesis type
Master - Computer Science
Supervisor(s)
Kairit Sirts
Defence year
2019