Transformer-based Approach for Named Entity Recognition from Business News Articles for Company Detection
Name
Vladyslav Umerenko
Abstract
The advancements in mass media technologies made an enormous amount of news articles available online that mention private and public companies. Some institutions can benefit from the information that the texts contain about companies. However, the number of news articles is enormous, which makes manual text processing impossible. Consequently, researchers have always been looking for effective and efficient methods of company data detection from news articles, particularly company names. Among the possible applications of a company information retrieval is to detect mentions of companies and associate them with known companies in a database, namely company detection. This would allow keeping track of the activity of companies that may be crucial for private and public companies that provide services of company activity analysis, systematic traders and other bodies that depend on the monitoring of company activity. One of the approaches for company name detection is Named Entity recognition (NER), which is a task that aims to locate and classify proper nouns into a defined set of categories. While the traditional approaches of NER involve the use of rule-based, word-based and sequence modelling models, such as Long-Short Term Memory (LSTM) and Conditional Random Fields (CRF), the recent developments in Deep Learning enable to employ Transformer-based models in a combination with the sequence modelling to produce a state-of-the-art performance of NER. The research starts with the evaluation of a simple approach to company detection. Then some word-based models are assessed to estimate the necessity of more sophisticated models. Next, the models based on Transformer, namely RoBERTa-base, in a combination with the transition-based algorithm for NER are presented. Finally, the named entities that are predicted on news articles are employed in the company detection task. The experiments have shown that the exploitation of Transformer-based models results in a significant improvement of both Named Entity recognition and company detection tasks (two-fold improvement). In addition, the presented approach successfully solves the task of open-world company name recognition that can be used to expand the existing database of companies.
Graduation Thesis language
English
Graduation Thesis type
Master - Software Engineering
Supervisor(s)
Rajesh Sharma
Defence year
2022