Scalable Named Entity Recognition in social media using Bi-MEMM in a Distributed Environment
Year: 2025
Type: article
Abstract: Data mining provides a wealth of actionable intelligence for enhancing internet-based, query-based AI. This study focuses on the importance of Named Entity Recognition (NER) in extracting valuable information from social media's dynamic and extensive realm. This research paper introduces a novel method for performing Named Entity Recognition in a distributed setting, specifically designed to address the unique difficulties presented by social media data. This research investigates the effectiveness of combining Bidirectional Long Short-Term Memory (Bi-LSTM) and Maximum Entropy Markov Model (MEMM) as Bi-MEMM for improving Named Entity Recognition (NER) accuracy. This research presents a model that uses Bi-LSTM to effectively capture the bidirectional context in social media text. By leveraging this approach, the model can accurately identify complex named entities within the text. This study utilises the Maximum Entropy Markov Model (MEMM) to effectively capture and model the dependencies between labels, thereby enhancing the accuracy and precision of entity recognition. This study focuses on the significance of a distributed environment in the context of social media, where data is generated rapidly. This research presents a system optimising performance by leveraging distributed computing resources for parallel processing. This study examines the performance evaluations of a model in identifying named entities in user-generated content across diverse datasets. The findings demonstrate the model’s effectiveness in this task with an accuracy of 99.3%. This research focuses on developing a system that operates in a distributed environment to ensure precision and efficiency. The plan addresses the specific requirements of social media platforms, where recognising named entities plays a crucial role in understanding and analysing user-generated content
Authors: K. Syed Kousar Niasi, K. Prakash, M. Krishna Kumar, P. Murugesan
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W4410369126