Combining Feature Embedding Based on the BERT Architecture and Random Forest Algorithm to Identify the Handling Section and Risk Priority Level in an ISP Company in Indonesia
Year: 2026
Type: article
Abstract: Presently, Internet usage in society has grown considerably. It presents significant opportunities for companies to develop business related to Internet Service Providers (ISPs). In Indonesia, both domestic and foreign ISPs are competing. To survive and remain popular with Indonesian customers, ISPs need to improve their products and services quickly. Hence, companies need to collect user feedback through customer loyalty applications and reviews to improve their service or technology. However, manually auditing thousands of textual reviews is labor-intensive and subject to human error, making it highly inefficient for agile, real-time corporate decision-making. The research aims to address this operational bottleneck by improving the automation process and providing decision-making information to ISP management. More specifically, the research contributes to implementing and comparing two transfer learning models: Bidirectional Encoder Representations from Transformers (BERT) and IndoBERT for tokenization and embedding. The embedding results are then fed into a random forest for classification. The dataset is collected via a scraping process from the Play Store application, focusing on user feedback and ratings. A total of 1,192 records of user feedback reviews from ISP loyalty apps on the Play Store in March 2024 were gathered. These are manually labeled by a general manager from the company into binary and multi-class categories. Splitting the dataset into 70% for training and 30% for testing achieves a classification accuracy of 70% for the problem domain with IndoBERT + Random Forest. However, multi-class classification for categorizing user feedback into a priority score model achieves an accuracy of 58%.
Authors: Rhemzy Putra Maulana, Yulianto Yulianto
Institution: Binus University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $131
OpenAlex ID: W7172015229