Work

Smarter Crowdsourcing With NLP and Attention Mechanisms for Task Complexity Prediction

Year: 2025

Type: article

Abstract: Competitive Crowdsourcing Software Development (CCSD) has emerged as a powerful tool for developing software solutions, attracting researchers and the development market. Using crowdsourced collective intelligence, CCSD ensures the delivery of innovative, cost-effective, and high-quality solutions within specified time frames, making it an attractive approach for addressing complex challenges in software development. However, as the CCSD environment gains popularity, it also introduces challenges, particularly in predicting job complexity, which must be tackled to optimize the crowdsourcing process. In software platforms, client organizations register and upload jobs related to development projects. These jobs are manually reviewed by the copilots responsible for assessing the complexity of the job, a process that can lead to delays and an overburden of experts. To streamline this process, we propose an approach that automatically predicts job complexity and classifies it accordingly. We collect data from the TopCoder CCSD platform, focusing on projects related to software development. The collected data are pre-processed and tokenized using NLP techniques. We convert the text data into a word embedding vector matrix using a pre-trained GloVe model, which captures the semantic and contextual meaning of the text. The Sequence Attention (SA) mechanism is introduced in LSTM to identify the key parts of the input sequence that are most relevant for predicting the output, thereby improving job complexity classification. The proposed approach is then trained on these word embeddings using SA-LSTM and benchmarked against LSTM and other state-of-the-art techniques. The proposed approach, GloVe-based (GB) SA-LSTM, outperforms other approaches by achieving an accuracy improvement of 31.4%, 152.04%, 20.07%, 10.99%, 9.16%, and 17.64% over ZeroR, RP, LR, BERT, SVM, and DT. To verify the impact of SA, the proposed approach is compared with GB-LSTM. The GB-SA-LSTM outperforms GB-LSTM by 4.88%. This enhanced accuracy in task complexity prediction within software crowdsourcing platforms can significantly contribute to the efficient and effective development of crowdsourced projects.

Source: IEEE Access

DOI: 10.1109/access.2025.3615745

Authors: Yasir Munir, Qasim Umer, Muhammad Waheed Aslam, Muhammad Faheem, Hanadi Hakami

Institutions: Superior University, King Fahd University of Petroleum and Minerals, University of Vaasa, VTT Technical Research Centre of Finland, University of Business and Technology

Language: English

Topic: Software Engineering Research

Subfield: Information Systems

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

Estimated APC paid: $2,075

Funder: Academy of Finland