Wage Rates and Job Requirements Prediction: An Application to Logistics Online Job Postings Using Search Tools and Web Scraping
Year: 2025
Type: conference-paper
Abstract: This paper predicts offered wage rates and job requirements in the logistics industry by utilizing data from online job postings collected through two methods: search tools and web scraping. We apply conventional estimation techniques, such as ordinary least squares and kernel density estimation, to analyze the collected data. Additionally, for the first time, we employ nowcasting methods (linear regression, decision tree, and K-nearest neighbor methods) in this context to generate robust results. Our main findings are as follows: First, the average real wage derived from online job postings aligns with officially published GDP per capita data for the studied countries and regions. Second, we identify significantly positive causal effects of work experience on real wages in the logistics industry. Third, skill requirements exhibit year-over-year variations. Finally, the decision tree method generates the closest nowcasted results in line with the actual web scraped data. The proposed methodologies and their findings establish a reliable approach using search tools and web scraping to define and predict labor demand for stakeholders in this sector as well as others.
Authors: Khoa Huu Dang Tran, Huong Quynh Nguyen, Hang My Hanh Le, Lina Doan Tran, Nhi To Yen Tran
Institution: Foreign Trade University
Language: English
Subfield: Economics and Econometrics
Domain: Social Sciences
Open Access status: gold
OpenAlex ID: W4411371554