Year: 2025
Type: article
Abstract: The acquisition of organized product data continues to be a crucial obstacle in the dynamic world of e-commerce. This problem is made worse by the growing complexity of contemporary websites, which include dynamic content and anti-scraping features. By addressing the shortcomings of current approaches, this paper offers a thorough methodology for creating a reliable web scraper designed especially for Indian e-commerce platforms. To efficiently handle static as well as dynamic material, the suggested approach incorporates Beautiful Soup and Selenium with Flask and React.js. Overcoming anti-scraping mechanisms, guaranteeing data accuracy through sophisticated preprocessing approaches, and offering actionable insights through data visualization are some of the research's main accomplishments. This study also includes scalability to manage big datasets across various e-commerce platforms, ethical scraping methods, and compliance with robots.txt instructions. The scraper's ability to extract, clean, and analyze data is confirmed by experimental findings, providing a scalable and morally sound option for automated e-commerce data extraction.
Authors: Mustafa Sultan -, Manavlal Nagdev -, Md Muaviya Ansari -, Maya Baniya -, Vineeta Rathore -
Institution: Medi-Caps University
Language: English
Topic: Web Data Mining and Analysis
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4410906168