Optimizing Data Analytics Workflows Through User-Driven Experimentation: Progress and Updates
Year: 2025
Type: conference-paper
Abstract: This paper provides a progress update to the doctoral symposium paper accepted at CAIN 2024 [1], which introduced the Experimentation Engine for optimizing data analytics workflows through user-driven experimentation. Over the past year, the framework has been significantly refined to better address diverse user needs by adapting the experimentation process for different user types, including domain experts and data scientists. The updated Experimentation Engine emphasizes continuous user involvement throughout the experimentation process, facilitating seamless interaction and iterative refinement. A demonstration video of the initial prototype, showcasing its capabilities, is available at https://doi.org/10.5281/zenodo.14288226. These enhancements aim to further streamline the experimentation process, providing accurate, meaningful, and trustworthy results while dynamically adapting to varying user requirements and system constraints. The key updates and lessons learned during this process are discussed in this paper.
Author: Keerthiga Rajenthiram
Institution: Vrije Universiteit Amsterdam
Language: English
Subfield: Information Systems and Management
Field: Decision Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W4411337765