Work

Longitudinal evaluation methods in human-computer studies and visual analytics

Year: 2007
Type: conference-paper
Abstract: — In Human-Computer studies and visual analytics, the majority of the evaluation methods applied, collect data at a single point in time, in form of cross-sectional data. In several studies [e.g. 8] numerous visualization tools were evaluated in controlled experiments. Although the experiments discovered valuable findings, certain drawbacks of the research method were expressed. The time constraints of one-time experiments reduce the amount of training which can be given to the participants. Furthermore, when the studies tried to measure the insight derived from the visualization tools the time constraints didn’t allow observing how these insights develop over time or their interdependency. Further problems of cross-sectional studies are well known, like the selection of appropriate tasks, the mostly extrinsic motivation of the participants, the influence of a laboratory environment compared to a realistic work setting and whether a visualization tool does meet the work requirements in the long run. In this position paper we argue for applying longitudinal research methods in human-computer studies as an extension to cross-sectional studies and present a first approach towards a methodological research framework. We suggest a set of research questions and performance measures that would be benefical for extending cross-sectional studies with longitudinal ones. We also describe in two case studies, in which only cross-sectional research methods were used, how they can improved by longitudinal methods. Index Terms—longitudinal, long-term, methodology, research framework, experiment, visual analytics, evaluation 1
Language: English
Open Access status: green
OpenAlex ID: W1585988886
FWCI: 0.472
Cites: 9
Cited by: 13
Related to: 10