Year: 2004
Type: dissertation
Abstract: Continuous testing is a new feature for software development environments that uses excess cycles on a developer's workstation to continuously run regression tests in the background, providing rapid feedback about test failures as source code is edited. It is intended to reduce the time and energy required to keep code well-tested, and to prevent regression errors from persisting uncaught for long periods of time. The longer that regression errors are allowed to linger during development, the more time is wasted debugging and fixing them once they are discovered. By monitoring and measuring software projects, we estimate that the wasted time, consisting of this preventable extra fixing cost added to the time spent running tests and waiting for them to complete, accounts for 10-15% of total development time. We present a model of developer behavior that uses data from past projects to infer developer beliefs and predict behavior in new environments -in particular, when changing testing methodologies or tools to reduce wasted time. This model predicts that continuous testing would reduce wasted time by 92-98%, a substantial improvement over other approaches we evaluated, such as automatic test prioritization and changing manual test frequencies. A controlled human experiment indicates that student developers using continuous testing were three times more likely to complete a task before the deadline than those without, with no significant effect on time worked.
Author: David Saff
Language: English
Subfield: Software
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Funder: National Science Foundation
OpenAlex ID: W1491586549