Work

An investigation on automatically assessing an application tutorial’s difficulty

Year: 2019
Type: dissertation
Abstract: Online step-by-step tutorials play an integral role in how users learn feature-rich software applications (e.g., Photoshop, AutoCAD, Fusion360). However, when searching for a tutorial, users can find it difficult to assess whether a given tutorial is designed for their level of software expertise. Novice users can struggle when a tutorial is out of their reach, whereas more advanced users can end up wasting time with overly simple, first-principles instruction. To assist users in selecting tutorials based on expertise, I investigate the feasibility of using machine learning techniques to automatically assess and label a tutorial’s difficulty level. Using Photoshop as a testbed, I develop a set of distinguishable tutorial features and use these features to train a classifier that can label a tutorial as either Beginner or Advanced with 85% accuracy. To illustrate a potential application of my classifier, I developed a tutorial selection interface called TutVis. TutVis annotates each tutorial with its difficulty level, along with visual representations of other tutorial features that contribute to this difficulty assessment. An initial evaluation comparing TutVis to two other interfaces (which varied in the number of different tutorial features displayed) showed a strong preference for and use of TutVis’s novel features.
Language: English
Open Access status: green
OpenAlex ID: W3029182087
Cites: 75
Cited by:
Related to: 10