Understanding Natural Keyboard Typing Using Convolutional Neural Networks on Mobile Sensor Data
Year: 2018
Type: article
Abstract: Mobile phones and other devices with embedded sensors are becoming increasingly ubiquitous. Audio and motion sensor data may be able to detect information that we did not think possible. Some researchers have created models that can predict computer keyboard typing from a nearby mobile device; however, certain limitations to their experiment setup and methods compelled us to be skeptical of the models’ realistic prediction capability. We investigate the possibility of understanding natural keyboard typing from mobile phones by performing a well-designed data collection experiment that encourages natural typing and interactions. This data collection helps capture realistic vulnerabilities of the security of typed data. This thesis presents an implementation and analysis of a data collection experiment from twenty participants that systematically controls for keyboard type, ambient audio noise, and table position while collecting sensor data from eight mobile phones. We found these variables to be the most important to control because they may greatly affect result capabilities. Additionally, we allow participants to type and interact normally, so we can generalize our model to realistic scenarios. We use multimodal convolutional neural networks to show that mobile phones have some capability at predicting natural keyboard typing in various evaluation scenarios.
Author: Travis Siems
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W2807062126