Assessing the Consequences of Text Preprocessing Decisions
Year: 2016
Type: preprint
Abstract: We consider the impact of preprocessing decisions, a crucial first step in all text-as-data investigations. We note that grounded practical help on feature selection for the mostly unsupervised context in which social scientists work is scant: instead, advice is lifted from the supervised literature with little thought as to whether it is optimal, or even appropriate, for the tasks at hand. Worryingly, as we show with real data, substantive inferences and model performance for unsupervised approaches are not generally robust to perturbations of common preprocessing steps. We introduce a statistical procedure and easy-to-use software — preText — allowing scholars to examine the sensitivity of their findings under alternate preprocessing regimes. For a range of datasets, we show that while some steps are mostly harmless, researchers should be cautious about others, since they transform the data in ways likely to lead the unwary down different forking paths of inference.
Source: SSRN Electronic Journal
Authors: Matthew J. Denny, Arthur Spirling
Language: English
Subfield: General Social Sciences
Field: Social Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W2531372442