Representing the Twittersphere: Archiving a representative sample of Twitter data under resource constraints
Year: 2019
Type: article
Abstract: The rising popularity of social media posts, most notably Twitter posts, as a data source for social science research poses significant problems with regard to access to representative, high-quality data for analysis. Cheap, publicly available data such as that obtained from Twitter's public application programming interfaces is often of low quality, while high-quality data is expensive both financially and computationally. Moreover, data is often available only in real-time, making post-hoc analysis difficult or impossible. We propose and test a methodology for inexpensively creating an archive of Twitter data through population sampling, yielding a database that is highly representative of the targeted user population (in this test case, the entire population of Japanese-language Twitter users). Comparing the tweet volume, keywords, and topics found in our sample data set with the ground truth of Twitter's full data feed confirmed a very high degree of representativeness in the sample. We conclude that this approach yields a data set that is suitable for a wide range of post-hoc analyses, while remaining cost effective and accessible to a wide range of researchers.
Authors: Airo Hino, Robert A. Fahey
Institution: Waseda University
Language: English
Subfield: General Social Sciences
Field: Social Sciences
Domain: Social Sciences
Open Access status: hybrid
Estimated APC paid: $1,300
Awards: Establishing time-series databases of party manifestos and estimating party positions(Fostering Joint International Research), Economic and psychological effects of network diversification, Building a Twitter Database to Answer Political Science Research Questions, Divided Democracies and Roles of Electoral System
OpenAlex ID: W2943369147