Work

Metrics as Proxies: A Theoretical and Empirical Study of Reasonings Based on Bibliometric Indicators

Year: 2025

Type: article

Abstract: ABSTRACT Increasingly, reviewers are asked to engage with citation‐based indicators, yet we know little about how those numbers are actually translated into judgements. This article builds on an extensive literature review in which the author identifies “chains of reasoning” that connect metrics with factors such as quality, prestige, recognition, and career potential. The study analyses 174 negative habilitation review reports (2012–2021) from four disciplines in Poland—Art History, Sociology, Mathematics, and Materials Engineering—to map how reviewers reason with metrics in practice. We identify and code passages where quantitative indicators are transformed into qualitative claims and reconstruct the “chains of reasoning” that link numbers to evaluative conclusions. The prevalence of metric‐based reasoning varies sharply by field—from about 4% of reviews in Art History to 71% in Materials Engineering—while Mathematics and Sociology occupy intermediate positions. Across disciplines, metrics are interpreted primarily as signals of recognition or visibility and influence or uptake; direct inferences from metrics to intrinsic quality are comparatively rare. Reviewers sometimes inspect citation contexts (e.g., self‐citations) and time lags, but explicit reflexivity about the limitations of indicators is infrequent. Crucially, metrics never stand alone: even the most metric‐intensive reviews combine numbers with content‐based critique. We catalogue nine recurring reasoning types and show how a uniform policy that mandates attention to metrics yields heterogeneous, discipline‐specific evaluative practices. The article advances scholarship on responsible research assessment by explaining how metrics travel from numbers to judgements and by offering practical guidance: use indicators as contextual evidence, make interpretive assumptions explicit, and calibrate benchmarks to disciplinary evaluative cultures.

Source: Higher Education Quarterly

DOI: 10.1111/hequ.70095

Author: Łukasz Remisiewicz

Institution: University of Gdańsk

Language: English

Topic: scientometrics and bibliometrics research

Subfield: Statistics, Probability and Uncertainty

Field: Decision Sciences

Domain: Social Sciences

Open Access status: gold

Estimated APC paid: $2,900

Funder: Uniwersytet Gdański