Why HDBSCAN*'s flat clustering changes with minPts: shrinking clusters, a one-run predictor and a certificate
Year: 2026
Type: preprint
Abstract: HDBSCAN* is a density-based clustering method that measures density at each point by its minPts nearest neighbours, builds a tree of clusters and extracts one flat partition, discarding clusters smaller than minPts. Because users act on that partition, we ask how often and why it changes when minPts, usually treated as a smoothing setting, rises by one step. On 47 benchmark datasets not used before, 16.9% of such steps changed the partition markedly (adjusted Rand index below 0.8), and in single-cell, text and astrometry pipelines most such changes altered the number of clusters. The change usually starts in the tree, where a selected cluster shrinks below the minimum size and is dropped, and synthetic clusters just above that size were lost far more often than clusters twice as large. From one run we count each cluster's points that stay dense enough at the next value of minPts. On further OpenML tables, 82% of clusters with fewer such points than the next minimum size were lost, far more than other clusters of the same size. We also prove a condition, checked without refitting, under which the partition stays unchanged outside an explicit set of points, and it was never violated. This revised version improves the presentation and corrects the description of several prior works; results, data and claims are unchanged. Version 1 is the manuscript as first submitted. Not peer reviewed.
Author: Coutinho, Felipe Provezano
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225465048