The Entropic Scree: An Information-Theoretic Diagnostic Framework for Intrinsic Rank and Informational Gravity in Tabular Systems
Year: 2026
Type: article
Abstract: Determining intrinsic dimensionality is foundational for manifold learning and predictive modeling. However, standard methods relying on linear variance or spatial distance metrics require strict assumptions that rarely hold in complex, high-dimensional environments, often fragmenting dependencies and driving spurious dimensional inflation. This paper introduces the Entropic Scree, an information-theoretic diagnostic framework evaluating intrinsic dimensionality via the eigendecomposition of Normalized Mutual Information. Using Information-Theoretic Jaccard Similarity (Variation of Information), it produces a geometrically coherent metric space unifying continuous, ordinal, and discrete variables, compressing fragmented dimensions back towards their Intrinsic Generative Rank. Crucially, the method maintains rigid structural boundaries under root entanglement, where baselines like Kernel PCA systematically degrade. Evaluating logarithmic spectral decay separates primary generative drivers from the Extended Signal Tail and unstructured idiosyncratic informational variance. Additionally, the framework quantifies the total signal-to-idiosyncratic volume and computes Average Informational Gravity (AIG) and Factor-Specific Informational Gravity (FSIG) metrics to translate abstract matrix properties into actionable, variable-equivalent footprints. Furthermore, bipolar eigenvector loadings and the Dimensional Inflation Index can be used to identify decoupled sub-networks and assess a dataset's linear sufficiency, respectively. Ultimately, the Entropic Scree provides a non-parametric, distribution-free diagnostic whose comparative advantages compound at scale and with system complexity.
Author: Lee-St. John, Terrence J.
Language: English
Subfield: Statistics and Probability
Field: Mathematics
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7203866588
FWCI:
Cites:
Cited by:
Related to: