Year: 2024
Type: article
Abstract: Higher-order relationships are a central concept in the science of complex systems. A popular method of attempting to estimate the higher-order relationships of synergy and redundancy from data is through the O-information. It is an information-theoretic measure composed of Shannon entropy terms that quantifies the balance between redundancy and synergy in a system. However, bias is not yet taken into account in the estimation of the O-information of discrete variables. In this paper, we explain where this bias comes from and explore it for fully synergistic, fully redundant, and fully independent simulated systems of n=3 variables. Specifically, we explore how the sample size and number of bins affect the bias in the O-information estimation. The main finding is that the O-information of independent systems is severely biased towards synergy if the sample size is smaller than the number of jointly possible observations. This could mean that triplets identified as highly synergistic may in fact be close to independent. A bias approximation based on the Miller-Maddow method is derived for the O-information. We find that for systems of n=3 variables the bias approximation can partially correct for the bias. However, simulations of fully independent systems are still required as null models to provide a benchmark of the bias of the O-information.
Source: Entropy
DOI: 10.3390/e26100837
Language: English
Topic: Complex Systems and Dynamics
Subfield: Statistical and Nonlinear Physics
Field: Physics and Astronomy
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $2,955
Funders: Foundation for Cardiovascular Research, Yrjö Jahnssonin Säätiö, European Commission, ZonMw, Kela, Academy of Finland, Suomen Kulttuurirahasto, Jane ja Aatos Erkon Säätiö, Juho Vainion Säätiö, Signe ja Ane Gyllenbergin Säätiö, Emil Aaltosen Säätiö, Sydäntutkimussäätiö, Tampereen Tuberkuloosisäätiö, Diabetesliitto, Paavo Nurmen Säätiö, Tays, Syöpäsäätiö, Suomen kliinisen kemian yhdistys
Awards: G1037428335, G1255391969, G1335418636, G1375023953, G1601895641, G1763104746, G1872575776, G1891033315, G2026946236, G2230772921, G2330237237, G2513778610, G2641752829, A multidisciplinary approach for the stratification of patients with carotid artery disease, A high-dimensional approach for unwinding immune-metabolic causes of cardiovascular disease-depression multimorbidities, G3111916513, G3213590351, G33539551, Improving the early prediction of cardiovascular diseases by developing novel data-driven machine learning and multiomics approaches, G3582688621, G3764243850, G3858258543, Decision making and augmented reality support for automatic welding installations, G4139205196, G434337789, G4403028256, G4655867613, G4965406020, G5090981026, G5208171454, G523551622, G5474452856, G5489727732, G5512925732, G5646563246, G5678199996, G5770909501, G6159716710, G6164432047, G6310393984, G692875121, G69797456, Lifelong and transgenerational epigenetic profiles in relation to the development of cardiometabolic diseases, G7759301782
OpenAlex ID: W4403042613