Get started
Work

Towards resolving the multivariate information controversy

Year: 2011
Type: preprint
Abstract: Information theory is widely accepted as a powerful tool for analyzing complex systems and it has been applied in many disciplines. Recently, some central components of information theory, multivariate information measures, have found expanded use in the study of several phenomena. Despite this widespread use, there is disagreement regarding the interpretation and use of these information measures. Due to the broad use of multivariate information measures, this problem prevents progress in many areas of study. Here, we seek to bring clarity to the situation by comparing the results from many proposed multivariate information measures for a simple system: Boolean logic gates. These logic gates represent the building blocks of computation and are well known across many disciplines. We find that a recently introduced measure, the partial information decomposition, provides the most complete description of the interactions present in the logic gates under examination. In addition, we apply the multivariate information measures to a dynamic system: a back-propagation network designed to learn the logic gates. Using the partial information decomposition, we find that logic gates which possess higher levels of synergy require more time for a back-propagation network to learn. Conversely, we find that logic gates which possess higher level of redundancy require less time for a back-propagation network to learn. This relationship was obscured when using the previously proposed information measures. Finally, we have made software available online which allows the user to calculate all of the information measures discussed within this paper, as well as software that can be used to create the back-propagation networks discussed herein.
Language: English
Open Access status: green
OpenAlex ID: W2406114473
Cites: 2
Cited by:
Related to: 20