Analysing Information Distribution in Complex Systems
Name
Sten Sootla
Abstract
Information theory is a popular tool that is often utilized to capture both linear
as well as non-linear relationships between different parts of dynamical complex systems. Recently, an extension to classical information theory called partial information decomposition has been developed, which allows one to partition the information that two subsystems have about a third one into unique, redundant and synergetic information terms. To calculate these novel quantities in practice, a numerical estimator has been developed at the University of Tartu.
This thesis provides the very first examples of applying partial information de-
composition in complex systems research. Three complex systems are empirically analysed in terms of partial information decomposition using the numerical estimator.
First, the synergy in the Ising model was found to peak while the system was
still in the demagnetized, disorder regime. Second, a novel automatic and quantitative characterization of elementary cellular automata based on the information distribution in the automata was obtained. Last, feedforward neural networks were discovered not to be amenable to analysis with the current tools.
However, it was argued that analysing recurrent neural networks could yield more interesting results.
as well as non-linear relationships between different parts of dynamical complex systems. Recently, an extension to classical information theory called partial information decomposition has been developed, which allows one to partition the information that two subsystems have about a third one into unique, redundant and synergetic information terms. To calculate these novel quantities in practice, a numerical estimator has been developed at the University of Tartu.
This thesis provides the very first examples of applying partial information de-
composition in complex systems research. Three complex systems are empirically analysed in terms of partial information decomposition using the numerical estimator.
First, the synergy in the Ising model was found to peak while the system was
still in the demagnetized, disorder regime. Second, a novel automatic and quantitative characterization of elementary cellular automata based on the information distribution in the automata was obtained. Last, feedforward neural networks were discovered not to be amenable to analysis with the current tools.
However, it was argued that analysing recurrent neural networks could yield more interesting results.
Graduation Thesis language
English
Graduation Thesis type
Bachelor - Computer Science
Supervisor(s)
Raul Vicente Zafra, Dirk Oliver Theis
Defence year
2017