Kernekoncepter
Practical methodology using machine learning for information decomposition in complex systems.
Resumé
The content discusses the application of machine learning to decompose information in complex systems. It introduces a practical methodology that uses the distributed information bottleneck to identify relevant variation in data. The analysis focuses on two paradigms: Boolean circuits and amorphous materials undergoing plastic deformation. The study aims to bridge micro- and macroscale structures in complex systems.
Directory:
- Introduction and Background
- Mutual information as a measure of statistical dependence.
- Importance of identifying relevant variation in complex systems.
- Methodology: Distributed Information Bottleneck
- Lossy compression of measurements using machine learning.
- Optimization process for extracting important variation.
- Results: Application to Boolean Circuits and Amorphous Materials
- Analysis of Boolean circuits with logistic regression, Shapley values, and distributed IB.
- Decomposing structural information in amorphous materials under deformation.
- Discussion: Comparison with Logistic Regression and Shapley Values
- Interpretability and insights provided by the distributed IB approach.
- Conclusion: Practical implications for studying complex systems.
Statistik
Mutual information provides a natural means of linking variation across scales of a system.
The distributed information bottleneck method is used to decompose the information contained in measurements.
For every point on the spectrum, there is an allocation of information over the inputs.