Simon DeDeo, Joshua Grochow, Eric Libby, David Wolpert
Paper #: 15-06-017
To analyze high-dimensional systems, many fields in science and engineering rely on high- level descriptions, sometimes called “macrostates,” “coarse-grainings,” or “effective theo- ries”. Examples of such descriptions include the thermodynamic properties of a large collection of point particles undergoing reversible dynamics, the variables in a macroeco- nomic model describing the individuals that participate in an economy, and the summary state of a cell composed of a large set of biochemical networks.
Often these high-level descriptions are constructed without considering the ultimate reason for needing them in the first place. Here, we formalize and quantify one such pur- pose: the need to predict observables of interest concerning the high-dimensional system with as high accuracy as possible, while minimizing the computational cost of doing so. The resulting State Space Compression (SSC) framework provides a guide for how to solve for the optimal high-level description of a given dynamical system, rather than constructing it based on human intuition alone.
In this preliminary report, we introduce SSC, and illustrate it with several information- theoretic quantifications of “accuracy”, all with different implications for the optimal com- pression. We also discuss some other possible applications of SSC beyond the goal of accurate prediction. These include SSC as a measure of the complexity of a dynamical system, and as a way to quantify information flow between the scales of a system.