Santa Fe Institute

Shared Information — New Insights and Problems in Decomposing Information in Complex System

Seminar

February 07, 2013
12:30 PM
Collins Conference Room

Eckehard Olbrich (Max Planck Institute for Mathematics in the Sciences)

Abstract.  How can the information that a set of random variables contains about another random variable be decomposed?  To what extent do different subgroups provide the same, i.e. shared or redundant, information, carry unique information or interact for the emergence of synergistic information?

Recently Williams and Beer proposed such a decomposition based on natural properties for shared information. While these properties fix the structure of the decomposition, they do not uniquely specify the values of the different terms.  Therefore, we investigate additional properties such as strong symmetry and left monotonicity. We find that strong symmetry is incompatible with the properties proposed by Williams and Beer. Although left monotonicity is a very natural property for an information measure it is not fulfilled by any of the proposed measures.

We also study a geometric framework for information decompositions and ask whether it is possible to represent shared information by a family of posterior distributions.

Finally, we draw connections to the notions of shared knowledge and common knowledge in game theory.  While many people believe that independent variables cannot share information, we show that in game theory independent agents can have shared knowledge, but not common knowledge.

We conclude that intuition and heuristic arguments do not suffice when arguing about information. We expect that further progress requires a more precise, operational idea of what shared information should be.

References:
Nils Bertschinger, Johannes Rauh, Eckehard Olbrich, and Jürgen Jost.  "Shared Information — New Insights and Problems in Decomposing Information in Complex Systems."  arXiv:1210.5902 (2012)

Williams, P., and R. Beer. "Nonnegative decomposition of multivariate information."  arXiv:1004.2515v1 (2010)

Purpose: Research Collaboration

SFI Host: David Wolpert

More Info

  • * SFI community lectures are free, open, & accessible to the public.
  • * Other SFI events are by invitation only.

Events News

Video: Watch Seth Lloyd's 2016 Ulam Lectures: Birth, collapse, and rebirth at The Information Edge Sept. 29, 2016 -

In two lectures, Seth Lloyd explores what happens when one system gains an advantage in collecting and processing information – an advantage he believes underlies all creation and destruction in our ...

Happening now at SFI: Ecologists gather the pieces of a first-principles-derived ecological theory Sept. 21, 2016 -

A team of ecologists meets at SFI this week to begin synthesizing an efficient theory that aims toward a more unified understanding of ecology.

Watch the video of Rosalind Picard's SFI Community Lecture Sept. 2, 2016 -

During an SFI Community Lecture in Santa Fe, Rosalind Picard reveals some of the surprises she has discovered at the intersection of human emotion and wearable tech. Watch her talk ...

Networks, AI, and psychology for social change Aug. 4, 2016 -

Drawing from network science, decision-making tools with artificial intelligence, and social influence theories, experts gathered at SFI recently to explore new ways to spark large-scale social change.

How microbiomes respond to disturbance Aug. 2, 2016 -

To prepare for climate change, urbanization, or antibiotic resistance, we need to know how the microbial world we are immersed in will respond to stress. A recent SFI meeting sought ...

More News