Santa Fe Institute

Shared Information — New Insights and Problems in Decomposing Information in Complex System


February 07, 2013
12:30 PM
Collins Conference Room

Eckehard Olbrich (Max Planck Institute for Mathematics in the Sciences)

Abstract.  How can the information that a set of random variables contains about another random variable be decomposed?  To what extent do different subgroups provide the same, i.e. shared or redundant, information, carry unique information or interact for the emergence of synergistic information?

Recently Williams and Beer proposed such a decomposition based on natural properties for shared information. While these properties fix the structure of the decomposition, they do not uniquely specify the values of the different terms.  Therefore, we investigate additional properties such as strong symmetry and left monotonicity. We find that strong symmetry is incompatible with the properties proposed by Williams and Beer. Although left monotonicity is a very natural property for an information measure it is not fulfilled by any of the proposed measures.

We also study a geometric framework for information decompositions and ask whether it is possible to represent shared information by a family of posterior distributions.

Finally, we draw connections to the notions of shared knowledge and common knowledge in game theory.  While many people believe that independent variables cannot share information, we show that in game theory independent agents can have shared knowledge, but not common knowledge.

We conclude that intuition and heuristic arguments do not suffice when arguing about information. We expect that further progress requires a more precise, operational idea of what shared information should be.

Nils Bertschinger, Johannes Rauh, Eckehard Olbrich, and Jürgen Jost.  "Shared Information — New Insights and Problems in Decomposing Information in Complex Systems."  arXiv:1210.5902 (2012)

Williams, P., and R. Beer. "Nonnegative decomposition of multivariate information."  arXiv:1004.2515v1 (2010)

Purpose: Research Collaboration

SFI Host: David Wolpert

More Info

  • * SFI community lectures are free, open, & accessible to the public.
  • * Seminars & colloquia are geared for scientists but free & open to the interested public.
  • * All other SFI events are by invitation only.
  • * Note: We are unable to accommodate members of the public for SFI's limited lunch service; you're welcome to bring your own.

Events News

Sam Shepard on American culture, the stage and screen, and writing at SFI Sept. 17, 2014 -

A conversation with writer, actor, director and past SFI Miller Scholar Sam Shepard covers America’s cultural decline, the recent deaths of Philip Seymour Hoffman and Robin Williams -- and spending ...

SFI's Marcus Hamilton at SITE Santa Fe Tuesday night: Humans and unsettled landscapes Sept. 16, 2014 -

On Tuesday evening, September 23, at SITE Santa Fe, SFI's Marcus Hamilton will explore the changing nature of landscapes and the deep history of human landscape modification from prehistory ...

Ceremony to honor Murray Gell-Mann with prestigious Helmholtz Medal Sept. 5, 2014 -

Murray Gell-Mann will receive the prestigious Helmholtz Medal during a special ceremony on Thursday, September 25, at the Santa Fe Institute.

Video - Watch Steven Strogatz's SFI Community Lecture 'The Joy of x: A Guided Tour of Math' Aug. 12, 2014 -

In an August 6 SFI Community Lecture in Santa Fe, mathematician Steven Strogatz showed how math underpins our lives, from finding the perfect romantic partner to understanding how Google works ...

Video: Watch Seth Lloyd's talk 'Is time travel possible? Testing the Grandfather Paradox' July 30, 2014 -

In a July 16 SFI Community Lecture in Santa Fe, MITs Seth Lloyd proffered a mode of time travel that is both theoretically possible and creatively irresistible. Watch his talk.

More News