Quantifying synergy among stochastic variables is an important open problem in information theory. Information synergy occurs when multiple sources together predict an outcome variable better than the sum of single-source predictions. It is an essential phenomenon in biology such as in neuronal networks and cellular regulatory processes, where different information flows integrate to produce a single response, but also in social cooperation processes as well as in statistical inference tasks in machine learning. Here we propose a metric of synergistic entropy and synergistic information from first principles. The proposed measure relies on so-called synergistic random variables (SRVs) which are constructed to have zero mutual information about individual source variables but non-zero mutual information about the complete set of source variables. We prove several basic and desired properties of our measure, including bounds and additivity properties. In addition, we prove several important consequences of our measure, including the fact that different types of synergistic information may co-exist between the same sets of variables. A numerical implementation is provided, which we use to demonstrate that synergy is associated with resilience to noise. Our measure may be a marked step forward in the study of multivariate information theory and its numerous applications

 

Quantifying Synergistic Information Using Intermediate Stochastic Variables
Rick Quax, Omri Har-Shemesh and Peter M. A. Sloot

Entropy 2017, 19(2), 85; doi:10.3390/e19020085

Source: www.mdpi.com

Open-ended evolution (OEE) is relevant to a variety of biological, artificial and technological systems, but has been challenging to reproduce in silico. Most theoretical efforts focus on key aspects of open-ended evolution as it appears in biology. We recast the problem as a more general one in dynamical systems theory, providing simple criteria for open-ended evolution based on two hallmark features: unbounded evolution and innovation. We define unbounded evolution as patterns that are non-repeating within the expected Poincare recurrence time of an equivalent isolated system, and innovation as trajectories not observed in isolated systems. As a case study, we implement novel variants of cellular automata (CA) in which the update rules are allowed to vary with time in three alternative ways. Each is capable of generating conditions for open-ended evolution, but vary in their ability to do so. We find that state-dependent dynamics, widely regarded as a hallmark of life, statistically out-performs other candidate mechanisms, and is the only mechanism to produce open-ended evolution in a scalable manner, essential to the notion of ongoing evolution. This analysis suggests a new framework for unifying mechanisms for generating OEE with features distinctive to life and its artifacts, with broad applicability to biological and artificial systems.

 

Formal Definitions of Unbounded Evolution and Innovation Reveal Universal Mechanisms for Open-Ended Evolution in Dynamical Systems

Alyssa M Adams, Hector Zenil, Paul CW Davies, Sara I Walker

Source: arxiv.org

The contribution of the body to cognition and control in natural and artificial agents is increasingly described as “offloading computation from the brain to the body,” where the body is said to perform “morphological computation.” Our investigation of four characteristic cases of morphological computation in animals and robots shows that the “offloading” perspective is misleading. Actually, the contribution of body morphology to cognition and control is rarely computational, in any useful sense of the word. We thus distinguish (1) morphology that facilitates control, (2) morphology that facilitates perception, and the rare cases of (3) morphological computation proper, such as reservoir computing, where the body is actually used for computation. This result contributes to the understanding of the relation between embodiment and computation: The question for robot design and cognitive science is not whether computation is offloaded to the body, but to what extent the body facilitates cognition and control—how it contributes to the overall orchestration of intelligent behavior.

 

What Is Morphological Computation? On How the Body Contributes to Cognition and Control

Vincent C. Müller, Matej Hoffmann

Artificial Life

Winter 2017, Vol. 23, No. 1, Pages: 1-24
Posted Online February 27, 2017.
(doi:10.1162/ARTL_a_00219)

Source: www.mitpressjournals.org

A major problem for evolutionary theory is understanding the so called {\em open-ended} nature of evolutionary change. Open-ended evolution (OEE) refers to the unbounded increase in complexity that seems to characterise evolution on multiple scales. This property seems to be a characteristic feature of biological and technological evolution and is strongly tied to the generative potential associated with combinatorics, which allows the system to grow and expand their available state spaces. Several theoretical and computational approaches have been developed to properly characterise OEE. Interestingly, many complex systems displaying OEE, from language to proteins, share a common statistical property: the presence of Zipf’s law. Given and inventory of basic items required to build more complex structures Zipf’s law tells us that most of these elements are rare whereas a few of them are extremely common. Using Algorithmic Information Theory, in this paper we provide a fundamental definition for open-endedness, which can be understood as {\em postulates}. Its statistical counterpart, based on standard Shannon Information theory, has the structure of a variational problem which is shown to lead to Zipf’s law as the expected consequence of an evolutionary processes displaying OEE. We further explore the problem of information conservation through an OEE process and we conclude that statistical information (standard Shannon information) is not conserved, resulting into the paradoxical situation in which the increase of information content has the effect of erasing itself. We prove that this paradox is solved if we consider non-statistical forms of information. This last result implies that standard information theory may not be a suitable theoretical framework to explore the persistence and increase of the information content in OEE systems.

 

Zipf’s law, unbounded complexity and open-ended evolution

Bernat Corominas-Murtra, Luís Seoane, Ricard Solé

Source: arxiv.org

Compartments are ubiquitous throughout biology, yet their importance stretches back to the origin of cells. In the context of origin of life, we assume that a protocell, a compartment enclosing functional components, requires N components to be evolvable. We take interest in the timescale in which a minimal evolvable protocell is produced. We show that when protocells fuse and share information, the time to produce an evolvable protocell scales algebraically in N, in contrast to an exponential scaling in the absence of fusion. We discuss the implications of this result for origins of life, as well as other biological processes.

 

Primordial Sex Facilitates the Emergence of Evolution
Sam Sinai, Jason Olejarz, Iulia A. Neagu, Martin A. Nowak

Source: arxiv.org