Abeba Birhane, Vinay Prabhu, Sang Han, Vishnu Naresh Boddeti

`Scale the model, scale the data, scale the GPU-farms’ is the reigning sentiment in the world of generative AI today. While model scaling has been extensively studied, data scaling and its downstream impacts remain under explored. This is especially of critical importance in the context of visio-linguistic datasets whose main source is the World Wide Web, condensed and packaged as the CommonCrawl dump. This large scale data-dump, which is known to have numerous drawbacks, is repeatedly mined and serves as the data-motherlode for large generative models. In this paper, we: 1) investigate the effect of scaling datasets on hateful content through a comparative audit of the LAION-400M and LAION-2B-en, containing 400 million and 2 billion samples respectively, and 2) evaluate the downstream impact of scale on visio-linguistic models trained on these dataset variants by measuring racial bias of the models trained on them using the Chicago Face Dataset (CFD) as a probe. Our results show that 1) the presence of hateful content in datasets, when measured with a Hate Content Rate (HCR) metric on the inferences of the Pysentimiento hate-detection Natural Language Processing (NLP) model, increased by nearly 12% and 2) societal biases and negative stereotypes were also exacerbated with scale on the models we evaluated. As scale increased, the tendency of the model to associate images of human faces with the `human being’ class over 7 other offensive classes reduced by half. Furthermore, for the Black female category, the tendency of the model to associate their faces with the `criminal’ class doubled, while quintupling for Black male faces. We present a qualitative and historical analysis of the model audit results, reflect on our findings and its implications for dataset curation practice, and close with a summary of our findings and potential future work to be done in this area.

Read the full article at: arxiv.org

Your brain is not the root of cognition.

Read the full article at: nautil.us

Ute Deichmann

Entropy 2023, 25(6), 873

The debate about what causes the generation of form and structure in embryological development goes back to antiquity. Most recently, it has focused on the divergent views as to whether the generation of patterns and form in development is a largely self-organized process or is mainly determined by the genome, in particular, complex developmental gene regulatory processes. This paper presents and analyzes pertinent models of pattern formation and form generation in a developing organism in the past and the present, with a special emphasis on Alan Turing’s 1952 reaction–diffusion model. I first draw attention to the fact that Turing’s paper remained, at first, without a noticeable impact on the community of biologists because purely physical–chemical models were unable to explain embryological development and often also simple repetitive patterns. I then show that from the year 2000 and onwards, Turing’s 1952 paper was increasingly cited also by biologists. The model was updated to include gene products and now seemed able to account for the generation of biological patterns, though discrepancies between models and biological reality remained. I then point out Eric Davidson’s successful theory of early embryogenesis based on gene-regulatory network analysis and its mathematical modeling that not only was able to provide a mechanistic and causal explanation for gene regulatory events controlling developmental cell fate specification but, unlike reaction–diffusion models, also addressed the effects of evolution and organisms’ longstanding developmental and species stability. The paper concludes with an outlook on further developments of the gene regulatory network model.

Read the full article at: www.mdpi.com

Marcin Korecki

Entropy 2023, 25(7), 982

We studied the ability of deep reinforcement learning and self-organizing approaches to adapt to dynamic complex systems, using the applied example of traffic signal control in a simulated urban environment. We highlight the general limitations of deep learning for control in complex systems, even when employing state-of-the-art meta-learning methods, and contrast it with self-organization-based methods. Accordingly, we argue that complex systems are a good and challenging study environment for developing and improving meta-learning approaches. At the same time, we point to the importance of baselines to which meta-learning methods can be compared and present a self-organizing analytic traffic signal control that outperforms state-of-the-art meta-learning in some scenarios. We also show that meta-learning methods outperform classical learning methods in our simulated environment (around 1.5–2× improvement, in most scenarios). Our conclusions are that, in order to develop effective meta-learning methods that are able to adapt to a variety of conditions, it is necessary to test them in demanding, complex settings (such as, for example, urban traffic control) and compare them against established methods.

Read the full article at: www.mdpi.com

Steven Ceron, Gaurav Gardi, Kirstin Petersen, and Metin Sitti

PNAS 120 (24) e2221913120

Microscale collectives composed of simple, locally reactive constituents can harness the effects of self-organization to enable diverse global behaviors. While phase separation of homogeneous collectives is well studied, heterogeneous collectives are relatively unexplored. This study focuses on a collective of magnetic microdisks of different sizes and examines how the group can self-organize into homogeneous subgroups using an external magnetic field. We find that heterogeneity enables collective behaviors including morphology reconfiguration, organized aggregation, dispersion, and locomotion, and caging and expulsion of external objects. Our work furthers insights into self-organization of heterogeneous microrobot collectives and may provide useful insights into the future of active matter.

Read the full article at: www.pnas.org