Show Notes
Communication is a physical process. It’s common sense that sending and receiving intelligible messages takes work…but how much work? The question of the relationship between energy, information, and matter is one of the deepest known to science. There appear to be limits to the rate at which communication between two systems can happen…but the search for a fundamental relationship between speed, error, and energy (among other things) promises insights far deeper than merely whether we can keep making faster internet devices. Strap in (and consider slowing down) for a broad and deep discussion on the bounds within which our entire universe must play… Welcome to COMPLEXITY, the official podcast of the Santa Fe Institute. I’m your host, Michael Garfield, and every other week we’ll bring you with us for far-ranging conversations with our worldwide network of rigorous researchers developing new frameworks to explain the deepest mysteries of the universe. This week we speak with SFI Professor David Wolpert and MIT Physics PhD student Farita Tasnim, who have worked together over the last year on pioneering research into the nonlinear dynamics of communication channels. In this episode, we explore the history and ongoing evolution of information theory and coding theory, what the field of stochastic thermodynamics has to do with limits to human knowledge, and the role of noise in collective intelligence. Be sure to check out our extensive show notes with links to all our references at complexity.simplecast.com. If you value our research and communication efforts, please subscribe, rate and review us at Apple Podcasts or Spotify, and consider making a donation — or finding other ways to engage with us, including a handful of open postdoctoral fellowships — at santafe.edu/engage. Lastly, this weekend — October 22nd & 23rd — is the return of our InterPlanetary Festival! Join our YouTube livestream for two full days of panel discussions, keynotes, and bleeding edge multimedia performances focusing space exploration through the lens of complex systems science. The fun begins at 11 A.M. Mountain Time on Saturday and ends 6 P.M. Mountain Time on Sunday. Everything will be recorded and archived at the stream link in case you can’t tune in for the live event. Learn more at interplanetaryfest.org… Thank you for listening! Join our Facebook discussion group to meet like minds and talk about each episode. Podcast theme music by Mitch Mignano. Follow us on social media: Twitter • YouTube • Facebook • Instagram • LinkedIn Referenced in this episode: Nonlinear thermodynamics of communication channels by Farita Tasnim and David Wolpert (forthcoming at arXiv.org) Heterogeneity and Efficiency in the Brain by Vijay Balasubramanian Noisy Deductive Reasoning: How Humans Construct Math, and How Math Constructs Universes by David Wolpert & David Kinney Stochastic Mathematical Systems by David Wolpert & David Kinney Twenty-five years of nanoscale thermodynamics by Chase P. Broedersz & Pierre Ronceray Ten Questions about The Hard Limits of Human Intelligence by David Wolpert What can we know about that which we cannot even imagine? by David Wolpert Communication consumes 35 times more energy than computation in the human cortex, but both costs are needed to predict synapse number by William Levy & Victoria Calvert An exchange of letters on the role of noise in collective intelligence by Daniel Kahneman, David Krakauer, Olivier Sibony, Cass Sunstein, David Wolpert When Slower Is Faster by Carlos Gershenson & Dirk Helbing Additional Resources: The stochastic thermodynamics of computation by David Wolpert Elements of Information Theory, Second Edition (textbook) by Thomas Cover & Joy Thomas Computational Complexity: A Modern Approach (textbook) by Sanjeev Arora & Boaz Barak An Introduction to Kolmogorov Complexity and Its Applications (textbook) by Ming Li & Paul Vitányi
More from COMPLEXITY
In the final episode of the season, Abha sits down with Melanie to hear her perspective. They chat about Melanie’s career and research with Douglas Hofstadter, the author of Gödel, Escher, Bach. They also discuss her opinions on LLMs’ current capabilities, what she thinks of existential questions like the alignment problem, how sustainable the industry is, the difficulty of making claims about concepts like “intelligence” and “understanding,” and what she thinks future technological development should focus on.
When it comes to assessing intelligence, people have all kinds of tests — the SAT, IQ tests, and so on. There’s controversy over how fairly these tests really measure human intelligence, but at the very least, we know that they correlate with some general reasoning skills when people take them. That assumption breaks down when we try to assess intelligence in non-humans. What does it mean when a large language model passes an intelligence test meant for humans? Does it actually have the same reasoning skills that a human does, or is it doing something else? In today’s episode, with guests Erica Cartmill and Ellie Pavlick, we investigate the best ways to assess intelligence in non-humans, whether animals or machines.
There’s an argument to be made that if we train AI systems to learn the way babies do, we’ll get them closer to human-like intelligence. But how our own learning development functions in babyhood is still a mystery that researchers are untangling. We know that the information babies absorb is very different from how an LLM learns, and in today’s episode, with guests Linda Smith and Michael Frank we’ll attempt to look at the world through an infant’s eyes and examine why they’re able to do more with, seemingly, less information.
Large language models, like ChatGPT and Claude, have remarkably coherent communication skills. Yet, what this says about their “intelligence” isn’t clear. Is it possible that they could arrive at the same level of intelligence as humans without taking the same evolutionary or learning path to get there? Or, if they’re not on a path to human-level intelligence, where are they now and where will they end up? In this episode, with guests Tomer Ullman and Murray Shanahan, we look at how large language models function and examine differing views on how sophisticated they are and where they might be going.
Complex language is unique to the human species. It’s part of how we evolved, the backbone of our societies, and one of the primary ways we judge others’ intellect. Is it our intelligence that leads to our language abilities, or conversely, does our ability for language enhance our intelligence, or both? How do language and thinking interact? And can one exist without the other? Guests: Evelina Federenko, Steve Piantadosi, and Gary Lupyan.