Show Notes
Jim talks with Nate Soares about the ideas in his and Eliezer Yudkowsky's book If Anybody Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All. They discuss the book's claim that mitigating existential AI risk should be a top global priority, the idea that LLMs are grown, the opacity of deep learning networks, the Golden Gate activation vector, whether our understanding of deep learning networks might improve enough to prevent catastrophe, goodness as a narrow target, the alignment problem, the problem of pointing minds, whether LLMs are just stochastic parrots, why predicting a corpus often requires more mental machinery than creating a corpus, depth & generalization of skills, wanting as an effective strategy, goal orientation, limitations of training goal pursuit, transient limitations of current AI, protein folding and AlphaFold, the riskiness of automating alignment research, the correlation between capability and more coherent drives, why the authors anchored their argument on transformers & LLMs, the inversion of Moravec's paradox, the geopolitical multipolar trap, making world leaders aware of the issues, a treaty to ban the race to superintelligence, the specific terms of the proposed treaty, a comparison with banning uranium enrichment, why Jim tentatively thinks this proposal is a mistake, a priesthood of the power supply, whether attention is a zero-sum game, and much more.
More from The Jim Rutt Show
Jim talked with Cory Doctorow—prolific sci-fi and nonfiction author, journalist, activist, EFF special adviser, and author of Enshittification: Why Everything Suddenly Got Worse and What to Do About It—about how structural forces degraded the internet, and what citizens (not consumers) can actually do about it.
Jim talks with Tyson Yunkaporta—indigenous Australian scholar and author of Sand Talk, one of Jim's top ten favorite books—about his metaphysics and worldview, the ecology of sex and creation, and how to wear rationalist and traditional knowledge frameworks simultaneously.
Jim talks with Lisa Buckingham—a veteran HR leader at Vialto Partners, US Soccer, Lincoln Financial, and Thomson—about how the LLM era is reshaping hiring and job architecture, and how companies and workers can roll with the changes.
Jim talks with Peter Wang—chief AI officer, cofounder and CEO of Anaconda, board member of the Center for Humane Technology, and founder of the Austin STEM Center—about Robert Pirsig's metaphysics of quality, how modernity encourages defection, and a secular conception of the sacred.
Jim talks with recurring guest and deep systems thinker Jordan Hall about the scaffolding of his worldview. They discuss the waking-up scenario as a window into consciousness and personal identity, Jordan's phenomenology of waking and the "latent potential of all possible memory," the soul as the binding of finite and infinite, Jim's counter-framing of consciousness as a fusion of perception, interoception, and unconscious memory, the infinite as genuinely real, the Platonic triangle as a concrete example of transcendentals that have no particular location in the causal field, Forrest Landry's distinction between being and existence, knowing with confidence vs. knowing with certainty, Jordan's basic ontological commitment to realism, the incoherence of simulation theory, Jim's "Minimum Viable Metaphysics," the incoherence of unmediated access as the meaning of the word reality, Father Stephen DeYoung's critique of Western substantive essentialism, Bonitta Roy's idea that reality is shareable and participatory, Michael Levin's pragmatic epistemology, how purpose collapses reality to a tractable slice, "begottenness" in Christian metaphysics and the generativity of relationships, Jordan's onto-epistemology as the register before ontology and epistemology are distinguishable, Jordan's recent adoption of "smorthodox" Christianity, the phenomenology of waking as evidence that space-time is secondary, prioritizing meaningfulness over causation as a metaphysical commitment, Updike as "still alive" in the realization of his work, the Greek preoccupation with legacy and honor after death, Eric Weinstein's desire for Einsteinian legacy as a category error, love as the real currency of legacy, the Mark Twain reading as an example of a soul genuinely present in a room, Jim's father as an ongoing example of realization twenty-six years after his death, noticing a parent's turn of phrase in oneself, the sweetness of impermanence, the good vs. abusive father and different relationships to a parent's memory, values and virtues as real, the distinction between courage and bravery, culture as the progressive discovery and embodiment of virtue space, the crab-in-the-bucket problem, fallenness as local optimization, and much more.