Rosenfeld Review Podcast

AI, Human Judgment, and High-Stakes Design with Joy KendiMwiti

Rosenfeld Review Podcast

Show Notes

As AI becomes easier and cheaper to deploy, designers face a new challenge: deciding not just what can be automated, but what should be. Lou Rosenfeld talks with Joy Kendi of Dalberg Design — and an upcoming speaker at the Designing with AI 2026 conference — about introducing AI into high-stakes systems where design decisions can directly affect access to healthcare, services, and critical resources. Joy shares examples from her work in public health systems, including an AI concept intended to help community health workers prioritize patient visits. What initially seemed like an obvious efficiency gain quickly raised deeper concerns around incomplete data, shifting real-world conditions, and the irreplaceable role of human judgment and community trust. Rather than treating AI as a decision-maker, Joy argues for designing systems where AI supports — but does not override — human expertise. Their conversation also explores how AI is reshaping the role of designers themselves. Joy makes the case that designers must move “upstream” in the process, helping define boundaries, risks, trust, and governance before automation decisions are made.

What You'll Learn from this Episode:

How Joy Kendi defines “high-stakes” design systems Why automation can fail in public health contexts The risks of relying on incomplete or outdated data Why AI should support human judgment, not replace it How designers are moving upstream in AI decision-making What human-centered AI governance can look like in practice

Quick Reference Guide: 0:11 - Meet Joy and learn about high-stake systems 3:26 - Joy’s backstory 6:10 - Seeing beyond the everyday facade 8:55 - Why you need the Rosenverse 11:08 - Designing responsible AI for high-stakes public health decisions 16:06 - Why human judgment still matters in AI-assisted decision-making 18:29 - Designers must shape AI boundaries, not just interfaces 22:06 - Joy’s gift for listeners

Resources and Links from Today's Episode: Designing with AI 2026 https://rosenfeldmedia.com/designing-with-ai/

Quotes: “When I say high stakes systems, I mean systems where the design choices carry very real consequences beyond the typical interface or screen that a lot of designers are used to.” “The hardest question is not just can we build this tool. It’s also deciding whether we should automate some of these complex social decisions at all.” “Even when the quality of data is good, and the data is available, there's more complexity to how human beings make decisions. A lot of that is also based on the social dynamics.”

More from Rosenfeld Review Podcast

Sep 30, 2026

What if researchers could learn as much from what people do with their hands as from what they say? Lou talks with Dane DeSutter and Stephanie Scopelitis, authors of the forthcoming Body Talk: Using Gestures to Rethink Human-Centered Products, about the overlooked role gestures play in research, communication, and design. Drawing on backgrounds spanning learning science, dance, music, and product research, Dane and Stephanie explain how gestures can reveal mental models, clarify ideas that are difficult to put into words, and give researchers new opportunities to dig deeper. Rather than relying on intuition alone, they argue that researchers can learn to observe gestures systematically—and use their own gestures intentionally to support richer interactions. Through examples involving everything from justice to data integration, they show how paying closer attention to the body can uncover insights that words alone might miss—and ultimately help designers create products that make more sense to the people using them.

Sep 21, 2026

As organizations race to automate with AI, the most important question may not be what machines can do, but what they should do. Lou talks with cognitive neuroscientist and UX researcher Stef Hutka, author of the forthcoming Rosenfeld Media book What Your Machines Should Do: The Science and Strategy of Human-Centered Automation, about how teams can make more thoughtful decisions about automation. Stef introduces the Autonomy Decision Matrix, a framework that asks teams to evaluate automation along two dimensions: Could you automate it? and Should you automate it? Using examples from augmented reality and AI-powered customer service, she explains why technical feasibility alone isn’t enough — and how shared frameworks can help leaders and practitioners make better decisions together. The conversation ultimately makes the case for “humanist automation”: using technology to extend human capabilities rather than simply replace people. Stef will continue exploring that idea as a curator of the human-centered tooling track at Rosenfeld Media’s upcoming Shift UX 2026, where she and the speakers will examine how our tools are reshaping not only products, but organizations, communities, and society.

Sep 15, 2026

What if uncertainty isn’t something UX professionals need to survive — but an opportunity to reshape what comes next? Catt Small and Lou explore how designers can reclaim agency at a moment when familiar roles, tools, and assumptions about the industry are shifting. Drawing on her own evolution from web designer to staff designer, Catt argues that career paths aren’t fixed: designers can identify where they bring the most value and help shape roles around their strengths. Catt and Lou discuss why today’s upheaval isn’t entirely new, how experimentation can help designers navigate ambiguity, and why the optimism that once surrounded technology has become more complicated. They also explore design’s role within the realities of business—and why making things better sometimes means slowing down, adding friction, asking harder questions, and helping organizations make more thoughtful decisions. Catt, who will open the inaugural Shift UX conference, ultimately offers a hopeful message: even in uncertain times, designers have agency—and uncertainty can be an ideal time to make meaningful moves. What You'll Learn from this Episode:

Sep 9, 2026

As UX roles and titles continue to shift, the ability to make sense of complexity may matter more than ever. Lou talks with information architect, author, and longtime collaborator Abby Covert about how her work evolved from information architecture into the broader practice of sensemaking—and why she believes that work extends far beyond people with “IA” in their job titles. Abby also offers a preview of Shift UX, Rosenfeld Media’s hybrid conference on September 24–25, where she is one of the main curators. She explains how the event itself is being redesigned to move away from “design theater”: speakers will share the stage, dedicated sensemakers will connect ideas across talks, and the audience will become part of the conversation. The conversation explores Abby’s Sensemakers Club, the importance of creating genuine community, and her new book, Timeless Sensemaking for Modern Sensemakers. Throughout, Abby makes the case that sensemaking isn’t simply about organizing information—it’s also about understanding people, adapting how we communicate, and helping others turn complexity into something they can actually use.

May 28, 2026

Product management is changing fast — and according to Julia Barham, many PMs are being pushed to the “left,” beyond delivery work into deeper discovery, customer insight, and strategic thinking. Julia joins Lou to discuss her forthcoming book, The Product Management Playbook: Create, Ship, and Optimize Winning Products, and why modern product managers need broader, end-to-end skills to thrive. Julia explains how AI and automation are reducing the amount of manual coordination and backlog management product managers traditionally handled — creating both anxiety and opportunity. As delivery work becomes easier, PMs are increasingly expected to spend more time understanding customers, validating ideas, collaborating across disciplines, and driving outcomes instead of simply shipping features. She also argues that great product managers succeed not because they know everything, but because they cultivate curiosity, build strong partnerships, and continuously learn across business, design, data, technology, and customer domains. The conversation also explores why many product managers feel disempowered inside organizations, the growing overlap between product and UX research, and how self-education, mentorship, and optimism can help professionals adapt during an uncertain AI-driven transition.

See all episodes →