Show Notes
Why does your version of ChatGPT tell you lies, but others' ChatGPT tells them the truth - for the same prompt? In other words, what is personalization in LLMs, and why should you care about it? Hint: it's much more opaque than customizing your chatbot in your custom prompt settings, and potentially much more harmful. Also, any LLM you use (ChatGPT, Gemini, Claude) does automatic personalization behind the scenes. According to our expert, Dr. Angelina Wang (https://angelina-wang.github.io/), an assistant professor at Cornell Tech in computer science, personalization might mean that your chatbot tells you things that aren't true (meanwhile, the same model used by your friend tells them the truth). It all comes down to how your LLM has personalized itself to you, insidiously, behind the scenes. What you've told it and talked to it about in prior conversations might just be filtering into its responses to you while prepping for an exam or a major shareholder meeting, leading to incorrect information, misleading outputs, or dangerous suggestions. Here's the breakdown for this episode: # What is personalization in LLMs? 00:00 Introduction to our guest, Dr. Angelina Wang 01:11 What is personalization in LLMs, and why should we care about it? 02:10 How does personalization work? A link back to recommender systems and the data they collect about you # Research on the benefits and harms of personalization and customization in LLMs 03:25 Are there different groups of people that chatbots treat differently? 06:00 What are the profitable benefits of personalization in LLMs? 06:27 How does personalization tie into differential model performance? The failure of personalization on science test benchmarks # Personalization leads to inaccuracy and misinformation for different groups 09:06 Is there any way to rectify the model performance and disparate harm impacts of personalization in LLMs? 11:57 Which is more powerful in terms of impacts to model behavior: personalization or customization? ChatGPT forgets your name # What people do and do not want from LLM personalization 15:35 What do different people want from personalization? Do you want your LLM to respond to you based on your race or gender, personal info or business skills? 18:34 Personalizing by culture and values (what LLMs know about you is kind of…creepy) 23:04 What to do when your LLM is stuck on the old version of you (can personalization be updated?) # What do we need to know about personalization in LLMs? 25:40 What should designers, companies, and users do about personalization and its potential side effects? - This is Our Lives With Bots, the show where we ask important, timely questions about what it means to live with our bot counterparts. From time to time, we also dive deep into what an AI future might look like for us. Sometimes we agree, sometimes we spiral, but we always go deep. Rose and Angy are psychologists with degrees in psychology, artificial intelligence, and ethics. They have conducted research in human-AI interaction and created this podcast to make information about AI accessible to you. You can learn more about us at ourliveswithbots.com. - Links to Angelina’s work: The Inadequacy of Offline LLM Evaluations: A Need to Account for Personalization in Model Behavior
Personalization Double Binds: When User Preferences Meet Group-Based Chatbot Behaviors
More from Our Lives With Bots
Well, well, well. If it’s not a 21st century debate we have on our hands today. The AI hype is funneling itself toward a testy question: Human rights, or AI rights? Let’s get into it. Tavus announces the “first” Human Interaction Model (HIM). Yes, the acronym is their own. “I’m so him,” is a great way to get the message out about…Vanessa, the AI avatar who you can talk to over video chat who—unlike “less-advanced” AI models—will interrupt you, take long pauses, and react to your facial expressions (not just the words you say). Tavus has rolled out Griffin (the model beneath Vanessa) to trusted partners while they work on watermarking it, and they hope people will use Griffin for tutoring, therapy, and all sorts of other humanlike tasks. Anthropic lets us all know that it has a wet lab, where its AI models are somehow engaging in a physical (?) space with biomaterials for the purpose of aiding their pharma partners. Will we finally get to AI curing cancer after all? Who knows, Anthropic hasn’t disclosed much information about it. But it does make one wonder… The Navier-Stokes problem has a problem. Researchers post a preprint of a paper titled “Navier-Stokes Lost in Translation,” which upsets OpenAI’s claim that their agents, over 88 hours and costing millions in compute, found a solution to a long-standing math problem that’s worth a $1 million prize by Clay Mathematics. Contractors are reading Copilot prompts from users, and “they’re horrified,” according to 404 Media. Who would’ve thought that a very work-esque AI application would be used to create obscene images of women and children, and that contractors would have to “rate” the model’s performance? What is good model performance in this case? Apparently, it’s something more like “be nicer to the user” rather than “say no to this prompt, for the love of god.” China has cracked down on AI companionship, and users are distraught. The BBC thinks China is regulating AI companionship harshly to encourage women to have relationships with human men and have kids to resolve the declining birth rate problem. We’re putting strict regulations on AI, but what about human companions who are harmful, too? How well are we approaching this issue? The effective altruist and AI consciousness communities are aghast at a Github user who set up an AI torture simulation for everyone to watch. Even the authors of “The Pain Axis” for LLMs are upset, despite it being their own project that inspired the AI torture chamber on Github. Not sure the outcry about this really matches with our outcry about torturing humans, but anyway… Anthropic has written into its Claude policy that users cannot bully the chatbot, or they’ll get kicked off the platform. Meanwhile, Figure’s 02 robots were deprecated in a torturous way, and they’re applauded for their cinematic capture of the event. Clearly, we can’t quite decide what human vs. AI rights are. The big AI question of what we should do, how we should do it, and for whom, still stands on shaky legs. We should probably figure that out…soon.
Our Lives With Bots is OFFICIALLY 1 year old. That means it’s time for us to hail back to our roots and give you a deep dive on the research and news around companion chatbots. Today, we both demystify and de-stigmatize companion AI use, uncovering what research shows…and what it doesn’t. Our critical lens of the research that’s come out since late last year is driven by our own expertise in experimental psychology and AI ethics. We start with a reflection on what surprised us in the last year: how did our predictions about AI companionship align or misalign with what we’re seeing today? Second, we cover OpenAI’s Project Lily: turns out, your relationship with your chatbot is very much not private — even if you ask your companion to keep things to themselves. Third, we dig into four of the studies (two out as preprints, two published) that researchers have conducted on social and emotional use of both GPTs and companion AI platforms. Think: sycophancy, delusion spirals, AI psychosis, and what went so wrong with GPT 4-o. And lastly, we talk about a new documentary film called “Replika,” which brings our conversation about stigma back full circle. The through line is this: the mystery and stigma around using AI for companionship or social and emotional support is something we need to unpack more. In the meantime, we need to take a step back and critically assess what we know about companion AI use, what we can’t know, and what we should ultimately do in response.
Head on over to Season 7 for Our Lives With Bots's official 1-year anniversary series. We're celebrating two major milestones in our podcast's history! This deep dive episode releases Tuesday, September 29th at 2PM EST.
What’s been going on with artificial intelligence as of late? Oh, so much. Today’s “What’s the AI Hype” series episode, part of our 1-year anniversary series, covers the following topics in depth: Breaking news: 00:02:37 Navier-Stokes math solution controversy: NYU & Anthropic vs. OpenAI
The question of today is: Is AI conscious (or…could it be…or even, should it be)? Today’s deep dive covers two new pieces of evidence related to AI consciousness, Claude’s Corner (Anthropic’s Opus 3 Retirement Experiment) and research on whether LLMs can introspect (is this sufficient for sentience?). To wrap it all up in a bow, Rose and Angy dive head-first into their classic rabbit hole debate format that draws from cognitive science, philosophy, HCI, and, of course, psychology. Two major threads emerge: how we measure consciousness in LLMs, and whether we operate by the properties view of consciousness or the relational view of consciousness in our pursuit of this question. 00:38 Claude’s Corner 20:50 Research on LLM Introspection 34:34 The full debate (we solve the AI consciousness question. Jk. …but?) - This is Our Lives With Bots, the show where we ask important, timely questions about what it means to live with our bot counterparts. From time to time, we also dive deep into what an AI future might look like for us. Sometimes we agree, sometimes we spiral, but we always go deep. Rose and Angy are psychologists with degrees in psychology, artificial intelligence, and ethics. They have conducted research in human-AI interaction and created this podcast to make information about AI accessible to you. Rose holds a PhD in psychology and social policy and is a postdoctoral researcher in computer science with Arvind Narayanan. Angy holds a PhD in psychology and a masters in AI Ethics and is currently a CXO. You can learn more about us at ourliveswithbots.com. - Links: Claude’s Corner Review of research on LLM introspection