Show Notes
Women gained seven grade points with Mainstay, making chatbot student performance impossible to ignore in AI college courses. In this episode: Women in an undergraduate Microeconomics course saw their final grades increase by seven percentage points due to course-specific chatbot messages, demonstrating how AI improved grades. The Mainstay chatbot improved chatbot student performance by providing timely administrative nudges and support links, rather than generating academic content. Research at Georgia State University showed that chatbot student engagement was most effective for the middle 80 percent of students, offering vital scaffolding without necessarily building transferable study habits. Implementing AI in higher ed for course support requires significant human design, oversight, and existing infrastructure for it to be effective, as evidenced by the NBER working paper. To successfully adopt AI college courses, educators should first audit specific student friction points and measure real outcomes and staff workload, rather than focusing solely on claims of AI improved grades. Chapters: 00:00 — Cold open & welcome 00:30 — Women gained seven points with chatbot student performance 01:00 — Mainstay chatbot: Nudging, not generating 01:45 — The invisible administration and chatbot student engagement 02:45 — Strongest effects for the middle 80 percent 03:45 — Gender effects and the Microeconomics course 04:30 — Performance vs. capability: No transfer of skills 05:30 — Implementation details for successful AI college courses 06:15 — Research credibility and limitations of the NBER paper 07:00 — Advice for leaders: Audit the problem first How can teachers use AI marking safely? The study suggests AI is best used for administrative nudges like reminders and connecting students to resources, rather than for academic decision-making or grading content directly. What specific benefits did the Mainstay chatbot provide in AI college courses? The Mainstay chatbot improved chatbot student performance by sending personalized text messages for deadlines, missing work alerts, encouragement, and links to tutoring, leading to higher grades and engagement, particularly for women. Did AI improved grades for all students? While students assigned to the chatbot were generally more likely to earn an A or B, the most significant gain was observed in women in the Microeconomics course, who earned grades seven points higher; men showed no comparable effect in this study. Featuring: Dan Fitzpatrick, NBER, Georgia State University, Mainstay, Registry of Efficacy and Effectiveness Studies, National Institute for Student Success, Katharine E. Meyer, Lindsay C. Page, Catherine Mata. Follow AI in Education with Dan Fitzpatrick for more on AI in education.
More from AI for Educators
Are screen time bans protecting children, or taking away tools some need to learn? Discover the accessibility blind spot in the screen time debate and what it means for parents navigating AI in education. Paddy McGrath, author of The AI Equity Paradox, joins Dan Fitzpatrick to explore how technology can help children communicate, build confidence and express ideas that might otherwise stay hidden. The conversation begins with a girl who couldn’t speak aloud. An iPad helped her deliver a speech about owls and win a prize. The technology hadn’t made her more intelligent. It had given her a way to show what she already knew. That story raises a difficult question: when we remove screens from classrooms, whose opportunities do we remove with them? As parents, we want our children to spend less time staring at screens. We also want them to grow up capable, creative and prepared for a world shaped by artificial intelligence. Paddy and Dan explore how to navigate those decisions, with practical examples from schools and their own families. IN THIS EPISODE ■ The screen time blind spot: Why blanket restrictions can overlook children who rely on assistive technology to learn and communicate. ■ “Necessary for some, useful for all”: How captions, dictation and other accessibility tools can benefit a much wider range of learners. ■ AI and homework: How to recognize when AI is supporting your child’s thinking and when it is doing the thinking for them. ■ The smartphone dilemma: Navigating friendships, online risks, parental controls and boundaries at home. ■ Children’s creativity: How AI can help bring a child’s original ideas to life, including Dan’s experience turning his son’s drawing into an animated creature. ■ The AI equity paradox: Why two schools with the same access to AI can offer their students very different opportunities. ■ Beyond test scores: Paddy’s argument for paying closer attention to confidence, independence and the barriers a tool helps remove. ■ When your school bans AI: How parents can model responsible use at home and share useful examples with teachers. If you’re a parent, teacher or school leader trying to make sense of screen time, AI literacy and inclusive education, this conversation offers questions you can take into your next discussion at home or school. What has technology helped your child do that they previously struggled with? Share your experience in the comments. Find us at https://theaieducator.io
OpenAI's always-on AI agents can now work around the clock. Here's what school leaders need to decide before they arrive in your district. This week's AI in education news: ChatGPT's new "dots" agents, the OpenAI model pulled over safety concerns, Meta's approval-first agent for small businesses, the America.gov AI assistant, the executive order directing federal agencies to say "Super Intelligence", Anthropic's leaked IPO prospectus, and Pope Leo XIV on AI risk. Each story comes with a practical action for principals, teachers and district leaders. Three actions for this week: build an agent register with an answer, draft, act rule, decide what student data never goes into shared AI workspaces, and put your AI vendors through a short risk and contract check. More at theaieducator.io
The UN launched a child-protection AI coalition this week, while Stanford's review shows schools have almost no US classroom evidence to go on. This week's episode covers the UN coalition and the White House access initiative, what NPR and Stanford found about the evidence base for AI in K-12, MIT's "illusion of learning" warning, England's Public First report on young people and AI, and what Massachusetts districts and the Microsoft/AFT privacy standard mean for your school's policies and contracts.
What should schools expect from the companies bringing AI into their classrooms? Microsoft President Brad Smith joins Dan Fitzpatrick to discuss the new AI Safety and Privacy Standard agreed with the American Federation of Teachers and its New York City affiliate, the UFT. Their conversation covers enforceable protections, school AI bans, teacher control, independent schools and whether the agreement could extend beyond the United States. Brad also addresses whether stronger safeguards could slow innovation. Dan then examines the latest PISA findings, the evidence behind Utah’s reported critical thinking gains, AI in special education advocacy and OpenAI’s Learning Lab. The episode closes with three practical actions to help schools check both the protections around their tools and what students are actually learning. Join The AI Educator Lab waitlist: https://lab.theaieducator.io Help shape an ongoing professional development journey and community, with no commitment required. Follow the podcast for your weekly roundup of AI news and research in education.
The two biggest school districts in America pulled students back from AI this week. In the same 72 hours, the AI labs handed teachers everything for free. Dan walks through the five stories that matter and what each one means for your school on Monday. NYC's one-year moratorium through eighth grade and the carve-outs nobody reported. LAUSD's district-wide restriction. The Royal Society finding that only 14% of teachers are confident with AI, with principals scoring lower than the teachers they lead. IBM's survey showing 77% of parents want input while 20% understand what's happening. OpenAI's 16-state privacy agreement, and why that matters more than the free access. And the Stanford review and Tennessee trial on what AI tutoring actually does. Plus three things to do this week: separate your staff and student policies, teach the thing before you police it, and redesign one task. The Educators' AI Guide 2027: theeducatorsaiguide.com Back to School AI Summit, 8-9 September: summit.aieducator.tools You can also watch on YouTube at https://www.youtube.com/@IamTheAIEducator