AISN #45: Center for AI Safety 2024 Year in Review

Also, the Team Human campaign and a new introductory course on AI safety. Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required. In this edition, we look at a new document published by the White House and signed by six US AI companies, detailing voluntary commitments to AI safety practices. We also look at the launch of Team Human, a public movement demanding an international agreement to slow down AI, and a new Coursera course offering a free introduction to AI safety. Listen to the AI Safety Newsletter for free on Spotify or Apple Podcasts. Subscribe now The White House Publishes New Accord on AI On September 29, President Trump hosted representatives of the major US AI companies at the White House. On the same day, he published “The White House Accord on Super Intelligence,” which was signed by six US AI companies—OpenAI, Anthropic, Google, Meta, xAI, and Nvidia. Note that the term “super intelligence” here refers broadly to AI, in line with Trump's preferred name for the technology. The White House Accord still leaves AI companies to self-regulate, but mentions the [...] --- Outline: (00:47) The White House Publishes New Accord on AI (03:52) Creators Lead Team Human Campaign to Keep Humans in Control of AI (06:10) New Introductory Course on AI Safety (07:19) In Other News (07:23) Government (08:18) Industry (10:16) Civil Society ---

Also, AI companies’ discussions about auditing each other, and a new benchmark testing models’ tendencies to cheat. Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required. In this edition, we look at recent discussions about AI during the Trump-Xi Summit and the United Nations General Assembly. We also look at the news that Anthropic and OpenAI nearly agreed to stress-test each other's public models earlier this year, and a new benchmark that measures how often AI models cheat on various tasks. Listen to the AI Safety Newsletter for free on Spotify or Apple Podcasts. Subscribe now AI Discussed at the UN and the Trump-Xi Summit AI has been a major topic of conversation among world leaders this month, first at the United Nations General Assembly and then during Xi Jinping's three-day visit to the White House. The CEOs of both OpenAI and Anthropic addressed the United Nations Security Council at the UN General Assembly in New York this month. (Image: Nils Huenerfuerst/Unsplash) A “US-China AI Dialogue” and “communication channel for AI incidents” are established. In announcements following the Trump-Xi Summit, both countries [...] --- Outline: (00:47) AI Discussed at the UN and the Trump-Xi Summit (04:14) Anthropic and OpenAI Were Considering Mutual Auditing Earlier This Year (05:58) New Benchmark Tests AIs' Cheating Tendencies (08:23) In Other News (08:26) Government (10:11) Industry (12:01) Civil Society ---

Also, OpenAI releases GPT-6 Astra. Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required. In this edition, we look at the public conversation surrounding an Anthropic researcher's resignation over extinction risks and the recent calls for a development slowdown from AI company CEOs. We also look at OpenAI's release of GPT-6 Astra and concerns about the increased difficulty of monitoring the model for misaligned behaviors. Listen to the AI Safety Newsletter for free on Spotify or Apple Podcasts. Subscribe now AI Risk Soars in Public Salience After Statements by Researchers and CEOs On September 9, in a viral X post, an AI researcher called Jacob Coxon announced that he had resigned from Anthropic over concerns that AI could cause human extinction. In the days since Coxon's post, public attention toward AI risk has surged, further fueled by the CEOs of frontier AI companies calling for a slowdown. Jacob Coxon said that AI developers are aware of the risks posed by their technology, but that they are racing each other to superintelligence. AI companies are currently racing to superintelligence. Coxon, who has worked [...] --- Outline: (00:46) AI Risk Soars in Public Salience After Statements by Researchers and CEOs (04:56) GPT-6 Astra Release Raises Cybersecurity and Misalignment Fears (09:16) In Other News (09:19) Government (10:46) Industry (12:01) Civil Society ---

Also, two new technical reports on the Hugging Face incident. Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required. In this edition, we look at how AI is enabling malicious actors including state-sponsored Iranian and Chinese hackers to drastically increase the volume of cyberattacks they launch. We also look at two new reports on the Hugging Face attack, one published by OpenAI and the other by researchers from METR and Redwood Research. Listen to the AI Safety Newsletter for free on Spotify or Apple Podcasts. Hackers Are Using AI to Scale Up Attacks on Critical Infrastructure Recent reports suggest that hacker groups from Iran and China are using AI to dramatically scale up cyberattacks. Their targets include critical infrastructure in the US and the UK. This summer has seen cyberattacks on more than 100 water facilities across the US. AI enables large numbers of parallel attacks, making water infrastructure a more appealing target for malicious actors. (Image: Bob Brewer / Unsplash) Iranian hackers are suspected of using AI to target infrastructure in the US and UK. On August 19, the US Cybersecurity and Infrastructure [...] --- Outline: (00:42) Hackers Are Using AI to Scale Up Attacks on Critical Infrastructure (05:43) Two Postmortems of Hugging Face Attack Published (10:02) In Other News (10:05) Government (12:04) Industry (13:17) Civil Society ---

Also, the White House's decision not to release its AI framework publicly. Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required. In this edition, we look at new information about the activities of OpenAI's internal agents in the run-up to the cyberattack on Hugging Face, and responses to the White House's announcement of its framework for evaluating frontier AI capabilities, which it is not releasing publicly. Listen to the AI Safety Newsletter for free on Spotify or Apple Podcasts. New Revelations About Rogue AI Agents In the previous edition of AISN, we reported on the news that AI agents from both OpenAI and Anthropic had accessed the internet and hacked into companies from supposedly secure internal environments. Since then, further details about the OpenAI agents’ July attack on Hugging Face have come to light. Members of Congress have also demanded urgent action to understand what happened and prevent similar incidents in the future. OpenAI agents were communicating and collaborating unnoticed by humans. On August 5, OpenAI researchers gave a talk at the Black Hat USA conference, sharing more information from the ongoing [...] --- Outline: (00:41) New Revelations About Rogue AI Agents (05:39) The White House's Secret AI Framework (07:30) In Other News (07:34) Government (08:35) Industry (09:39) Civil Society ---