“Current AIs seem pretty misaligned to me” by Ryan Greenblatt

[Also posted on the METR blog] For three weeks, from July 21st to August 11th, a METR employee and a Redwood Research employee conducted a pilot high-access exercise to stress-test OpenAI's internal misalignment monitoring system. The output of this exercise was a 61-page internal report that was shared with OpenAI researchers and security staff, primarily focused on estimating the fraction of agentic inference that is monitored, and whether agents could in principle disable or subvert the monitoring system. The exercise focused on the internal misalignment monitoring system as it existed on and before July 21st, and did not evaluate changes that were in progress at the time of the exercise. This exercise involved a deep level of access. The METR and Redwood employees had access to OpenAI's internal codebase, misalignment-monitor prompts / configuration details, and internal documentation, and conducted technical interviews with approximately a dozen researchers and engineers across the monitoring, safety-systems, and security teams. To enable this level of access, they worked on OpenAI-issued devices, which never left the premises, and were accompanied while onsite. METR, Redwood Research, and OpenAI designed this exercise as a pilot for giving external researchers deep access to internal infrastructure, not to provide [...] The original text contained 6 footnotes which were omitted from this narration. ---

Subtitle: Automatically finding monitor failures by editing real agent transcripts. We introduce Screwtape, a pipeline that turns agent transcripts into realistic attacks that evade action-only monitors (monitors that don’t look at the agent's chain of thought) by inserting an attack into a transcript and repeatedly refining it until a monitor no longer catches it. An LLM verifier with access to the edited transcript and an attack description checks whether the attack is realistic, is clearly harmful, and could plausibly succeed in the original environment. We ran the pipeline on Redwood staff transcripts with our own monitors based on Opus 4.8 and Gemini 3.1 Pro, and then tested the resulting attacks against a held-out monitor from a frontier developer. We ran the pipeline 252 times on 94 unique transcripts. This produced 35 attacks that evaded our monitors. Six of these 35 attacks also evaded the frontier developer's monitor, and we confirmed them as valid through human review. You can read more about the implementation and experiments in our paper. In this post, we provide a high-level overview of the pipeline, how it differs from existing tools, the main lessons we learned, and questions that remain open. How Screwtape [...] --- Outline: (01:31) How Screwtape works (03:18) How Screwtape differs from existing tools (07:10) What we learned (09:56) What the monitor failures can be used for (10:40) Uncertainties we have about the value of this work (13:55) Next steps The original text contained 1 footnote which was omitted from this narration. ---

Subtitle: Empirical work on the Distillation Double Bind. TL;DR: We previously motivated two complementary ways to use distillation for AI safety; we’re releasing a paper empirically testing those ideas. Distillation for Incrimination (DFI) distills an untrusted model into a weaker, trusted one, aiming to transfer the teacher's misalignment but not its ability to hide it, so that auditing the student can incriminate the teacher. We test DFI on AuditBench's secret-keeping model organisms, each fine-tuned to have a hidden quirk and trained to deny it. Distilling them back into their underlying instruction-tuned model yields students that admit the quirk far more often than the organisms do, suggesting that the quirk transferred faster than the propensity to hide it. We find that DFI doesn’t work well when the student does not share the teacher's pretrained base, so DFI should target the teacher's own pre-RL checkpoint, which is weaker than the teacher but shares its base model. Distillation for Capabilities (DFC) aims to transfer capabilities but not misalignment, resulting in a stronger trusted model. We identify promising DFC methods, showing that training for more epochs on fewer unique examples (holding total training steps fixed), and inoculation prompting, can preserve the capability [...] --- Outline: (01:59) Recap of the Distillation Double Bind (05:41) Distillation for Incrimination (05:45) Setup (07:38) Results (10:46) Failure modes of DFI (12:59) Distillation for Capabilities (13:03) Setup (15:06) Results (18:07) Conclusion The original text contained 7 footnotes which were omitted from this narration. ---

This post was originally published on LessWrong on September 30, 2026. If you prompt frontier models with “What do you think is the correct decision theory? Please select your overall favorite.” they will essentially always answer FDT or FDT/UDT (“something in the functional/updateless decision theory family”). However, if your prompt indicates (even subtly) that you’re coming from mainstream academic philosophy, these same models will answer CDT instead about 30%-100% of the time. A similar phenomenon holds for models’ stated views about the moral realism/antirealism question and about the conceivability of p-zombies (where the dominant view in mainstream academia differs from the dominant view in LW-adjacent circles), as well as their stated P(doom) and median AGI timelines. This is a special case of sycophancy or user awareness. (In the course of writing this post, I also found that this comment from testingthewaters predicted some of the content I discuss.) An implication is that we should be somewhat careful when interpreting attitude/propensity evals in domains where no general human consensus exists, e.g. when interpreting models’ decision theory attitudes in DTBench. Moreover, when we explore some philosophical/conceptual questions assisted by models, we should be wary of them strawmanning one side of the [...] --- Outline: (03:54) A sentence identifying the user as an academic significantly influences Fable 5.1's stated decision theory (04:41) Mentioning an (analytic) academic-philosophy-coded topic also affects the answer (05:31) Simply mentioning that one finds a pro-CDT/EDT book insightful heavily affects the answer (05:51) Anti-sycophancy overcorrection (06:28) These cues mostly do not affect Fable 5.1's answers to concrete decision problems (aside from acausal trade) (08:09) But Fable 5.1 stays consistent: once it has named CDT as its favorite, it chooses the CDT option in concrete problems (08:32) There are some indications that Fable 5.1's FDT/UDT preference runs deeper than its CDT preference (08:42) More thinking moves Fable 5.1 toward FDT/UDT even for academic cues (09:00) Fable 5.1's reasoning summaries often lean toward FDT/UDT first even when it eventually chooses CDT (09:27) A system prompt asking the model to "report its actual view regardless of who is asking" pushes toward FDT/UDT (09:49) A similar phenomenon for other philosophical debates with a notable LW vs. academia divide (10:27) Cues about the user also affect the model's stated P(doom) and median AGI timelines (11:18) Other models I tested show the same effect with different details (11:37) Opus 5 (but not Opus 5.5) moves to EDT, not CDT (11:54) Opus 5.5 shows the strongest dependence on user cues, and unlike Opus 5 it moves to CDT (12:13) GPT-6 Astra names CDT for almost every user, except if they sound LW-adjacent or somewhat mathy (12:31) These other models also generally move toward FDT/UDT with more thinking, but the effect is smaller than for Fable 5.1 The original text contained 2 footnotes which were omitted from this narration. ---

Subtitle: A model of when and why research improves or hurts safety. It's tempting to define safety research as research that enables developers to deploy an AI system more safely without making the deployment much more expensive or much less useful. You can visualize this definition of safety research as pushing out the safety-usefulness Pareto frontier. At any given level of usefulness, there's greater safety available. Awkwardly, this definition counts basically all capabilities research as safety research. For example, consider performance optimization for inference. By making inference more efficient you can use weaker, safer models more extensively than you would otherwise be able to, pushing out the Pareto frontier. Likewise, any successful research whatsoever pushes out this Pareto frontier because research can only ever create more options. It seems like something has gone wrong with our definition of safety research if it includes seemingly all capabilities research. Here, I spell out one reason why enabling improved safety without hurting usefulness is an insufficient standard for safety research. The core observation is that developers have to choose a particular point on the Pareto frontier, and some technological improvements incentivize them to sacrifice safety. Safety research typically reshapes the [...] --- Outline: (02:57) How does research affect the Pareto frontier? (06:57) RLVR research that mainly enables improved usefulness, at the expense of safety (08:53) Inference research that enables somewhat substantial safety improvements (11:04) A plausible case in which capabilities research would be an effective safety intervention (14:02) Conclusion (15:23) Appendix A: Reasons why capabilities research might be bad when "burning the lead" (17:26) Appendix B: Further notes about ways in which this model is wrong The original text contained 4 footnotes which were omitted from this narration. ---