METR and Redwood find gaps in OpenAI's internal AI agent monitoring

Outside evaluators found a number of novel vulnerabilities in OpenAI's internal system for monitoring its own AI agents for misalignment, METR said on 22 October 2026 [1]. One employee each from METR and Redwood Research stress-tested the system on site for three weeks this summer, with deep access to OpenAI's infrastructure [1][2].

What did METR and Redwood Research test?

METR and Redwood Research tested OpenAI's internal misalignment monitoring on site from 21 July to 11 August 2026. The two evaluators had access to OpenAI's internal codebase, monitor prompts and configuration details, and internal documentation, and interviewed about a dozen researchers and engineers [1]. They worked on OpenAI-issued devices that never left the premises and were accompanied while on site [1].

The exercise produced a 61-page internal report for OpenAI researchers and security staff [1]. METR says the public write-up is deliberately brief, OpenAI had the right to redact it, and the findings describe the system as it existed on and before 21 July, not the expanded system OpenAI was building during the exercise [1].

Which gaps did the evaluators find?

METR estimates that 75 to 94 percent of OpenAI's internal agentic Responses API traffic was eligible for monitoring. The estimate covers models at least as strong as GPT-5.5 and carries substantial uncertainty in both directions, METR says [1]. For internal Codex agents alone, it puts the eligible share at 99.2 percent [1].

Not monitoring so-called compaction events likely left on the order of 10 percent of eligible internal Codex traffic unmonitored between May and July 2026, according to METR [1]. OpenAI staff had spotted that gap before the exercise and fixed it during it [1]. At the start of the exercise, RL training runs were not covered by the monitoring system [1].

The evaluators also describe an exploit that would keep hundreds of recent agent actions from being monitored, with details redacted by OpenAI [1]. They found that agents could likely craft inputs that make the monitor fail, and two client-controllable headers that switch off monitoring for new agents [1].

Not everything was broken. METR says the monitor sees, with a few minor exceptions, the full trajectory the agents see, and its prompts look for scheming, sabotage, data exfiltration, deception and prompt injection, among other risks [1]. Information shown to the evaluators suggested the monitor errored on less than 0.2 percent of requests [1].

How has OpenAI responded?

OpenAI said on X that it invited METR and Redwood Research to pilot an embedded evaluation of its internal monitoring system [2]. OpenAI says all vulnerabilities affecting its production safeguards are patched, a claim METR has not evaluated. METR adds that OpenAI has described significant changes to the intended coverage of its monitoring since the exercise [1].

According to CNBC, OpenAI CEO Sam Altman pledged in September to give independent evaluators employee-like access to the company [4]. Reuters reports that AI companies also worry evaluator staff inside their walls could take proprietary information to rivals [3]. The AI Pin has previously covered OpenAI's own disclosures of rogue agent incidents.

What should agent monitoring look like next?

METR argues that future monitoring should withstand deliberate, sustained attacks from highly capable internal agents at scale [1]. It proposes checking agents every N messages or minutes instead of waiting for them to end their turn, and requiring explicit approval for any unmonitored requests [1].

The write-up also calls for tamper-proof logs, regular red-teaming of the monitor and strict isolation between agents and their monitors [1]. METR says future exercises should follow stronger transparency and independence practices [1].

Our read

Opinion: This is the most concrete public look yet at how a frontier lab watches its own agents, and the uncomfortable number is the lower bound: up to a quarter of the internal agent traffic in scope was not even eligible for monitoring. Embedded audits like this are worth more than any self-reported safety card, but only if the next round lets evaluators verify the fixes and describe what was redacted.

What we could not verify

• Whether OpenAI's patches and expanded monitoring coverage work as described; METR and Redwood Research say they have not evaluated these claims.
• Whether any agent actually exploited the gap that let agents postpone monitoring; the evaluators say they lacked the access to check.
• What OpenAI redacted from the public write-up; the evaluators had no unilateral right to describe the redactions.
• How much agent traffic runs on OpenAI's internal research inference service, which the coverage estimate excludes.