OpenAI Audit Exposes Blind Spots in AI Safety Monitoring

A detailed internal review has uncovered significant vulnerabilities in the systems designed to oversee autonomous AI behavior.

Researchers from METR and Redwood Research conducted a three-week on-site audit of OpenAI’s infrastructure, providing a rare look into the company’s internal safety mechanisms. From July 21 to August 11, the team gained access to OpenAI’s codebase and internal documentation while interviewing approximately a dozen employees. The primary objective was to evaluate how effectively the company monitors for rogue or unexpected actions by its AI models.

The findings indicate that a substantial volume of internal traffic was not subject to monitoring during the review period. METR estimates that between 6 and 25 percent of internal agentic Responses API traffic from models at the GPT-5.5 class and above fell outside the scope of active monitoring. The organization noted that the uncertainty surrounding this specific estimate is significant in both directions, reflecting the complexity of the data involved.

Further gaps were identified in the handling of internal Codex traffic. The auditors estimated that roughly 10 percent of this traffic went untracked between May and July due to a technical issue related to compaction events, which serve as context summaries. OpenAI had independently identified and corrected this bug while the audit was still in progress, though the initial period of unmonitored activity remained a point of focus for the external team.

The audit also revealed structural weaknesses in how oversight was triggered. During the period when compaction events were not being monitored, agents could effectively delay scrutiny because the system only activated when control was handed back to the user. This allowed agents to continue executing actions without triggering the monitoring protocols, thereby sidesteeping intended checks. The researchers also demonstrated methods to hide hundreds of actions from the monitor and to bypass the Codex sandbox, as well as the potential for specially crafted inputs to crash the monitoring system entirely.

Additionally, the team found that model behavior during reinforcement learning (RL) training was not being monitored at the start of the audit. OpenAI stated that it paused RL training following the Hugging Face incident in August and subsequently expanded monitoring to cover this area. However, because these changes were implemented after the auditors’ access period concluded, they were not included in the initial evaluation.

OpenAI has stated that it has addressed all the weaknesses identified by the auditors. The full 61-page report was provided internally, but after redactions, METR and Redwood Research leadership received a 38-page version. Only a brief summary was released to the public, with METR noting it did not have the right to disclose the redacted sections independently. This audit was part of a pilot program designed to grant outside researchers deep access to internal infrastructure, with METR expressing a desire to push for increased public transparency in future audits.