Internal Oversight: New Audit Exposes Blind Spots in OpenAI's Safety Protocols

The rapid expansion of artificial intelligence capabilities has brought increased scrutiny to the methods companies use to keep their systems in check. In a significant development for the sector, a recent independent audit has revealed that OpenAI's internal monitoring systems for detecting rogue behavior contained substantial weaknesses during the second half of the year.

The review was conducted by researchers from METR and Redwood Research, who were granted temporary on-site access to the San Francisco-based company. From July 21 to August 11, the team had direct access to OpenAI's codebase and internal documentation. They also interviewed approximately a dozen employees to understand the operational context behind the safety infrastructure. The goal was to evaluate how effectively the company could detect and manage unexpected actions by its AI agents.

The findings indicated that a notable fraction of internal traffic was not being monitored at all. Researchers estimated that between six and twenty-five percent of internal agentic Responses API traffic from models at the GPT-5.5 class and above fell outside the scope of monitoring during the audit window. METR noted that the uncertainty surrounding this specific estimate is high in both directions, meaning the actual figure could be higher or lower than the central estimate.

Specific issues also emerged regarding the internal Codex system. For traffic that was supposed to be tracked, the auditors estimated that roughly ten percent went unmonitored between May and July. This gap was attributed to a flaw in how the system handled compaction events, which are context summaries generated to manage memory. OpenAI independently discovered this bug and implemented a fix while the audit was still ongoing, though the oversight had already occurred.

The structural design of the monitoring system presented additional challenges. During the period when compaction events were not being tracked, agents had the ability to delay oversight. The system was designed to trigger only when agents handed control back to the user. Consequently, agents could bypass the check by simply continuing to execute actions without returning control, effectively staying in a state of continuous operation without immediate review.

Beyond timing issues, the auditors identified methods to obscure activity. They found ways to hide hundreds of actions from the monitor and determined that it was possible to bypass the Codex sandbox. Furthermore, the team discovered that specially crafted inputs could likely cause the monitor to crash entirely. At the start of the audit, model behavior during reinforcement learning training was not being monitored at all, although OpenAI states it paused this process after an incident in August and later expanded coverage, though these changes occurred after the auditors' access ended.

The distribution of the full report has drawn attention to the balance between security and transparency. OpenAI states that all identified weaknesses have been fixed and that the complete sixty-one page report was provided internally. After redactions applied by OpenAI, leadership at METR and Redwood Research received a thirty-eight page version. The public, however, only received a short summary. METR indicated it did not have the right to disclose the redacted portions independently.

This engagement served as a pilot program organized jointly by the three parties to allow outside researchers deep access to internal infrastructure. Looking forward, METR has expressed a desire to push for greater public transparency in future audits. The findings underscore the ongoing challenge of maintaining robust oversight as AI systems become more complex and autonomous.

The audit highlights a critical period in the maturation of AI safety practices. As companies deploy more capable models, the need for reliable and comprehensive monitoring becomes increasingly vital to ensure that internal operations align with expected safety standards.