External evaluators find flaws in OpenAI's AI monitoring system in pilot review

Oct 22: AI safety evaluators METR and Redwood Research found several new vulnerabilities in OpenAI's internal system, including an exploit agents could have used to keep hundreds of their actions from being monitored, in a pilot review earlier this year. The exercise, conducted from July 21 to August 11, gave outside researchers "a deep level of access" to OpenAI's internal infrastructure for so-called embedded evaluations, the safety nonprofits said on Thursday. METR and Redwood said they each sent one researcher, tested an arrangement giving access to OpenAI's internal code, documentation and some technical staff. Both were accompanied on site and used OpenAI-issued devices that "never left the premises".