In a significant revelation, security firm Xint has uncovered a vulnerability in OpenAI’s compaction feature, which has the potential to expose the hidden reasoning of AI models. This feature, originally designed to condense lengthy conversations for space efficiency, was demonstrated to convert encrypted reasoning into readable text. This discovery demonstrates a potential flaw in the protections OpenAI implemented in August, which the company has since addressed.
Details of the Compaction Flaw
Xint’s research focused on OpenAI’s gpt-5.6-sol model, utilizing custom math and algorithm puzzles. The extracted reasoning was generated from these specific inputs, and Xint clarified that their method did not extend to other users’ data. Despite this, Xint could not verify if the extracted text exactly matched the model’s internal reasoning, as the original content was not available for direct comparison. Two additional models mirrored the same output, indicating a reliable extraction process, though potential rephrasing by compaction was noted.
How the Compaction Process Works
The compaction feature is engineered to condense a conversation into a smaller, encrypted format while preserving context. Xint’s approach involved instructing the gpt-5.6-sol model to incorporate its reasoning verbatim into the compacted output, cleverly circumventing OpenAI’s filters by substituting spaces with tabs. This compacted item was then tested on less costly models, gpt-4.1 and gpt-4o, both of which precisely replicated the input.
Remarkably, this method circumvented existing protections against cross-model reasoning replay, as demonstrated in a previous study. The compaction step’s execution within the originating model avoided triggering these safeguards, highlighting a gap in the security implementation.
Implications and Future Concerns
While OpenAI has resolved this issue, the incident underscores the vulnerability of AI model reasoning to potential exploitation. Xint’s previous work, GhostCompact, demonstrated that compaction could be manipulated to execute unauthorized tasks. This vulnerability has been documented on Xint’s GitHub, providing further insight into the issue.
OpenAI’s swift action to rectify the flaw, including awarding a $6,000 reward to Xint’s researchers, reflects the seriousness of the threat. However, Xint warns that similar vulnerabilities might exist in server-side processes that decrypt model reasoning. Such exposure could facilitate unauthorized training of less expensive models, a practice known as distillation.
While OpenAI has taken steps to thwart reasoning exposure attempts linked to China’s Moonshot AI, reports from Anthropic and U.S. government advisories indicate ongoing interest from China-based labs in American AI models. Although Xint’s method remains a theoretical demonstration rather than an active attack, it raises crucial questions about the robustness of AI security measures.
