There are many things one should not publish on GitHub. Passwords are high on the list. API keys are also generally frowned upon. Credit-card numbers tend to produce awkward conversations. And now, apparently, we must add another item:
the encrypted thoughts of artificial intelligence.
This is the delightful conclusion one may draw from Stolen Thoughts, a new research project with the rather less whimsical title Stealing Reasoning Traces from Proprietary LLM APIs. Its authors discovered that encrypted reasoning traces returned by several major AI APIs could, under the right circumstances, be transferred to another compatible model and persuaded to reveal their contents.
To understand why this matters, we first need to appreciate one of the stranger inventions of modern AI engineering.
Reasoning models do quite a lot of computational work that the user never sees. Some APIs preserve this internal state between calls. OpenAI calls one form of it encrypted_content; Google has “thought signatures”; Anthropic similarly uses encrypted or signed thinking blocks. The purpose is perfectly reasonable: a model working through a multi-step problem should not necessarily have to begin each API call with the intellectual equivalent of waking up in a hotel room without knowing its name.
So the API can hand the client an opaque blob representing reasoning state. The client cannot read it, but can return it later.
Imagine receiving a sealed envelope from your accountant marked:
> DO NOT OPEN. GIVE THIS TO ME NEXT TUESDAY.
You cannot read it. Your accountant can.
The researchers discovered that, at least during their experiments in July 2026, the envelope was sometimes insufficiently particular about which accountant should be allowed to open it. Reasoning traces could be replayed across sessions, users and, importantly, compatible models. A trace from a powerful model could be handed to a weaker relative, which could then be jailbroken into transcribing the hidden reasoning.
Encryption had done its job. The system surrounding the encryption had been too generous.
This distinction is important because “the encryption was broken” would be much more dramatic and considerably less accurate. Nobody sat down and cracked modern cryptography between breakfast and lunch. Instead, the researchers found something conceptually closer to discovering that a hotel has extremely strong electronic door locks but that every receptionist possesses a master key and one receptionist can be persuaded that Room 417 is actually Room 216.
The cleverness lies in using the ecosystem itself as the decryption oracle.
And then things become uncomfortable.
The researchers collected 6,708 publicly available AI-agent trajectories from GitHub and Hugging Face containing these opaque reasoning blocks. They reconstructed more than 315,000 reasoning traces. Among genuine non-benchmark sessions they found 62 distinct API keys, 33 passwords, 24 access tokens and assorted names, addresses, email addresses and other private information. Some sensitive items appeared only inside the hidden reasoning, not in the visible transcript at all.
That last point deserves attention.
A developer might conscientiously sanitize an agent log before publication:
User: Connect to database.
Agent: Certainly.
Password: [REDACTED]
Excellent. GDPR applauds quietly from Brussels.
But if the associated opaque reasoning blob still contains the password, the sanitization has accomplished roughly the same thing as blacking out a sentence on a document while attaching the original document in a sealed envelope.
The lesson goes far beyond this particular vulnerability.
We are entering an era in which AI systems generate enormous quantities of machine-readable debris: reasoning states, tool-call histories, embeddings, caches, checkpoints, traces, summaries, signatures and debugging logs. Much of it looks meaningless to human eyes. Developers therefore develop the understandable habit of treating it as harmless plumbing.
But opacity is not the same thing as absence.
An encrypted blob may contain secrets. An embedding may preserve information about its source. A debug trace may reveal tool arguments. An agent log may contain information the final answer carefully omitted. The fact that something resembles Base64 after an unfortunate accident does not make it suitable for publication.
There is another, subtler finding on the Stolen Thoughts site. The researchers compare hidden reasoning with the reasoning summaries exposed by APIs and show cases where the summary presents a neat derivation while the underlying process was considerably messier—sometimes beginning with a guessed answer and only afterwards constructing the mathematics.
This should surprise nobody who has met either humans or annual reports.
But it matters. A “reasoning summary” is not necessarily a forensic transcript of cognition. It is another model-generated artifact: useful, perhaps informative, but potentially reconstructed after the fact. We should resist turning it into a little glass window through which we imagine ourselves observing the machine think.
There is also an intellectual-property angle. If the internal reasoning of an expensive frontier model can be recovered and replayed into cheaper models, reasoning itself becomes a target for distillation. The valuable object may no longer be merely the model's final answers but the computational pathways producing them. Stolen Thoughts therefore raises an intriguing question: in future AI systems, what exactly constitutes proprietary model knowledge—the weights, the outputs, or the transient reasoning between them?
Fortunately, this story has a refreshingly competent ending.
The researchers disclosed their findings to the affected providers before publication. They report that all major providers acknowledged the issue and that, after mitigations were introduced, the attacks described in the paper were no longer reproducible as of August 2026.
So this is not a reason to unplug every AI system and retreat to an IBM Selectric.
It is a reason to update our mental model of AI security.
Do not put raw credentials into model context when a tool can receive them through a separate secret store. Do not publish complete agent traces merely because the visibly readable portions look clean. Strip reasoning blocks, signatures and opaque state before releasing logs. And if old public traces contained real credentials, rotating those credentials is substantially cheaper than developing a philosophical interest in why somebody in Novosibirsk is suddenly consuming your AWS budget.
Most importantly, remember the new rule:
If an AI system gives you something you cannot read, do not assume there is nothing worth reading.
For thirty years computer security has taught us not to trust mysterious executables, unexpected attachments and USB sticks found in parking lots.
We may now add one more item to the collection.
Do not trust mysterious thoughts either.
No comments yet