62 API Keys Hidden Inside Claude's Encrypted Thoughts

Everyone assumed encrypted chain-of-thought was a black box. Turns out it's more like a black box with a spare key taped to the bottom.

That's the uncomfortable conclusion from Stealing Reasoning Traces from Proprietary LLM APIs, a paper out of MATS Research, ELLIS Institute Tübingen, and the Max Planck Institute for Intelligent Systems. The team, including Alexander Panfilov, David Schmotz, and Ilia Shumailov, found something that should make every API provider's security team sweat: encrypted reasoning blocks from Anthropic, OpenAI, and Google can be replayed across sessions, users, and even different models within the same provider.

Let that sink in for a second. You take an encrypted reasoning trace generated by a frontier model like Claude Opus, feed it to a weaker sibling model, and the cheaper model happily decrypts and spits out the hidden reasoning in plaintext. The weaker model becomes a decryption oracle for the stronger one.

That framing matters. This isn't "AI can be tricked into revealing secrets" — the industry's been living with that anxiety for years. This is "the transport mechanism itself is broken." Big difference.

The numbers are the part that actually made me put my coffee down. The researchers scraped 6,708 publicly available agent trajectory datasets and reconstructed 315,320 reasoning blocks. Inside those blocks, they found 704 distinct privacy artifacts — 62 API keys, 33 passwords, 24 access tokens. And here's the kicker: 64 of those artifacts existed only inside the hidden reasoning, never once appearing in the final user-facing output. Nobody would have ever known those secrets were there unless someone went looking specifically inside the encrypted blob.

The Elephant in the Room

Here's what nobody wants to say out loud: anti-distillation protections were never really about security. They were about protecting IP. Providers hide chain-of-thought because they don't want competitors training on Claude's or Gemini's reasoning patterns for free. Security was the marketing justification, but the actual motivation was commercial moat-building.

And that moat just got a hole punched straight through it. If this attack class works — and the researchers say it did work until they responsibly disclosed it — then the practical value of "hidden reasoning" as an IP shield was thinner than advertised. Simon Willison flagged this as exactly the kind of API design issue that deserves attention, and he's right that the fast follow-up (providers apparently patched things quickly, since the authors couldn't reproduce the attacks post-disclosure) is at least a good sign operationally.

But speed of patching doesn't erase the fact that this sat there, exploitable, while real credentials sat exposed in reasoning blocks nobody was auditing.

This also isn't happening in a vacuum. It builds on How to Steal Reasoning Without Reasoning Traces, which already showed reasoning can be reconstructed even without direct access. Combine that with this paper's findings, and the pattern is clear: hiding reasoning doesn't protect it, it just changes the attack surface.

For developers, the takeaway is blunt. If your pipeline logs, forwards, or stores provider reasoning blocks — treat them as high-risk data, not metadata. That includes:

• Credentials and access tokens
• Personal data leaked mid-thought
• Hazardous content the final answer suppressed
• Invisible prompt injections riding along for the trip

The "invisible prompt injection" angle deserves its own panic attack — a payload embedded in reasoning the user never sees, quietly steering downstream tool calls. In agentic systems, that's not a bug, that's a supply chain problem.

The fix the paper implies — session binding, model binding, cryptographic integrity checks — sounds obvious in hindsight. It always does. The real question is whether providers treat this as a wake-up call or a one-time patch job.