Chain-of-thought reasoning by a closed, proprietary model such as Claude Opus 4.8 is typically hidden. (Image: Unsplash)A new research paper could fan the flames of a hotly contested debate over ‘distillation’, an AI training technique that has come under scrutiny following the rise of Chinese open-weight models such as Moonshot AI’s Kimi K3.With frontier AI models capable of working through complex problems, researchers said that they have discovered a way to extract the hidden reasoning traces that these models generate before delivering their final outputs.They further said that Chinese developers may have employed the same techniques to extract the reasoning information and train their models on it via distillation. These findings are part of a research paper published by a group of researchers from the University of Tubingen, the Max Planck Institute, the AI safety institute MATS Research, and security company Snyk.The findings reportedly show that Kimi K3 produces strikingly similar outputs to the hidden reasoning traces of Anthropic’s Claude Opus 4.8 and OpenAI’s GPT 5.6 Sol for certain prompts. However, the paper stops short of providing conclusive proof that Chinese firms like Moonshot were ‘distilling’ reasoning information from US models.Despite the similarities, the work “cannot causally establish distillation,” researchers noted. Additionally, an open-weight model called Inkling from US-based Thinking Machines and another model from China’s DeepSeek did not exhibit similarities in reasoning with the Claude Opus model.The research comes in the backdrop of a high-stakes race between the US and China, with distillation emerging as a key flashpoint.For years, distillation has been recognised as a well-established, widely used AI/ML technique for efficiently copying the capabilities of existing models over to new ones. However, distillation has become a hot-button issue lately. When DeepSeek took the AI industry by storm, OpenAI alleged that the Chinese AI startup copied one of its models to build the R1 reasoning model. In June, Anthropic accused Chinese tech giant Alibaba of the same.Story continues below this adBut others in Silicon Valley have mounted a strong defence of distillation and open-weight AI models. In a lengthy blog post this week, Meta CEO Mark Zuckerberg said it “is an important principle of how the open source ecosystem works” and warned that restricting the practice would put the US at a disadvantage.How was the model’s ‘thinking’ uncovered?Chain-of-thought reasoning by a closed, proprietary model such as Claude Opus 4.8 is hidden to prevent other models from training on them. However, an encrypted version of the reasoning information is sent to a user’s computer in order to improve compute efficiency.To uncover this information, the researchers fed the encrypted reasoning traces to a smaller model of the same model family. Since smaller models do not undergo as much alignment training than their larger counterparts, they are more likely to reveal the reasoning information.Also Read | OpenClaw AI agent asked to book gym class ends up hacking system: What went wrong?In another experiment to find out whether open-weight models may have been distilled from closed ones, the researchers fed 90 questions to each of the models.Story continues below this adWhen open-weight models were fed the first few words of reasoning traces captured from a proprietary model, the researchers said that they saw the open-weight models, particularly Kimi K3, generate similar answers.Other key findingsThe technique of relying on smaller models to uncover a model’s inner artificial reasoning could be misused to retrieve sensitive user data such as passwords and API keys, the researchers warned.Also Read | Anthropic accuses Chinese AI labs of distilling Claude; Elon Musk calls it ‘guilty’In testing frontier models from OpenAI, Anthropic, and Google, the researchers said they were able to retrieve the sensitive data via API access to the models. They further said that the tech companies were alerted about the security vulnerability last month, and it has since been patched with each company adjusting its model API to mitigate the issue.“We value independent research on our models and have begun building short-term mitigations for the replay behaviors described in the report,” Michael Aciman, a spokesperson for Anthropic, was quoted as saying by Wired. He also denied that the researchers accessed Anthropic’s infrastructure, and recovered encryption keys and personal data from its systems.