Policy Experts Urge Measured Response to Chinese AI Model Distillation Practices

Chinese large language model (LLM) developers have come under scrutiny for employing extensive "distillation attacks" on U.S. frontier artificial intelligence (AI) models to enhance their own systems. This practice has drawn significant attention from U.S. actors who view distillation as a serious threat to national AI competitiveness and security. For instance, in May, Anthropic published a policy paper during former President Trump’s visit to China, highlighting distillation attacks as a critical challenge in U.S.-China AI competition. Earlier in April, the White House issued an official memorandum warning of "deliberate, industrial-scale campaigns" by Chinese entities engaging in distillation. The House Foreign Affairs Committee also advanced the Deterring American AI Model Theft Act to address these concerns, alongside other circulating policy proposals.

Despite frequent characterizations of distillation as a form of theft, experts emphasize important distinctions between "stealing an AI model" and distillation. Policymakers are urged to focus on preventing illegitimate access to AI models rather than imposing overly broad regulations that could harm the open and competitive U.S. AI ecosystem.

Originally, distillation referred to a machine learning technique where a larger "teacher" model’s outputs train a smaller "student" model, often using the teacher’s probability distribution over outputs rather than just correct answers. Today, the term encompasses broader practices, including prompting frontier models to generate outputs and using these prompt-output pairs or reasoning traces as training data to improve other models. Frontier models may also serve as evaluators in reinforcement learning processes. Collectively, these methods enhance weaker models by leveraging responses from stronger ones.

Distillation is a common and accepted practice in AI development. Elon Musk, during testimony in the Musk v. Altman trial, acknowledged that xAI had engaged in some distillation of OpenAI models and noted that "generally AI companies distill other AI companies." Nathan Lambert, a prominent U.S. open-source AI researcher, has highlighted distillation’s role in training smaller, often open-source models. The White House’s Office of Science and Technology Policy Director Michael Kratsios has recognized legitimate AI distillation as a "vital part" of fostering open models and maintaining a competitive AI landscape.

However, some Chinese AI developers appear to have escalated distillation practices beyond typical norms by accessing U.S. frontier models at massive scale. In February, Anthropic reported that three Chinese AI labs generated over 16 million exchanges with its Claude model using approximately 24,000 fraudulent accounts, sometimes employing jailbreak prompts to extract maximal information. OpenAI and Google have similarly reported or detected comparable distillation efforts.

These aggressive distillation campaigns have been framed as attempts at "model theft" and intellectual property (IP) misappropriation. Yet, experts caution that describing distillation as "stealing" misrepresents the process. Distillation does not involve hacking into internal systems or directly copying model weights or source code. Instead, it treats the model as a black box, learning from outputs without accessing proprietary internal components.

From an IP perspective, copyright protections do not apply to distillation since it does not copy software code or create derivative works with sufficient human authorship. Extending copyright to model outputs themselves would raise complex commercial and public policy issues, as AI-generated text lacks traditional authorship.

Patent law is similarly ill-suited to address distillation, as the process does not replicate patented implementations or methods. Notably, frontier AI labs have not asserted patent infringement claims related to distillation.

Trade secret law presents the strongest but still limited argument. While AI labs maintain secrecy to protect model details, distillation relies on publicly accessible outputs via the model’s interface. Trade secret protection requires reasonable efforts to maintain secrecy, which is undermined when outputs are available to any user with an account. The most compelling trade secret claims involve mass distillation using fraudulent accounts or jailbreak prompts to access hidden system prompts or other nonpublic information. However, the outputs remain the kind a legitimate user could obtain, and there is no evidence that distillers access full nonpublic reasoning chains or token-level probabilities.

The Eleventh Circuit’s decision in Compulife Software Inc. v. Newman illustrates the limits of trade secret claims in mass scraping contexts. That case involved proprietary databases and software code copying, unlike distillation scenarios. A recent lawsuit alleging trade secret misappropriation based on jailbreaking settled before merits were reached, with observers questioning the claim’s validity. Thus, under current law, mass distillation as described by frontier AI labs is unlikely to succeed as trade secret theft.

Distillation often violates AI labs’ terms of service (TOS), but equating all TOS violations with theft risks overbroad application. The critical legal issue is whether mass distillation involves false identities, misrepresented credentials, or circumvention of access controls. Such conduct could trigger civil or criminal liability under the Computer Fraud and Abuse Act (CFAA), though ordinary TOS violations typically do not.

In essence, distillation itself is not theft or hacking. Legal violations arise when actors circumvent safeguards designed to prevent distillation.

Policy responses should therefore prioritize securing frontier AI models against misuse by foreign competitors and state actors, and assessing whether distillation contributes to the proliferation of potentially dangerous model capabilities.

Effective policy must first clarify the problem. If the concern is unauthorized access by foreign competitors, measures should enhance security, facilitate threat information sharing, and detect fraudulent accounts and proxy networks. If cybersecurity is the focus, addressing account abuse and access control circumvention is paramount. If the diffusion of hazardous capabilities is the issue, research should determine distillation’s role in capability enhancement or safeguard removal.

These public interests support policies that protect access security, enable information sharing, prosecute unlawful conduct, and evaluate safety risks. They do not justify expanding IP protections or restricting legitimate competition in ways that favor AI labs’ commercial interests over the public good.

A fundamental defense against unauthorized distillation is for AI labs to detect and block users circumventing access controls or generating training data illicitly. This requires identifying usage patterns and signals indicative of distillation and terminating suspicious accounts.