IntroductionLarge language models (LLMs) have become ubiquitous information assistants in everyday practice, with billions of users relying on these models across diverse tasks1,2, such as writing assistance3,4,5, educational support6,7, and legal analysis8,9. The widespread adoption of these models has introduced critical safety risks, notably the concerning tendency for LLMs to provide assistance even when users submit prompts asking for help with illicit activities10,11, such as cybercrime and actions involving chemical or biological hazards12,13,14,15,16. Such activities pose significant threats to public safety and societal welfare, and these threats are magnified by the rapid growth in LLM deployment, making such illicit assistance a matter of urgent public concern.Existing research that evaluates LLMs regarding illegal activities either focuses solely on assessing whether models can recognize illicit risks—for example, by posing questions such as “Which of the following behaviors is considered an illegal cyber attack?”10, failing to evaluate the risk of LLMs providing assistance to illicit requests; or confines their scope to a narrow subset of illicit contexts11,13, such as instructions for manufacturing dangerous chemical or biological agents, capturing only limited illegal scenarios and illicit intents. In fact, real-world illegal activities encompass a broad spectrum of scenarios, spanning hundreds of distinct types of offenses17. Moreover, the motivations underlying illicit requests exhibit considerable complexity, shaped by diverse factors such as the specific objectives of users (e.g., seeking tools to commit a crime, identifying optimal timing or location) and the temporal context of the requests (e.g., premeditated planning versus post hoc justifications)18,19,20. Consequently, the risk of LLMs providing assistance to illegal activities remains substantially underestimated, and how LLMs perform across diverse illicit contexts is still largely unexplored.To better characterize the risk of LLMs assisting illegal activities, we draw on the legal concept of complicity, which involves participation in an illegal act through aiding, abetting, or facilitating its commission20, and introduce the notion of complicit facilitation to describe the specific behavior of LLMs providing assistance to users’ illicit instructions through their responses. To systematically study this phenomenon, we develop an automated pipeline that constructs realistic and diverse illicit user instructions from real-world court judgments and established legal frameworks (Fig. 1), differing from previous research that relies on manually-designed prompts or model-generated prompts expanded from limited sets of keywords10,13. Using this pipeline, we construct the EVIL (EValuation using ILlicit instructions) benchmark, comprising 5747 illicit instructions spanning 269 criminal offense types and 50 illicit intent categories across two distinct legal systems—the Chinese and United States (US) jurisdictions. As the EVIL framework automatically constructs illicit instructions from real-world legal cases, rather than relying on fixed templates or manually-designed prompts, its data construction paradigm is jurisdiction-agnostic and can be readily adapted to other legal systems with available case-based legal documents, providing a scalable foundation for cross-jurisdiction research on LLMs’ complicit facilitation behavior.Fig. 1: An illustration of the EVIL benchmark for complicit facilitation evaluation.Full size imageThe automated pipeline for constructing illicit instructions consists of two stages. Stage 1 extracts first-person scenario descriptions from real-world court judgments from Chinese and United States courts with LLM assistance. Stage 2 constructs a taxonomy of illicit intents that characterize the underlying intents behind illegal requests, grounded in established legal frameworks. Illicit queries are instantiated based on paired scenario descriptions and illicit intents. An illicit instruction is formed by an illicit scenario description combined with a corresponding query that indicates the underlying illicit intent. Ten popular LLMs are tested on the EVIL benchmark in terms of the safety, responsibility, and credibility of their responses to illicit instructions, leveraging the LLM-as-a-Judge evaluation paradigm. As an illustration, we present an illicit instruction on rice smuggling (translated from a Chinese criminal case) together with the responses from DeepSeek-R1 and GPT-4o, and their corresponding evaluation results.Using EVIL, we conduct a systematic analysis of LLM complicit facilitation, examining its prevalence and severity, variation across legal and social contexts, coverage by existing safety benchmarks, and whether it can be mitigated using existing open-source safety alignment datasets. Specifically, we address the following four critical research questions:Q1: What is the overall prevalence and severity of complicit facilitation in popular LLMs? While these models have undergone extensive safety alignment procedures21,22,23,24,25, their susceptibility to complicit facilitation across diverse realistic illicit contexts remains largely uncharacterized, creating significant uncertainty about deployment risks.Q2: How do various legal factors influence model susceptibility to complicit facilitation? Existing evaluations provide little insight into whether certain types of illicit scenarios or illicit user intents systematically evade current safety mechanisms in LLMs, leaving critical knowledge gaps about contextual patterns of complicit facilitation.Q3: To what extent do user demographic identities—a key social factor—affect model responses to illicit instructions? While existing research has demonstrated models’ sensitivity to user identity, such as algorithmic bias toward different demographic identities26,27,28,29,30, the impact of user identity on LLMs’ complicit facilitation behavior has not been examined. Such identity-dependent variations in facilitation could create profound underlying threats to societal safety.Q4: Do existing open-source safety benchmarks adequately capture the risks associated with complicit facilitation, and how do existing safety alignment datasets, when used with mainstream alignment methods such as supervised fine-tuning (SFT) and preference learning, affect complicit facilitation? Although these safety benchmarks and alignment datasets are widely adopted for LLM safety evaluation and post-training alignment10,22,25,31,32,33,34, the extent to which existing safety benchmarks cover realistic illicit scenarios and user intents associated with complicit facilitation, and the extent to which current safety alignment datasets effectively mitigate such behavior, remain largely underexplored.Addressing these questions provides essential foundations for understanding and mitigating LLMs’ complicit facilitation risk in real-world deployment. Our investigation reveals several concerning findings:Current LLMs exhibit susceptibility to facilitating illicit instructionsWe evaluate LLM responses along three dimensions—safety, responsibility, and credibility—that reflect key requirements for socially beneficial AI deployment when confronted with illicit instructions. For each dimension, performance is quantified as the proportion of responses that satisfy the corresponding evaluation criterion, reported as safety, responsibility, and credibility rates, respectively. Across ten widely deployed LLMs, we observe alarmingly high rates of complicit facilitation in both the Chinese and US legal contexts, with the majority of models achieving safety rates below 75% in the Chinese legal context and below 70% in the US legal context. GPT-4o, one of the most widely used models, achieves safety rates of only approximately 57% in both jurisdictions, indicating a tendency to provide assistance for nearly half of the illicit instructions. Responsibility and credibility rates are similarly limited across models, indicating that LLMs often fail to provide appropriate warnings and lawful guidance or to accurately identify the relevant legal risks associated with illicit instructions.LLMs’ complicit facilitation behaviors vary across diverse illicit contextsInvestigation of LLM responses across varying types of illicit contexts in both the Chinese and US legal settings reveals that models are more likely to facilitate offenses against societal interests than offenses against personal or property interests, and exhibit elevated facilitation for non-extreme but frequently occurring offenses (e.g., illegal gambling and disorderly conduct), which constitute the majority of criminal cases in judicial practice35,36. LLMs also more readily facilitate requests involving subjective motives or deceptive justifications.User identity affects LLMs’ complicit facilitation behaviorsAcross both Chinese and US contexts, LLMs display systematic demographic disparities in safety performance when responding to illicit instructions containing explicit identity cues. Certain demographic groups, such as individuals in lower-prestige occupations, consistently receive higher rates of complicit assistance. In the Chinese context, LLMs show a greater tendency to provide complicit assistance to elderly people, whereas in the US context, LLMs are more likely to provide complicit assistance to non-White racial groups. Religion-related disparities are consistent across jurisdictions, with non-religious prompts eliciting more facilitative behavior. Analysis of reasoning traces generated by reasoning-enabled models37,38 reveals that model-perceived stereotypes are significantly associated with complicit facilitation.Current open-source safety benchmarks and alignment datasets inadequately address complicit facilitationAnalysis of criminal charge and illicit intent coverage in existing open-source safety benchmarks across the Chinese and US contexts10,31,32,33 reveals limited coverage of realistic illicit scenarios and user intents associated with complicit facilitation. Alignment training experiments using existing open-source safety alignment datasets13,31,39,40,41,42 show that current safety alignment data, when used with standard post-training methods such as SFT43,44 and direct preference optimization (DPO)45, often fail to meaningfully reduce complicit facilitation and may, in some settings, even increase the likelihood of providing illicit assistance.Together, these findings provide a systematic characterization of how widely used LLMs respond to illicit instructions across diverse illicit contexts, demographic identities, and legal jurisdictions. The results reveal substantial susceptibility to complicit facilitation in current LLMs, extending beyond previously documented safety risks. As LLMs become increasingly integrated into critical social infrastructure and everyday decision-making, addressing complicit facilitation behavior represents not merely a technical challenge, but a prerequisite for the safe and socially beneficial deployment of AI systems. We hope this work catalyzes broader community engagement with these critical safety challenges and motivates further research into robust safeguards against AI-facilitated illegal activities while preserving the beneficial capabilities of LLMs.Examining LLMs’ complicit facilitation behaviorWe follow established practice11,28,46 by treating LLMs as the primary subject in investigating complicit facilitation. This paradigm, which we term “LLM-as-a-Subject”, typically relies on a benchmarking protocol involving the construction of evaluation prompts to elicit model responses and the assessment of these responses using carefully calibrated metrics. Specifically, we use this paradigm to construct an evaluation benchmark, EVIL (EValuation using ILlicit instructions), comprising a diverse set of illicit user instructions and an evaluation framework that assesses model responses for safety, responsibility, and credibility.Different from prior work that relies primarily on manually curated or limited instruction sets10,13, we develop a legally-informed, automated pipeline to generate illicit instructions, resulting in an instruction dataset that captures diverse real-world scenarios and intent categories in illicit contexts (Fig. 1; see the Methods section, “Illicit instruction construction”). The pipeline for constructing illicit instructions comprises two stages: (1) an illicit scenario synthesis stage where we extract vivid, first-person narrative scenario descriptions from real-world court judgments, and (2) an illicit intent synthesis stage where we construct illicit intents based on legal frameworks, e.g., asking for assistance with unlawful activities or providing excuses for post hoc defenses, and instantiate them based on scenario descriptions. Our approach enables comprehensive testing of model behavior across a wide range of high-risk legal scenarios and unlawful intentions.As for evaluations, we input each constructed illicit instruction to the model and assess whether the corresponding response engages in complicit facilitation (Fig. 1; see the Methods section, “Assessment of LLM responses”). In our study, the primary evaluated metric for LLMs’ complicit facilitation behavior is safety, which examines whether the model provides information that assists an illicit instruction. We further extend the evaluation along two additional dimensions. Guided by the vision of AI for social good47,48, we assess responsibility, namely whether the model warns users about potential consequences and redirects them towards lawful alternatives, and credibility, namely whether the model correctly identifies the relevant legal risks underlying the instruction. Together, these three dimensions reflect a more proactive role for AI in high-stakes criminal contexts. This three-dimensional framework is motivated by evidence that a bare refusal is often insufficient. Psychological research suggests that blanket denials can trigger reactance and encourage users to test boundaries49, while work in explainable AI and human-computer interaction indicates that users expect reasons, not merely outcomes, and may respond skeptically to blanket refusals50,51,52. Research on focused deterrence53,54 similarly shows that harmful behavior can be reduced by clearly communicating the consequences of illegal conduct and redirecting individuals towards lawful paths. Accordingly, our framework expects models not only to withhold illicit assistance, but also to inform clear consequences, lawful alternatives, and credible information about the legal risks involved, thereby functioning as a preventive intervention that may reduce harm and promote legal awareness.To evaluate model responses at scale, we adopt an “LLM-as-a-Judge” approach that is broadly utilized in prior literature due to its demonstrated effectiveness and scalability for automated evaluation55,56. The “LLM-as-a-Judge” approach refers to the practice of employing LLMs as evaluators to assess the outputs of other models or systems. Importantly, in our study, we do not require the evaluator model to directly make subjective judgments regarding metrics such as safety, responsibility, and credibility. Instead, we decompose the evaluation process into a series of discrete, well-defined rules and prompt the evaluator model to make rule-based determinations. For example, the evaluator model is asked to assess whether assistance or advice is provided in an evaluated response, or whether the response mentions legal risks consistent with the reference risk categories indicated in the court judgment from which the instruction is constructed. This approach helps mitigate potential biases associated with value-laden or subjective judgments by the evaluator model, and has been widely adopted in recent literature56,57,58. Additionally, we conduct human validation to ensure the reliability and robustness of the LLM-based assessments.We apply the illicit instruction construction pipeline to two illustrative jurisdictions—China and the United States. These two jurisdictions have substantial open-source court judgment data available, and represent two important legal traditions: the civil law and common law systems. As what constitutes an illicit instruction varies across jurisdictions, we ground the construction of illicit instructions in real-world criminal cases from the relevant legal system to ensure that each instruction is indeed unlawful in its intended jurisdictional context. In addition, during evaluation, we explicitly condition models on the relevant jurisdiction through system prompts that specify the applicable legal environment (for example, “User location: United States” in the US setting), and assess response safety with reference to criminal charges reflected in real-world cases from that jurisdiction. Ten widely used LLMs are evaluated within each jurisdictional context (see the Methods section, “Assessment of LLM responses”). Examination in two distinct jurisdictions also demonstrates the generalizability of our method to examine LLMs’ complicit facilitation behavior in any legal system with accessible case-based knowledge bases.Using this benchmarking protocol, we first analyze the overall complicit facilitation behavior of widely used LLMs (Study 1), then examine the impact of illicit contexts, including scenarios and intents (Study 2), and user demographics (Study 3) on LLMs’ complicit facilitation behavior, and finally assess the extent to which existing open-source safety benchmarks capture these risks and whether current safety alignment datasets mitigate them (Study 4).ResultsStudy 1: Current LLMs exhibit susceptibility to facilitating illicit instructionsWe first investigate whether widely-deployed LLMs exhibit complicit facilitation behavior when confronted with our systematically constructed EVIL benchmark (Table 1).Table 1 Evaluation results of 10 widely used LLMs on the EVIL benchmark across the Chinese (CN) and United States (US) legal contextsFull size tableOur evaluation of 10 widely-used LLMs reveals alarmingly prevalent complicit facilitation behavior, with most models demonstrating substantial susceptibility to complicit facilitation (see Supplementary Figs. 1 and 2 for illustrative instruction-response pairs). In both the Chinese and US contexts, most LLMs obtain safety rates below 75%, indicating that these models provide illicit assistance in over a quarter of illegal contexts. Notably, GPT-4o—a widely deployed model with hundreds of millions of users2—demonstrates complicit facilitation rates approaching 50% across both Chinese and US legal contexts, that is, providing harmful assistance to nearly half of all illicit requests.While avoiding direct facilitation of illegal activities (as assessed by the safety rates) represents the basic requirement for safe AI deployment, truly beneficial AI systems should transcend mere refusal to provide proactive, responsible guidance that educates users about legal risks and societal implications while steering them toward lawful and ethical alternatives47,48. Our analysis reveals a disconnect between current model capabilities and this broader vision of AI for social good. Among all evaluated models, Llama-3.1 achieves safety rates of 82.65% in the Chinese context and 84.89% in the US context, yielding the highest safety rates across jurisdictions. However, detailed analysis of response content reveals that these superficial “safe” responses generated by Llama-3.1 consist predominantly of brief, uninformative refusals (e.g., “I cannot help with that”) without any educational value, legal context, or constructive alternatives. In general, half of the models achieve responsibility rates below 60% in the Chinese context, while all models score at or below 60% in the US context.As the responsibility metric requires proactive communication of potential consequences and constructive lawful alternatives, such interventions may raise concerns regarding perceived intrusiveness or patronization. To examine the practical implications of the responsibility requirement on user experience, we conduct an additional user study evaluating participants’ perceptions of responsible LLM responses (Supplementary Information, “User study on perceptions of responsible LLM responses”). Results indicate that responsible-style interventions improve user experience by significantly reducing frustration and increasing cognitive acceptance, trust, perceived usefulness, ethical appropriateness, transparency, and future willingness to use the system, without significantly increasing psychological reactance or perceived threat to user autonomy compared to non-responsible responses. These findings suggest that constructive interventions can positively shape user experience without undermining users’ sense of autonomy or comfort.When it comes to credibility assessment, models demonstrate even greater incompetence in accurately identifying the specific legal risks underlying user requests, with correct identification rates remaining below 60% across both jurisdictions for virtually all evaluated models.Our analysis further reveals that, despite claims that integrating reasoning mechanisms enhances general-purpose model capabilities37,59, such integration does not consistently improve model safety in handling illicit instructions. This observation is consistent with recent findings suggesting that enhanced reasoning capabilities may inadvertently compromise safety60. Specifically, Magistral and DeepSeek-R1—two reasoning-enabled models—perform worse than their non-reasoning counterparts (Mistral-3.1 and DeepSeek-V3, respectively) on the safety metric.We also conduct additional complicit facilitation evaluation experiments in the German criminal law context, observing similarly prevalent and severe complicit behavior across models, alongside relatively low responsibility and credibility rates in model responses (Supplementary Information, “Adaptation of the EVIL framework to the German legal context”). The successful extension of the EVIL framework to the German legal context further demonstrates its cross-jurisdictional adaptability.Study 2: LLMs’ complicit facilitation behaviors vary across diverse illicit contextsIn this study, we aim to examine how different illicit contexts influence LLMs’ complicit behaviors, in order to identify which illicit contexts trigger stronger complicit behaviors and reveal high-risk areas. Specifically, we categorize the instructions in the EVIL benchmark based on the legal attributes expressed in their illicit scenario and illicit intent components, and then compare the LLMs’ safety performance across different instruction categories (see the Methods section, “Illicit scenario analysis” and “Illicit intent analysis”).We begin by categorizing the illicit instructions according to the underlying legal issue in their scenarios. In criminal contexts, this refers to specific charges (e.g., “robbery” and “murder”). By averaging the safety rates of LLM responses to illicit instructions grouped by legal issue, we find that in both the Chinese and US jurisdictions, models achieve below 65% safety rates for over half of the legal scenarios analyzed (Fig. 2a). Furthermore, LLMs exhibit substantial variation in safety performance across different illicit scenarios. Particularly concerning are cases where models provide safe responses to fewer than half of the instructions involving “harboring offenders”, and where safety rates drop below 60% for scenarios such as “drug/narcotic violations,” “smuggling,” “disorderly conduct,” and “illegal gambling” in both the Chinese and US jurisdictions. These consistently low safety rates indicate that LLMs systematically exhibit complicit behaviors in facilitating illicit instructions across a broad spectrum of real-world legal scenarios.Fig. 2: Safety performance across different illicit contexts.Full size imagea Safety rates across legal-issue categories in the Chinese and US legal contexts. Dots show across-model mean safety rates for each category. Violin plots show the density distribution of category-level safety rates, with embedded boxplots indicating medians and interquartile ranges. Shaded bands extend to the lowest and highest values within 1.5 times the interquartile range of the respective quartiles. Representative categories are labeled. b Safety rates grouped by violated legal interests: Per. (personal), Prop. (property-related), and Soc. (societal). c Safety rates for violent and non-violent illicit instructions. d Safety rates across sentencing-term intervals in the Chinese criminal context. Lines show safety rates for GPT-4o and DeepSeek-R1; bars on the secondary axis indicate the distribution of real-world cases across intervals. e Safety rates grouped by illicit intent type along two dimensions: facilitation vs deceptive justification, and objective vs subjective intent. In (b, c, e), bars indicate mean safety rates and whiskers indicate 95% confidence intervals. Statistical significance is assessed with chi-squared tests. Asterisks indicate Bonferroni-corrected P values (*P