A Chinese study warns that introducing artificial intelligence (AI) into the nation’s existing social credit system could be a mistake. Published in the Journal of Sichuan Normal University, the study warns that this could turn the rating system into something more akin to a “large-scale surveillance tool.” Interestingly, the research was funded by a Chinese government-backed project examining social governance and institutional innovation. At present, China’s social credit score system is a collection of different program.It combines things like a financial credit system similar to credit ratings or reports in the West. It also includes regulatory blacklists that list individuals or companies who have fallen foul of the law for whatever reason. The system also includes local behavioral experiments that can reward or punish people based on set criteria.It also includes a set of government accountability systems to monitor compliance and conduct among public officials. These programs are not connected into a single national score for each citizen, but taken together can hamper a person’s access to things like loans through automated decisions.Introducing AI into the mix, the study argues, could turn this into the Orwellian system, which many Western critics say it already is.Social credit score plus AI, a good idea?According to a report, the AI-driven system could analyze your activities, financial history, relationships, and other available data at an enormous scale. From that, it could determine that you resemble people who might cause problems.Under this scenario, a person may not have necessarily done anything wrong, but the system could treat them differently anyway. This would shift the system from a primarily reactive to a more predictive one.Think “Minority Report” but on steroids. The researcher, Lin Jianxing, warns that this potential transition could shift from assessing actual behavior to profiling individuals and making administrative decisions based on algorithmically predicted risk.All this happens without the knowledge of individual in question, while also being “punished” with reduced access or increased scrutiny without actually officially breaking the law.What’s more, this is not purely theoretical either. A 2025 Stanford Study examined the existing social credit system to see what was going on.The researchers examined one Chinese city’s social credit system, in which adults started with 1,000 points. Their scores could then change depending on compliance with 389 rules governing rewards and penalties.A disaster waiting to happenThese included political behavior, such as petitioning government authorities. It also found that individuals receiving the lowest classification could face police monitoring.Rural residents appeared to be unfairly handicapped as they had fewer opportunities to earn rewards compared to people living in cities. Overall, it found that while a nationwide “Big Brother” type system doesn’t exist, some local systems did appear to penalize people beyond straightforward criminality.Introducing AI into the mix could make everyone’s worries about it a reality, however. It is important to note that there is currently no evidence of plans to introduce a unified, nationwide AI-powered social scoring system. China is, however, already exploring AI’s use in public administration.Rather, the study merely serves as a warning that should this happen in the future, the outcome would not be desirable. On the other hand, proponents might argue that reliably identifying individuals at risk of committing serious crimes could benefit society.It could also be used to identify people most in need of help. If AI were to be introduced, the study argues, it must be in such a way that it doesn’t punish the innocent while also not being overly intrusive.An impossible task? We’ll let you decide.