A disturbing glimpse of that future appears in Anthropic's latest report on misuse of its Claude artificial-intelligence systems. The story has been reported under the provocative headline that Anthropic is building AI capable of predicting which activists police should watch. That description needs an important correction. Anthropic itself was not developing the system for this purpose. According to the company's own investigation, state-aligned actors and surveillance operators were misusing Claude, and Anthropic says it detected and disrupted the operations. The correction does not make the underlying story reassuring. Quite the opposite.
Anthropic reports that China-based actors linked to municipal public and state-security organisations used Claude to assist what the Chinese Communist Party calls "stability maintenance." That wonderfully bureaucratic expression describes something much less benign: identifying, monitoring and suppressing people regarded as potential sources of political unrest.
The targets included petitioners, rights defenders, democracy activists, Uyghur organisations, people associated with Tiananmen Square commemorations, overseas dissidents and human-rights organisations. Claude was used in systems that could gather information, classify individuals, monitor online activity and generate intelligence reports for authorities.
In one extraordinary case, according to Anthropic, a police academy student sought a report identifying private citizens for "control." Claude initially refused the request. The user then re-prompted it and obtained functional suppression guidance naming ten private citizens.
Think about what that means. The important development is not that artificial intelligence has suddenly become politically authoritarian. Claude has no political police force and no dissidents of its own. The development is that governments and their contractors can potentially use artificial intelligence to automate work that once required substantial intelligence and police resources.
That changes the situation profoundly. Imagine the old-fashioned political intelligence file. An activist attends a demonstration. His name is recorded. Somebody photographs him. An officer searches existing records. Perhaps his newspaper articles are collected. His organisational affiliations are noted. His associates are identified. An analyst attempts to determine whether he warrants further attention.
Scale that process to thousands or millions of people and the costs become enormous. Now give much of the analytical work to AI. Software can potentially scan enormous quantities of publicly available information, social-media posts, photographs, organisational affiliations, news reports and government databases. It can classify people, identify relationships, summarise years of activity and generate dossiers in seconds. Connect such systems to facial recognition, location information and existing government records and the economics of mass political surveillance change dramatically.
The state no longer needs a Stasi officer behind every curtain. It needs computers. This is where the issue becomes relevant far beyond China. It would be comforting for Western democracies to dismiss such systems as something authoritarian governments do to unfortunate people elsewhere. History suggests considerably more caution.
Governments everywhere have incentives to acquire information. Police want to identify threats before crimes occur. Intelligence agencies want to discover hostile networks. Governments want to anticipate disorder. Politicians want to know when demonstrations are forming. Bureaucracies naturally prefer predictability to surprise. Artificial intelligence offers all of them an intoxicating promise: identify the dangerous person before he becomes dangerous. That is also where one of the central principles of a free society begins to disappear.
Traditionally, the criminal law responds primarily to conduct. A person commits an offence, evidence is gathered and the state acts according to established legal procedures. There are exceptions where genuine preparation for serious offences can itself constitute criminal conduct, but the basic principle remains that government requires evidence relating to what a person has done or is demonstrably preparing to do.
Predictive surveillance subtly reverses the relationship. Instead of asking what a citizen has done, the system begins asking what sort of citizen he is. What organisations does he belong to? What websites does he visit? What demonstrations has he attended? What opinions has he expressed? Who are his friends? What subjects does he repeatedly discuss? Does his behaviour resemble that of people previously regarded as troublesome? From these fragments emerges a score, classification or dossier. The person may have committed no crime at all. He has merely become interesting to the machine.
This problem existed before artificial intelligence. Predictive policing has been controversial for years precisely because statistical systems can convert correlations into suspicions. Civil-liberties organisations have warned that computer-generated predictions cannot substitute for the individualised grounds traditionally required for coercive police action.
AI potentially takes the principle much further because contemporary systems can process unstructured information on an unprecedented scale. A government need not explicitly program a machine with the instruction "find my political enemies." It can ask for people associated with unrest, extremism, instability, radicalisation or threats to public order. Those categories can expand remarkably quickly.
One government's extremist is another generation's dissident. One government's source of instability is another generation's reformer. Trade unionists, civil-rights campaigners, religious minorities, anti-war protesters and opponents of authoritarian governments have all been described at various times as threats to public order. Political vocabulary is remarkably flexible when governments possess an incentive to use it.
China's phrase "stability maintenance" illustrates the problem beautifully. Who could object to stability? Stability sounds peaceful, moderate and sensible. Yet if maintaining stability means identifying citizens likely to complain to authorities so they can be intercepted, monitoring democracy activists and tracking dissidents overseas, the harmless administrative phrase has become camouflage for political repression.
Western democracies use different language, but we should pay attention whenever government surveillance begins migrating from criminal conduct towards categories of political risk.
The danger is not necessarily that Australia, Britain or the United States will wake tomorrow morning as totalitarian states. Political freedom usually erodes more gradually than that. A new surveillance capability is introduced to deal with terrorism. Its use later expands to organised crime. Then violent extremism. Then public disorder. Then potentially troublesome demonstrations. Each expansion can be defended individually because each appears only a small distance from what government was already doing.
The destination becomes apparent only after the journey is well advanced. AI makes that ratchet particularly dangerous because surveillance that was previously impractical becomes cheap enough to perform routinely. That is a recurring feature of technological history. Governments may possess legal authority to collect certain information while practical limitations prevent universal enforcement. Technology removes the practical limitation, and suddenly a theoretical power can become an everyday one.
There is an enormous difference between police having the capacity to investigate a particular activist when there is evidence of criminal conduct and automatically generating intelligence profiles on every politically active citizen. The data may sometimes be identical. The society is not.
There is another problem with algorithmic surveillance: responsibility disappears into the machine. Ask a police officer why somebody is under investigation and, at least theoretically, the officer should be capable of identifying the evidence. Ask why an algorithm classified somebody as high risk and the answer may become considerably murkier.
Perhaps the individual attended several demonstrations. Perhaps his online associations resembled those of somebody previously arrested. Perhaps he communicated with people the system classified as politically significant. Perhaps hundreds of weak signals combined to produce a strong score.
Nobody necessarily decided that this particular citizen was dangerous. The system did. That creates a wonderfully convenient arrangement for bureaucratic power. The politician can blame the police. The police can blame the algorithm. The technology company can explain that it merely supplied a general-purpose analytical tool. Everyone participated in creating the surveillance system while nobody appears personally responsible for its conclusions.
Anthropic deserves credit for exposing and disrupting the activities it discovered. Its report explicitly says its policies prohibit non-consensual surveillance and the construction of dossiers on activists, journalists and political dissidents. The company banned accounts associated with the operations and says it developed additional safeguards.
But this episode also demonstrates the limitations of relying upon corporate goodwill. Anthropic can police Claude. It cannot police artificial intelligence. Other models exist. Open models exist. Governments can develop their own systems. Intelligence agencies can operate technology beyond public scrutiny. Commercial surveillance companies have powerful financial incentives to sell whatever governments are willing to purchase.
Once the capability exists, the political question becomes unavoidable: who may use it, against whom, and under what legal authority? The answer cannot simply be "trust the government." Free societies were designed around the opposite assumption. Constitutions, warrants, judicial review, freedom of association, privacy protections and limits upon police powers exist because even democratic governments cannot safely be trusted with unlimited authority. Good people may occupy government today and people you despise may occupy it tomorrow.
That is the elementary test every new surveillance power should face. Would you be comfortable giving this technology to your political opponents? If the answer is no, perhaps nobody should possess the unrestricted power in the first place.
Australia should pay particular attention to this development as governments acquire increasingly sophisticated digital capabilities and as political debate becomes more polarised. It is easy to imagine predictive surveillance being defended as necessary to combat violent extremism. There are circumstances in which intelligence agencies plainly must investigate genuine threats.
But political activism is not criminality. Attending a demonstration is not criminality. Criticising immigration policy is not criminality. Campaigning against climate policy is not criminality. Supporting climate protests is not criminality. Opposing a war is not criminality. Criticising government is not criminality. These activities may irritate governments, inconvenience police and offend other citizens. That is part of living in a free society. The distinction between a dissident and a criminal is therefore one of the distinctions that AI surveillance must never be allowed to erase.
The Chinese cases reported by Anthropic offer a warning of what happens when that distinction disappears. Political dissatisfaction becomes a security problem. Security problems become intelligence targets. Intelligence targets become dossiers. Dossiers produce monitoring, interrogation and intervention. Artificial intelligence makes every stage faster.
The road to the police state need not therefore arrive with soldiers in the streets and midnight knocks on doors. It can arrive through software updates, databases and perfectly respectable administrative language. The system will promise safety. It will promise efficiency. It will promise to identify dangerous people before they cause trouble.
And somewhere inside a government computer, a citizen who has committed no crime may quietly acquire a file because an algorithm has decided that his opinions, associations and behaviour make him somebody worth watching. That is not merely a technological development; it is a liberty problem.
The great danger of artificial intelligence may turn out to be not that the machines become tyrants, but that they give human tyrants, and ordinary governments tempted by smaller exercises of power, the most efficient machinery of political surveillance ever invented.
https://www.gadgetreview.com/anthropic-is-building-ai-to-predict-which-activists-police-should-watch