China is shoring up the great firewall for the AI age

China faces a problem familiar to dictatorships throughout history: how to strike a balance between growth-boosting innovation, which thrives in a free society, and the paranoia of an authoritarian state. Its leader, Xi Jinping, wants the country to become a hyper-advanced economy. His government is aggressively promoting the commercialisation of high technologies it likes, from electric vehicles to quantum computing.

At the same time, it is tightening the screws on those it disapproves of. In 2021 it regulated a booming online-tutoring industry into oblivion almost overnight, apparently out of fear that high tuition fees were making children’s education so expensive that Chinese were put off the idea of parenthood. On December 22nd the government took a wrench to the video-gaming industry, introducing rules to, among other things, limit how much players can spend on in-game purchases—and so how much developers can make. The market value of Tencent, one of China’s most innovative firms that also has a big gaming business, tumbled by 12%.

Nowhere is this tension clearer than in the hottest technology of 2023—artificial intelligence (AI). In many countries, command of AI is seen as both economically and strategically important. Politicians everywhere fret about machines going rogue or, more realistically, being harnessed by human mischief-makers. In Beijing the added worry is that the technology, which thrives on unlimited data and, at its current stage of development, in unregulated spaces, could prove subversive if not kept in check. It is therefore busily shoring up its “great firewall” for the AI age.

In 2000 Bill Clinton, then America’s president, likened China’s attempt to control the internet to “trying to nail Jell-O to the wall”. Today the jelly seems firmly in place. Western internet services, from Facebook and Google search to Netflix are unavailable to most Chinese (apart from those willing to run the risk of using illegal “virtual private networks”). On local platforms, any undesirable content is deleted, either pre-emptively by the platforms themselves, using algorithms and armies of moderators, or afterwards, as soon as it is spotted by government censors. A tech crackdown in 2020 brought China’s powerful tech giants, such Tencent and Alibaba, to heel—and closer to the government, which has been taking small stakes in the firms, and a big interest in their day-to-day operations.

The result is a digital economy that is sanitised but nevertheless thriving. Tencent’s super-app, WeChat, which combines messaging, social media, e-commerce and payments, alone generates hundreds of billions of dollars in yearly transactions. Mr Xi now hopes to pull off a similar balancing act with AI. Once again, some foreign experts predict a jelly situation. And once again, the Communist Party is building tools to prove them wrong.

The party’s efforts begin with the world’s toughest rules for Chinese equivalents of ChatGPT (which is, predictably, banned in China) and other consumer-facing “generative” AI. Since March companies have had to register with officials any algorithms that make recommendations or can influence people’s decisions. In practice, that means basically any such software aimed at consumers. In July the government issued rules requiring all AI-generated content to “uphold socialist values”—in other words, no bawdy songs, anti-party slogans or, heaven forbid, poking fun at Mr Xi. In September it published a list of 110 registered services. Only their developers and the government know all the ins and outs of the registration process and the precise criteria involved.

In October a standards committee for national information security published a list of safety guidelines requiring detailed self-assessment of the data used to train generative-AI models. One rule requires the manual testing of at least 4,000 subsets of the total training data; at least 96% of these must qualify as “acceptable”, according to a list of 31 vaguely worded safety risks. The first criterion for unacceptable content is anything that “incites subversion of state power or the overthrowing of the socialist system”. Chatbots must decline to answer 95% of queries that would elicit unacceptable content (any that do get through would, presumably, be censored ex post). They must also reject no more than 5% of harmless questions.

Anything produced by unregistered algorithms is to be blocked and its creators punished. In May a man in Gansu province was arrested after he used ChatGPT to produce text and images of a fake train crash and publish them on social media. He may have been the first Chinese to be detained for spreading AI-generated misinformation. He will not be the last.

This heavy-handed strategy has slowed the uptake of consumer-facing generative AI in the country. Ernie Bot, built by Baidu, another tech firm, was ready around the time of ChatGPT’s launch but only released nine months later, in August—aeons given how fast the technology is evolving. It is still clumsy when it comes to expressing its devotion to the party. When asked sensitive questions about Mr Xi, it dutifully censors, offering no answer and deleting the query.

Model socialists

With more work the party may be able to make AI models into not merely good communists, but fluent ones. That would obviate the need for ex post censorship, says Luciano Floridi of Yale University. Yet the authorities seem in no rush to get there. Instead, they are promoting the technology’s business applications.

In contrast to consumer AI, enterprise AI faces few constraints on development, notes Mimi Zou of the Oxford Martin School, a research institute. As Steven Rolf of the Digital Futures at Work Research Centre, a British think-tank, explains, this has the effect of channelling capital and labour away from things like consumer chatbots and towards machine learning for business. This, the government seems to be betting, will allow China to catch up and even overtake America in AI without the hassle of dealing with potentially subversive AI-generated content.

In May the southern city of Shenzhen announced it would launch a 100bn-yuan ($14bn) AI-focused investment fund, the largest of its kind in the world. A number of city governments have launched similar investment funds. A lot of this money is going to firms such as Qi An Xin, which offers generative AI that manages data-security risks for companies. The company claims that the bot can do the work of 60 security experts, 24 hours a day. Before going public in 2020, it received big investments from state companies, like many similar startups.

For this strategy to work, the enterprise-AI firms need the right sort of raw material. Consumer chatbots use AI models trained on swathes of the public internet. Corporate applications need corporate data, a lot of which is squirrelled away inside companies. So the other plank of China’s strategy is to turn corporate data into a public good. The state does not want to own the data but—as with the other factors—to control the channels through which it flows.

To that end, the government is promoting data exchanges. These are meant to let businesses trade information, packaged into standardised products, about all areas of commercial life, from activity at individual factories to sales data at individual shops. Small firms will gain access to knowledge once reserved for the tech giants. Banks and brokers will get a real-time picture of the economy.

Chinese cities began launching data exchanges about a decade ago. Now there are around 50 around China. And they are finally gaining speed. The Shanghai Data Exchange (SDE), which was launched in 2021, has started dealing in a number of new data products. In one of its first transactions, ICBC, a bank, bought information from the energy sector. This can be used to assess companies’ power consumption and, because it reflects real levels of activity, to create alternative credit profiles for companies. SDE sells satellite-derived data on steel output in China’s heartland and environmental violations by mining companies. Another product gives real-time data on doctors, nurses and hospital beds across the country in order to help medical firms make business decisions. The SDE is also experimenting with using data as collateral for loans.

At scale, the datafication of industry could deliver a significant economic boost, says Tom Nunlist of Trivium, a consultancy in Beijing. And more data may be coming to the exchanges soon. In August the central government tasked state-owned firms with thinking about how to value their data. In the past few months teams of auditors have been trying to come up with ways to do this, with a view to adding such data to companies’ balance-sheets as intangible assets. They are meant to report back by January 1st (though the deadline looks likely to be missed, given the unprecedented nature of the task).

The government’s gamble on enterprise AI is not without problems, however. The car industry is a case in point. In 2022 about 185m vehicles on Chinese roads had an internet connection, and a national plan envisages mass production of semi-autonomous cars by 2025. For that to happen, companies devising self-driving algorithms need lots of data on which to train their systems. A company called WICV is building a platform for the data that is beginning to trickle out of cars.

For now, WICV returns a car’s data to the carmaker that built it, so BYD gets data from BYDs, Nio from Nios and so on. But the plan is eventually for the data to be traded on exchanges, where it could be bought by other developers of self-driving systems. For that to happen, though, driving data must first be “desensitised”, explains Chu Wenfu, WICV’s founder, by stripping out biometric and geolocation details that could help bad actors track the movements of specific people.

The potential for such tracking spooks Chinese authorities. A big reason why they cracked down on Didi Global days after the ride-hailing firm’s initial public offering in New York in 2021 was, it later transpired, a fear that data on Didi’s 25m daily rides, including geolocation information linked to passengers, could fall into the hands of the American authorities. The Chinese government is pre-emptively mapping out vast areas of China where data collection could potentially pose a national-security threat.

Many carmakers, including Western ones such as BMW, have little choice but to team up with state-backed companies to handle driving information and ensure that local data-compliance rules are followed. Just in case, some car companies are ditching certain features, such as allowing drivers to watch live footage of inside and outside their car on their phones. Some of that footage could, after all, inadvertently capture something sensitive.

Such trade-offs between innovation and security are unlikely to be limited to cars. Other industrial and corporate data, too, will probably need desensitising before it can be traded at scale on exchanges. That will slow the development of enterprise AI, even if algorithms remain unshackled. It is a price that the party appears willing to pay for its paranoia. ■