Blog post 2: China as a technological dystopia

A critical review of discourse regarding Chinese sesame credit score

By Yves Wienecke
January 20, 2019

Dystopian society

With emerging advances in AI and surveillance technology, attempts towards this new technology’s integration in society rouse debate amongst those who are not particularly fond of the the threat of misuse and corruption. In particular, China is receiving harsh criticism over the implementation of a social credit system along with the usage of facial recognition software in CCTV security cameras [3, 6, 9]. News sources describe China as a totalitaian surveilance state dystopia, not unlike the dystopia crafted by American author George Orwell in his book 1984[4].

In a similar fashion as the negative feelings toward government surveillance, critics of China show dissatisfaction with the usage of a social credit system for ranking individuals based on what is deemed as ‘good’ and ‘bad’ behavior. This method of behavior control is directly compared to a similar system seen in the episode ‘NoseDive’ from the British television show ‘Black Mirror.’ In this episode, citizens are perscribed a number rating that is affected by everyday interactions: people are able to upvote or downvote each other to affect their social credit score [3].

The episode follows a woman who is caught under a line of unfortunate circumstances that lead her to drop from a good social score to a pitiful one, all in the matter of a few hours. As skeptics of the Chinese social credit system are keen to point out, this same fate is bound to happen to normally upstanding, law abiding Chinese citizens who happen to misbehave due to a bad day or forces out of their control.

[4] Black Mirror, ‘Nosedive’ episode showcasing a concern for social credit systems.

Perhaps the events that occur in ‘Nosedive’ seem exaggerated in the context of real life, but news articles and western media sources insist that it is a very possible occurance in the society that China is creating. However, while there is a lot of discussion circulating the possible impacts of a social credit system on society, there is little talk about exactly how the system works and the Chinese perspective on the matter.

Impression of China from the perspective of Western media

To better understand why news articles are casting China in a negative light, it’s important to understand how China is commonly personified in Western media. Western news outlets are guilty of representing China in a negative manner: sometimes referring to Chinese practices as old [9] or fundametally flawed in some manner or another - a stark parallel to the new, democratic societies of Europe and America [8]. In the past, news articles presented either inaccurate or incomplete information regarding Chinese current events, which spreads incorrect assumptions about life in the red country. For example, in 2014, news sites included a picture of Tiananmen Square in Beijing that commented on the pollution of the city, stating that the pollution caused officials to display sunrises on large billboard signs in answer to the smog.

[2] Picture in question misused by news sites.

Although the picture in question is alarming and meant to shock viewers, the story associated with it is completely false [2, 5]. In reality, the billboard is a running ad for a travel agency, completely defying the allegations of Chinese officials using a display of a sunset to combat pollution. After some backlack for spreading false information, news outlets have slightly modified their articles, but have not changed their clickbait headlines, which seek to shock and appal readers [2, 5]. Although pollution is a real problem in China, misinformation is a western trend of journalism over Chinese topics that uses shock and fear to garner negative feelings towards China, rather than addressing Chinese issues accurately and seriously.

In more recent news, western media is quick to jump on the offensive side against China by mentioning an incident where a Chinese high school student was denied entry to a college despite his strong performance on all entrace exams due to the poor social credit score of his father [8]. In another case, citizens may be prevented from purchasing airplane, train tickets, or booking luxury hotels as a result of their bad social credit score, which can be accompanied with being on a government black list of untrustworthy individuals[8].

What these outlets fail to mention is the Chinse public’s reaction and the reasons for which a person can accrue negative social credit. In terms of the student who was denied entry to college, the public was outraged and strongly opposed how the college treated the student, arguing that, if a son is to be punished for his father’s actions, then similar punishment should follow corrupt politicians and their children [11].

Interesting to see the huge difference in which the Social Credit System is illustrated in foreign vs Chinese media. Check out these 10 state media cartoons on social credit here: https://t.co/lIyMhaMnFb #socialcredit #China pic.twitter.com/565DZqvD6L

— Manya Koetse (@manyapan) July 20, 2018

[11] A stark contrast between how the west and China view the social credit system.

Citizens placed on black lists are generally guilty of public offenses such as stealing, failure to pay fines, or violating traffic laws. The reasoning for barring black listed citizens from purchasing tickets or renting hotel rooms is as follows:, if the offender does not have enough money to pay off their debts, then they should not be able to purchase unnecessary luxuries. There are a number of blacklists for various services - from bike sharing companies to banks to the government, individuals can be listed on one or many of these lists. What I find to be very concerning is that ‘bad behavior’ worthy of being placed on a governmental black list is up to the discretion of the government only [11]. This risks punishment for political dissent or holding unpopular opinions.
Again, outlets fail to remind readers of similar American systems, where “courts and law enforcement officers can seize assets or skim off wages to force compliance with a civil judgement,” and “millions of Americans had their driver’s license revoked due to unpaid traffic fines” [11]. While skepticism is healthy in the debate over China’s social credit system, it is wrong to assume that western systems are not guilty of similar practices. Truly, China’s social credit should “prompt more public debate on aspects of concern in our own systems” [11].

What actually are the social credit systems?

Let’s take a closer look at the Chinese social credit system. In actuality, the system is composed of two different, separate credit systems: the government credit system, which is set to be used in 2020; and another system owned by the Chinese company ‘Alibaba’ [7]. This second system, dubbed ‘sesame credit,’ is moderated by Alibaba - a popular all-in-one mobile application for Chinese citizens that functions like Amazon, Uber, and paypal. Sesame credit assigns a numerical score to citizens, which can be impacted from certain purchases (video games, alcohol) and association with others who have either high or low credit scores. This opt-in system displays the user’s credit score upon purchasing something on the Alibaba app, and can have some benefits for those who have high scores. Scores range from 350 and 950, and high scoring users get benefits like having access to exclusive discouts and or being waived from paying a deposit for library books or renting bikes [7].

[7] A score of 385 is described to be in the low range, while 731 being in the good range.

These credit scores are well received by the public, with citizens going so far as to display their credit score on their dating profile to attract potential partners [11]. To udnerstand why this is so, let’s take a look at the cultural context of China. China has a high level of corruption and crime - rules and safety regulations are blatently broken on the streets. Additionally is not uncommon, especially of the elder generation, to observe a lack of social manners in public - people littering, jay walking, and spitting on sidewalks. Due to this, some applaud the efforts of Alibaba and the government. The possible negative effects connected to misbehavior and crime are enough to convince citizens to observe the laws and really change their behavior.

Should China be scared?

China has been technologically progressing at a rate quicker than it can accomodate. The technology that we are seeing being developed and integrated into Chinese society are upgrades to better and more efficiently enforce already existing laws and regulation. I believe that concern of using technology to craft a social credit score on a person, and using that score to deny acces to certain resources is very dangerous to democratic society and prone to corruption and abuse from those who make the decisions on what increases or decreases one’s score. Perhaps governments should consider popular vote for making these decisions, such that the rules can be decided by the public, rather than a small circle of political elites.

With current discussion over the Chinese credit system, criticisms are missing a huge point: their fears of oppression are already being seen in western China, embodied through the cultural genocide of Uighurs in the Xinjiang province. In cities, social credit systems may do very well, but in rural areas and areas housing large foreign populations (aka non-Han Chinese citizens), officials and law enforcement intimidate residents to do certain behaviors that are then later deemed to be bad or suspicious, landing otherwise innocent citizens on the government’s blacklist of untrustworthy individuals [1].

What started as an effort to quell terrorist attacks in the oil-rich city turned into an indiscriminate round-up of any and all Uighur citizens - regarless of whether or not they are a practicing muslim. In this case, facial recognition and the social credit system is used to identify Uighurs in large crowds and place them under close surveillance, even sending them to re-education camps where they are forced to abandon their culture and religion to adopt a Hand Chinese lifestyle [1].

While the Chinese social credit system has a clear benefit in the potential for encouraging positive behavior and discouraging negative behavior, it is unclear as to what exacly ‘is’ good or bad behavior, and who makes these decisions. In the Chinese cultural context, a social credit system is a natural progression of law enforcement and actually celebrated as an effective means of this. Nevertheless, many western media outlets take this system out of its cultural context, trying to compare it to their own cultural contexts. In doing this, their criticisms fall short of the actual concerning matters occuring in other parts of China. In future discussions regarding technology and China’s social credit system, I believe that inclusion of a Chinese perspective is integral for ensuring that the cultural context of China is considered, and so that articles do not become echo chambers of inaccurate shock-enducing misinformation.

---

Sources

[1] Apps, Peter. “Commentary: China’s Muslim minority faces a modern Orwellian nightmare.” Reuters, 14 May 2018, https://www.reuters.com/article/us-apps-china-commentary/commentary-in-chinas-northwest-a-hidden-high-tech-orwellian-nightmare-idUSKCN1IF27B.

[2] Bischoff, Paul. “No, Beijing residents are NOT watching fake sunrises on giant TVs because of pollution.” *Tech in Asia, *19 Jan. 2014, https://www.techinasia.com/beijing-residents-watching-fake-sunrises-giant-tvs-pollution.

[3] Dockrill, Peter. “China’s Chilling ‘Social Credit System’ Is Straight Out of Dystopian Sci-Fi, And It’s Already Switched On.” Science Alert, 20 Sep. 2018, https://www.sciencealert.com/china-s-dystopian-social-credit-system-science-fiction-black-mirror-mass-surveillance-digital-dictatorship.

[4] drab2000. “Black Mirror S03E01 Airport scene.” Youtube, 17 Feb. 2017, https://www.youtube.com/watch?v=YrpK90bHO2U.

[5] Guilford, Gwynn. “Westerners are so convinced China is a dystopian hellscape they’ll share anything that confirms it.” *Quartz, *20 Jan. 2014, https://qz.com/168705/westerners-are-so-convinced-china-is-a-dystopian-hellscape-theyll-share-anything-that-confirms-it/.

[6] Jefferson, Ed. “No, China isn’t Black Mirror – social credit scores are more complex and sinister than that.” 27 Apr. 2018, New Statesman, https://www.newstatesman.com/world/asia/2018/04/no-china-isn-t-black-mirror-social-credit-scores-are-more-complex-and-sinister.

[7] Koetse, Manya. “Open Sesame: Social Credit in China as Gate to Punitive Measures and Personal Perks.” Whats on’s on Weibo, 27 May 2018, https://www.whatsonweibo.com/open-sesame-social-credit-in-china-as-gate-to-punitive-measures-and-personal-perks/.

[8] Ma, Alexandra. “China has started ranking citizens with a creepy ‘social credit’ system – here’s what you can do wrong, and the embarrassing, demeaning ways they can punish you.” Business Insider, 29 Oct. 2018, https://www.businessinsider.com/china-social-credit-system-punishments-and-rewards-explained-2018-4#3-banning-you-or-your-kids-from-the-best-schools-3.

[9] Mozur, Paul. “Inside China’s Dystopian Dreams: A.I., Shame and Lots of Cameras.” The New York Times, 8 Jul. 2018, https://www.nytimes.com/2018/07/08/business/china-surveillance-technology.html.

[10] Song, Bing. “The West may be wrong about China’s social credit system.” The Washington Post, 29 Nov. 2018, https://www.washingtonpost.com/news/theworldpost/wp/2018/11/29/social-credit/?utm_term=.fc67b6dc0b0f.

[11] Wade, Samuel. “College Rejection Threat Highlights Social Credit.” China Digital Times, 20 Jul. 2018, https://chinadigitaltimes.net/2018/07/college-rejection-threat-highlights-social-credit-blacklists/.