▲ 113 points
back
51 comments
Obligatory reference to Black Mirror's Nosedive episode: https://en.wikipedia.org/wiki/Nosedive
Whilst real world events have caused many people to focus on the social ranking aspect of the story, when I watched it, I thought it was more concerned with privilege (and the loss thereof).
The main character actually worsens her fate by acting entitled and being self-centered. Had she maintained her higher status, she both wouldn't have been in situations of pain in the first place, nor found it necessary to make a (here, social) effort to get out of them. In real life, being considerate to people serving you pays off in the short term, and in aggregate makes for a better society to live in (it is generally accepted in Western society that insulting your waiter is really rude and antisocial, for example).
Interestingly, both people with low rankings (brother, truck driver) appear to have bad manners (because they are direct) but are nice people deep down; the point being the rating system is imperfectly implemented or optimising for the wrong thing.
The parallel with the Chinese rating breaks down precisely because the show rating is focused on this shallow idea of visible manners whilst the Chinese system appears to correct fundamentally inconsiderate, widespread behaviour (like stealing a shared bike), just as credit scoring in the West was originally a means of increasing general welfare (by lowering rates for most at the expense of the few who game the system).
The real parallel between the article and Black Mirror is this: the author, who is privileged (he already lives in a considerate society where everybody cooperates and the rule of law is a given) cannot understand the need for Sesame; upon losing that privilege (by moving to China with a blank score) and having to earn it back, he is upset just as the heroine of the episode was ("but I'm an honest guy, why should I pay a deposit to rent a bike?").
You're claiming the Chinese system optimises the right things, which might even be true for now. But my experience is that any system optimising some metrics eventually drifts from it's intended real goal as people and systems learn to game those metrics.
In the Black Mirror episode, there just isn't room for being genuine and maintaining a nice lifestyle because of social pressure. But it probably worked a lot better when the system just started.
As alarming as this is (and it is alarming) at least it's explicit.
Another worrisome case is of blacklists/rankings that you can't see. No-fly lists that no one knows why you are on. Secret watch lists. Conspiracies against people with opposing political views with no clear way to know what views may have triggered this and no path to straighten a misunderstanding up.
I oppose oppression in all forms but if we are going to have it, explicit is better than implicit. (And flat is better than nested:).
Note that if you are on the bad credit list, you aren’t allowed to fly or take the train. I guess this is so you can’t flee from your debts.
The article discusses two things:
> Sesame Credit, controlled by a private organization (Alipay)
> The government's List of Dishonest People, a blacklist of people who haven't paid fines.
(The article conflates them because the Sesame Credit scores are downgraded based on whether the person has unpaid fines. However, Sesame Credit doesn't affect the government list)
The "credit" list has no impact on trains/flying; nonpaying of government fines does. This isn't too dissimilar from other countries, where failing to pay government-issued fines results in disciplinary measures.
https://www.techinasia.com/china-citizens-scores-barred-from...
They call it a “social credit system” in china (the one that can get you barred from flights), besides not paying your fines you can get on the list if you happen to be a unharmonious dissident. It’s no wonder it is confused with the sesame credit system, since they are both credit systems that run in china. Here is a state-owned press link if you doubt tech Asia:
You might be able to take the train or fly, but only with the lowest class of seats. There might be something punitive in this, but another aspect of it is that you don't want to sell expensive things to people with bad credit.
There are myriad ways people try to get out of paying for things -- returning them, cancelling last minute, filing complaints -- that leave the merchant holding the bag even after a sale is seemingly complete.
Extra Credits made a video about Sesame Credit a couple of years ago. It's 7.5 minutes long, and well worth the watch:
https://www.youtube.com/watch?v=lHcTKWiZ8sI
Being a game design channel, they discuss Sesame Credit as obedience to the party, gamified.
Couple of questions come to find:
a. How is this data accessed? Can anyone in my network access this? Because even if we draw a parallel to credit scores, not everyone can access my credit scores.
b. What kind of behavior is this encouraging? Most credit scores track bigger picture items - credit card payment, loan payments etc. But, suppose I can see my credit score go up and down then I might end up trying to avoid a specific behavior.
c. How does the scoring work? If I buy say from a company known to sell VPN services, do I lose points?
This smells like the Cultural Revolution. Scary.
I don't see any similarity.
The article points this out specifically:
>"To understand the allure that social engineering holds for Chinese leaders, you have to go back decades, to long before apps and big data. In the years after the 1949 Communist Revolution, the government assigned everyone to local work units, which became the locus of surveillance and control. Individuals spied on their neighbors while also doing everything they could to avoid black marks on their own dang’an, or government files. But maintaining the system required massive state effort and oversight. As economic reforms in the 1980s led millions of people to leave their villages and migrate to cities, the work unit system fell apart. Migration also had a secondary effect: Cities filled up with strangers and pianzi."
I don't see any references to Cultural Revolution in that quote?
I don't know your background, so I'm not sure if you're being coy or genuinely don't know. But the gist is: mass persecution, made possible by citizens spying on each other. OK fair, nobody has been singled out or publicly humiliated (yet), but it's the same setup. Forgive me if the Cultural Revolution means something different to you; I'm going based on the description given to me by one of my close friends, whose parents are from China and experienced it.
Appreciate the benefit of doubt, I honestly didn't see a connection and thought you might have conflated Cultural Revolution with Communist Revolution.
I think that I can understand where your interpretation came from, but wouldn't really draw that connection here
Neighbors and friends and family "whistleblowing" each other to the government certainly was an aspect of Cultural Revolution, but it never really struck me as a distinctive/defining feature. The same things happened before Cultural Revolution as well (note that the time period pointed out in the article is 1949, which is the year Communist China was established and revolutionary zeal was high, but 17 years before the Cultural Revolution began), and the same tattling can also be found during McCarthy-era America, and briefly in post-9-11 America ("If You See Something, Say Something")
The more defining features of the Cultural Revolution (at least to Chinese) is the persecution specifically of educated people (professors being harassed and shamed into committing suicide. "the more educated, the more reactionary"), the shunning of education (My grandpa had to shut the doors and teach his kids in secret at home, quite a risky move if anyone found out), the scheduled public demonstrations of loyalty to the party via everyone getting together to dance the loyalty dance every morning, and a dogmatic adherence to party lines, no rational or logic debate allowed (you will be shunned and reported). Don't really see those features in this policy.
How much of this is happening in the US already, but without our knowledge -- unless of course someone steals it and we read about it here. It all sounds so convenient, yet at the same time I see where it will eventually wind up, as a new kind of slavery.
>How much of this is happening in the US already, but without our knowledge
Probably a lot. My cousin (Nth generation rich white dude) took a class (at an ivy league school) on the more technical side of terrorism where the professor outright told them that the research they would need to do to get a good grade they would result in them getting a lot more "random" searches at airports and the like. He said other students confirmed that statement (I don't think he had had occasion to travel via air between taking the class and our conversation).
Interesting price to pay for taking a class, is it part of any particular degree?
On the other hand I guess you really get to learn how your government watches you.
Definitely not part of his degree. I think they had a professor who's specialty was that sort of thing so it was a "sought after" elective.
Well, the new version of slavery was widespread credit - a continuum rather than discrete, with every interaction being a purported choice of assent (easy) or rejection (hard). The big-picture trend is clear, yet we mostly cling to the self-lie that we've voluntarily accepted this system.
The commercial surveillance enabling this has been with us a long time, long enough that people working in that industry casually refer to their databases' keys as a person's identity! The recent innovations in commercial surveillance just make it harder to run away and start fresh, while enabling more efficient extraction through fine-grained price discrimination.
More like indentured servitude; the “new” slavery is in prison labor.
This is similar to the "like graph" in FB.
FB definitively know a lot folks' "political leaning" base everyone's "likes" in global scale and has profit from the info in the last elections.
I hope what China does give people a hint on how they were suffering for so many years already, and can start doing something to prevent their own government one day make those suffering mandatory...
There's a strong implication that Sesame Credit and the Social Credit System idea follow the example of FICO scores. Has anybody in the Chinese government explicitly stated this?
Why would the Chinese government need to explicitly state this? The parallels are clear. The inventors of Sesame Credit and the Social Credit System have access to the same history we do about the FICO system.
The fact that they're doing this and the fact they're actively trying to export their political ideology via their Confucius institutes, etc. terrifies me. As China's influence grows, I worry they may make implementing similar systems of condition of doing business with them and that developing countries (in Africa especially) who are becoming increasingly reliant upon China for development funding, will lack the power to push back against this dystopian bullshit.
That political ideology is not particular to China. Suppression of political consent and overall increasing control are natural tendencies of government. People can be put on a list in the US for example and exempted from flying anywhere.
Did you mean 'dissent'?
Right. Stupid autocorrect.
We have private market ranking systems like your credit score.
Quantifying karma is a good idea, if implemented correctly and within limitations of accuracy.
> Quantifying karma is a good idea
Why? Many people hold many opinions about me, almost all of which are wrong. I don't see the value in a global database of opinions about me which I can't control and which will affect me in ways I can't predict.
If you can’t understand the utility of a social karma system then you should probably ignore all airbnb reviews, amazon, and totten tomato reviews when making a decision, and give them all a clean slate.
It’s hard to argue those systems are not useful if implemented properly. Yes it can lead to prejudice if done improperly, and some prejudice is unavoidable (the whole point of the review is to do a lot prejudging for you to save time) to save on time and effort.
> you should probably ignore all airbnb reviews, amazon, and totten tomato reviews when making a decision
Those are reviews of businesses and products, not individuals. It's not equivalent.
I'd find that helpful. If a lot of people have negative opinion about you, you have no place in my life, sorry. I don't want to deal with some society outsider
I believe that is called prejudice. Prejudiced people frighten me, their moral considerations so unthorough, and this makes me think I wouldn’t want to deal with someone like you.
But I’d give you a chance first.
It's not prejudice. And big surprise, you're exactly like me, just unwilling to vocalize it.
The fact that people are rushing to downvote me for voicing a contrary opinion (and emboldening others to disagree and downvoting me too) is the exact embodiment of what I just described. Like it or not, human beings don't like outsiders and people that others dislike
This post gave me a negative opinion of you. See how this works?
Exactly. Thanks for proving my point.
Your point seems to have changed. The point of those replying to you is that snap judgements happen, and rarely are they correct. Your point was that we should be putting them into a global database for others to make second-hand judgements from bad data.
Growing up in a racist area and taking a public stand against racism would give one a very poor score, even after leaving.
Ah if only there were more people like you in 1930s Germany.
"quantify" and "karma" don't belong in the same sentence.
Yep. That's like one of the key religious/spiritual beliefs. Humans are incapable of good judgment due to lack of omniscience. Nobody can make a proper moral judgment of any individual since you don't have perfect knowledge of all their past and future deeds and circumstances. Even Christianity subscribes to this idea "vengeance is mine". Ballancing of universal scales is best left to a mystical omnipresent entity that has access to every bit of information in existence.
That’s a pretty understated ‘if’
In case people aren't making the connection, this ties in with the net neutrality issue in the US.
The core thing that they are attempting to do is to declare that the internet is no longer a river of information that they just pass along. Making this change is really important if you want more control over what people say and do on the internet.
Without net neutrality, it is easier to start funnelling more and more people through monopolies that cooperate with the government. The Chinese apps are a good example of that.
Make no mistake, the United States is no longer the bastion of freedom you believe it to be. And many in government would _love_ to be able to more effectively stifle political dissent and move to a one party system.
This hyperbole about lack of Title II classification being the end of democracy, capitalism, the intetnet, or the United States is getting really tiresome.
I was quite aware that some people would be tired and consider it hyperbole when I wrote that. Unfortunately these trends have been ongoing for quite some time. This apathy and dismissal of the net neutrality incident as being an isolated independent problem worries me. If some people become tired from reading my comments, that's too bad. But if you are going to dismiss them anyway, maybe you should not even bother reading them.