▲ 468 points
back
498 comments
The current comments seem to say this is rings the death knell of social media and that this just leads to government censorship. I'm not so sure.
I think the ultimate problem is that social media is not unbiased — it curates what people are shown. In that role they are no longer an impartial party merely hosting content. It seems this ruling is saying that the curation being algorithmic does not absolve the companies from liability.
In a very general sense, this ruling could be seen as a form of net neutrality. Currently social media platforms favor certain content, while down weighting others. Sure, it might be at a different level than peer agreements between ISPs and websites, but it amounts to a similar phenomenon when most people interact on social media through the feed.
Honestly, I think I'd love to see what changes this ruling brings about. HN is quite literally the only social media site (loosely interpreted) I even have an account on anymore, mainly because of how truly awful all the sites have become. Maybe this will make social media more palatable again? Maybe not, but I'm inclined to see what shakes out.
I'm probably mis-understanding the implications but, IIUC, as it is, HN is moderated by dang (and others?) but still falls under 230 meaning HN is not responsible for what other users post here.
With this ruling, HN is suddenly responsibly for all posts here specifically because of the moderation. So they have 2 options.
(1) Stop the moderation so they can be safe under 230. Result, HN turns to 4chan.
(2) enforce the moderation to a much higher degree by say, requiring non-anon accounts and TOS that make each poster responsible for their own content and/or manually approve every comment.
I'm not even sure how you'd run a website with user content if you wanted to moderate that content and still avoid being liable for illegal content.
> I think the ultimate problem is that social media is not unbiased — it curates what people are shown.
This is literally the purpose of Section 230. It's Section 230 of the Communications Decency Act. The purpose was to change the law so platforms could moderate content without incurring liability, because the law was previously that doing any moderation made you liable for whatever users posted, and you don't want a world where removing/downranking spam or pornography or trolling causes you to get sued for unrelated things you didn't remove.
The rise of social media was largely predicated on the curation it provided. People, and particularly advertisers, wanted a curated environment. That was the key differentiator to the wild west of the world wide web.
The idea that curation is a problem with social media is always a head scratcher for me. The option to just directly publish to the world wide web without social media is always available, but time and again, that option is largely not chosen... this ruling could well narrow it down that being the only option.
Now, in practice, I don't think that will happen. This will raise the costs of operating social media, and those costs will be reflected in prices advertisers pay to advertise on social media. That may shrink the social media ecosystem, but what it will definitely do is raise the draw bridge over the moat around the major social media players. You're going to see less competition.
It all comes down to the assertion made by the author:
> There is no way to run a targeted ad social media company with 40% margins if you have to make sure children aren’t harmed by your product.
> social media is not unbiased ...
Media, generally, social or otherwise, is not unbiased. All media has bias. The human act of editing, selecting stories, framing those stories, authoring or retelling them... it's all biased.
I wish we would stop seeking unbiased media as some sort of ideal, and instead seek open biases -- tell me enough about yourself and where your biases lie, so I can make informed decisions.
This reasoning is not far off from the court's thinking: editing is speech. A for you page is edited, and is TikTok's own speech.
That said, I do agree with your meta point. Social media (hn not excluded) is a generally unpleasant place to be.
"Social media" is a broad brush though. I operate a Mastodon instance with a few thousand users. Our content timeline algorithm is "newest on top". Our moderation is heavily tailored to the users on my instance, and if a user says something grossly out of line with our general vibe, we'll remove them. That user is free to create an account on any other server who'll have them. We're not limiting their access to Mastodon. We're saying that we don't want their stuff on our own server.
What are the legal ramifications for the many thousands of similar operators which are much closer in feel to a message board than to Facebook or Twitter? Does a server run by Republicans have to accept Communist Party USA members and their posts? Does a vegan instance have to allow beef farmers? A PlayStation fan server host pro-PC content?
> I think the ultimate problem is that social media is not unbiased — it curates what people are shown.
It is not only biased but also biased for maximum engagement.
People come to these services for various reasons but then have this specifically biased stuff jammed down their throats in a way to induce specific behavior.
I personally don't understand why we don't hammer these social media sites for conducting psychological experiments without consent.
HN also has an algorithm.
I'll have to read the third circuit's ruling in detail to figure out whether they are trying to draw a line in the Sand on whether an algorithm satisfies the requirements for section 230 protection or falls outside of it. If that's what they're doing, I wouldn't assume a priori that a site like Hacker News won't also fall afoul of the law.
> is that social media is not unbiased
That's how I read it, too. Section 230 doesn't say you can't get in trouble for failure to moderate, it says that you can't get in trouble for moderating one thing but not something else (in other words, the government can't say, "if you moderated this, you could have moderated that"). They seem to be going back on that now.
Real freedom from censorship - you cannot be held liable for content you hosted - has never been tried. The US government got away with a lot of COVID-era soft censorship by just strong-arming social media sites into suppressing content because there were no first-amendment style protections against that sort of soft censorship. I'd love to see that, but there's no reason to think that our government is going in that direction.
This turns on what TikTok "knew":
"But by the time Nylah viewed these videos, TikTok knew that: 1) “the deadly Blackout Challenge was spreading through its app,” 2) “its algorithm was specifically feeding the Blackout Challenge to children,” and 3) several children had died while attempting the Blackout Challenge after viewing videos of the Challenge on their For You Pages. App. 31–32. Yet TikTok “took no and/or completely inadequate action to extinguish and prevent the spread of the Blackout Challenge and specifically to prevent the Blackout Challenge from being shown to children on their [For You Pages].” App. 32–33. Instead, TikTok continued to recommend these videos to children like Nylah."
We need to see another document, "App 31-32", to see what TikTok "knew". Could someone find that, please? A Pacer account may be required. Did they ignore an abuse report?
See also Gonzales vs. Google (2023), where a similar issue reached the U.S. Supreme Court.[1] That was about whether recommending videos which encouraged the viewer to support the Islamic State's jihad led someone to go fight in it, where they were killed. The Court rejected the terrorism claim and declined to address the Section 230 claim.
TikTok, Inc., via its algorithm, recommended and promoted videos posted by third parties to ten-year-old Nylah Anderson on her uniquely curated “For You Page.” One video depicted the “Blackout Challenge,” which encourages viewers to record themselves engaging in acts of self-asphyxiation. After watching the video, Nylah attempted the conduct depicted in the challenge and unintentionally hanged herself. -- https://cases.justia.com/federal/appellate-courts/ca3/22-3061/22-3061-2024-08-27.pdf?ts=1724792413
"""The Court held that a platform's algorithm that reflects "editorial judgments" about "compiling the third-party speech it wants in the way it wants" is the platform's own "expressive product" and is therefore protected by the First Amendment.
Given the Supreme Court's observations that platforms engage in protected first-party speech under the First Amendment when they curate compilations of others' content via their expressive algorithms, it follows that doing so amounts to first-party speech under Section 230, too."""
I've agreed for years. It's a choice in selection rather than a 'natural consequence' such as a chronological, threaded, or even '__end user__ upvoted /moderated' (outside the site's control) weighted sort.
For anyone making claims about what the authors of Section 230 intended or the extent to which Section 230 applies to targeted recommendations by algorithms, the authors of Section 230 (Ron Wyden and Chris Cox) wrote an amicus brief [1] for Google v. Gonzalez (2023). Here is an excerpt from the corresponding press release [2] by Wyden:
> “Section 230 protects targeted recommendations to the same extent that it protects other forms of content presentation,” the members wrote. “That interpretation enables Section 230 to fulfill Congress’s purpose of encouraging innovation in content presentation and moderation. The real-time transmission of user-generated content that Section 230 fosters has become a backbone of online activity, relied upon by innumerable Internet users and platforms alike. Section 230’s protection remains as essential today as it was when the provision was enacted.”
[1][PDF] https://www.wyden.senate.gov/download/wyden-cox-amicus-brief...
[2] https://www.wyden.senate.gov/news/press-releases/sen-wyden-a...
I'm not at all opposed to implementing new laws that society believes will reduce harm to online users (particularly children).
However, if Section 230 is on its way out, won't this just benefit the largest tech companies that already have massive legal resources and the ability to afford ML-based or manual content moderation? The barriers to entry into the market for startups will become insurmountable. Perhaps I'm missing something here, but it sounds like the existing companies essentially got a free pass with regard to liability of user-provided content and had plenty of time to grow, and now the government is pulling the ladder up after them.
> In other words, the fundamental issue here is not really whether big tech platforms should be regulated as speakers, as that’s a misconception of what they do. They don’t speak, they are middlemen. And hopefully, we will follow the logic of Matey’s opinion, and start to see the policy problem as what to do about that.
This is a pretty good take, and it relies on pre-Internet legal concepts like distributor and producer. There's this idea that our legal / governmental structures are not designed to handle the Internet age and therefore need to be revamped, but this is a counterexample that is both relevant and significant.
Fantastic write-up. The author appears to be making more than a few assumptions about how this will play out, but I share his enthusiasm for the end of the "lawless no-man’s-land" (as he put it) era of the internet. It comes at a great time too, as we're all eagerly awaiting the AI-generated content apocalypse. Just switch one apocalypse for a kinder, more human-friendly one.
> So what happens going forward? Well we’re going to have to start thinking about what a world without this expansive reading of Section 230 looks like.
There was an internet before the CDA. From what I remember, it was actually pretty rad. There can be an internet after, too. Who knows what it would look like. Maybe it will be a lot less crowded, less toxic, less triggering, and less addictive without these gigantic megacorps spending buku dollars to light up our amygdalas with nonsense all day.
The ruling itself says that this is not about 230, it's about TikTok's curation and collation of the specific videos. TikTok is not held liable for the user content but for the part that they do their 'for you' section. I guess it makes sense, manipulating people is not OK whether it's for political purposes as facebook and twitter do, or whatever. So 230 is not over
It would be nice to see those 'For you' and youtube's recomendations gone. Chronological timelines are the best , and will bring back some sanity. Don't like it? don't follow it
> Accordingly, TikTok’s algorithm, which recommended the Blackout Challenge to Nylah on her FYP, was TikTok’s own “expressive activity,” id., and thus its first-party speech.
>
> Section 230 immunizes only information “provided by another[,]” 47 U.S.C. § 230(c)(1), and here, because the information that forms the basis of Anderson’s lawsuit—i.e., TikTok’s recommendations via its FYP algorithm—is TikTok’s own expressive activity, § 230 does not bar Anderson’s claims.
So basically closer and closer to governmental control over social networks. Seems like a global trend everywhere. Governments will define the rules by which communication services (and social networks) should operate.
Disclosures: I read the ruling before reading Matt Stoller’s article. I am a subscriber of his. I have written content recommendation algorithms for large audiences. I recommend doing one of these three things.
Section 230 is not canceled. This is a significant but fairly narrow refinement of what constitutes original content and Stoller’s take (“The business model of big tech is over”) is vastly overstating it.
Some kinds of recommendation algorithms produce original content (speech) by selecting and arranging feeds of other user generated content and the creators of the algorithms can be sued for harms caused by those recommendations. This correctly attaches liability to risky business.
The businesses using this model need to exercise a duty of care toward the public. It’s about time they start.
> There is no way to run a targeted ad social media company with 40% margins if you have to make sure children aren’t harmed by your product.
More specific than being harmed by your product, Section 230 cares about content you publish and whether you are acting as a publisher (liable for content) or a platform (not liable for content). This quote is supposing what would happen if Section 230 were overturned. But in fact, there is a way that companies would protect themselves: simply don't moderate content at all. Then you act purely as a platform, and don't have to ever worry about being treated as a publisher. Of course, this would turn the whole internet into 4chan, which nobody wants. IMO, this is one of the main reasons Section 230 continues to be used in this way.
To me this decision doesn't feel it is demolishing 230, but reducing its scope, a scope that was exanded by other court decisions. Per the article 230 said not liable for user content and not liable for restricting content. This case is about liability for reinforcing content.
Would love to have a timeline only, non reinforcing content feed.
Might be a cultural difference (im not from the US), but leaving a 10 year unsupervised with content from (potentially malicious) strangers really throws me off.
Wouldn't this be the perfect precedence case on why minors should not be allowed on social media?
Hurting kids, hurting kids, hurting kids -- but, of course, there is zero chance any of this makes it to the top 30 causes of child mortality. Much to complain about with big tech, but children hanging themselves is just an outlier.
Part of the reason social media has grown so big and been so profitable is that these platforms have scaled past their own abilities to do what normal companies are required to do.
Facebook has a “marketplace” but no customer support line. Google is serving people scam ads for months, leading to millions in losses. (Imagine if a newspaper did that.) And feeds are allowed to recommend content that would be beyond the pale if a human were curating it. But because “it’s just an algorithm bro” we give them a pass because they can claim plausible deniability.
If fixing this means certain companies can’t scale to a trillion dollars with no customer support, too bad. Google can’t vet every ad? They could, but choose not to. Figure it out.
And content for children should have an even higher bar than that. Kids should not be dying from watching videos.
This is wonderful news.
The key thing people are missing is that TikTok is not being held responsible for the video content itself, they are being held responsible for their own code's actions. The video creator didn't share (or even attempt to share) the video with the victim- TikTok did.
If adults want to subscribe themselves to that content, that is their choice. Hell, if kids actively seek out that content themselves, I don't think companies should be responsible if they find it.
But if the company itself is the one proactively choosing to show that content to kids, that is 100% on them.
This narrative of being blind to the vagaries of their own code is playing dumb at best: we all know what the code we write does, and so do they. They just don't want to admit that it's impossible to moderate that much content themselves with automatic recommendation algorithms.
They could avoid this particular issue entirely by just showing people content they choose to subscribe to, but that doesn't allow them to inject content-based ads to a much broader audience, by showing that content to people who have not expressed interest/ subscribed to that content. And that puts this on them as a business.
From the article:
> Because TikTok’s “algorithm curates and recommends a tailored compilation of videos for a user’s FYP based on a variety of factors, including the user’s age and other demographics, online interactions, and other metadata,” it becomes TikTok’s own speech. And now TikTok has to answer for it in court. Basically, the court ruled that when a company is choosing what to show kids and elderly parents, and seeks to keep them addicted to sell more ads, they can’t pretend it’s everyone else’s fault when the inevitable horrible thing happens.
If that reading is correct, then Section 230 isn't nullified, but there's something that isn't shielded from liability any more, which IIUC is basically the "Recommended For You"-type content feed curation algorithms. But I haven't read the ruling itself, so it could potentially be more expansive than that.
But assuming Matt Stoller's analysis there is accurate: frankly, I avoid those recommendation systems like the plague anyway, so if the platforms have to roll them back or at least be a little more thoughtful about how they're implemented, it's not necessarily a bad thing. There's no new liability for what users post (which is good overall IMO), but there can be liability for the platform implementation itself in some cases. But I think we'll have to see how this plays out.
Whatever this means, I hope it means less censorship. That's all my feeble brain can focus on here: free speech good, censorship bad. :)
I love this.
Court: Social Media algos are protected speech
Social Media: Yes! Protect us
Court: Since you're speech you must be liable for harmful speech as anyone else would be
Social Media: No!!
If I spam filter comments am I subject to this? That is, the remaining comments are effectively like I was saying them?
The link to the actual decision:
https://cases.justia.com/federal/appellate-courts/ca3/22-306...
Section 230 is alive and well, and this ruling won't impact it. What will change is that US social media firms will move away from certain types of algorithmic recommendations. Tiktok is owned by Bytedance which is a Chinese firm, so in the long run - no real impact.
Anyone know what the reputation of the Third Circuit is? I want to know if this ruling is likely to hold up in the inevitable Supreme Court appeal.
The Ninth Circuit has a reputation as flamingly progressive (see "Grants Pass v. Johnson", where SCOTUS overruled the Ninth Circuit, which had ruled that cities couldn't prevent homeless people from sleeping outside in public parks and sidewalks). The Fifth Circuit has a reactionary reputation (see "Food and Drug Administration v. Alliance for Hippocratic Medicine", which overruled a Fifth Circuit ruling that effectively revoked the FDA approval of the abortion drug mifepristone).
Hoo boy.
So- platforms aren’t publishers, they are distributors (like news stands or pharmacies).
So they are responsible for the goods they sell.
They aren’t responsible for user content - but they are responsible for what they choose to show.
This is going to be dramatic.
Moderation doesn't scale, it's NP-complete or worse. Massive social networks sans moderation cannot work and cannot be made to work. Social networks require that the moderation system is a super-set of the communication system and that's not cost effective (except where the two are co-extensive, e.g. Wikipedia, Hacker News, Fediverse.) We tried it because of ignorance (in the first place) and greed (subsequently). This ruling is just recognizing reality.
This seems like it contradicts the case where YouTube wasn’t liable for recommending terrorist videos to someone.
I'm not sure that Big Tech is over. Media companies have had a viable business forever. What happens here is that instead of going to social media and hearing about how to fight insurance companies, you'll just get NFL Wednesday Night Football Presented By TikTok.
Pavel gets arrested, Brazil threatens Elon, now this.
I am not happy with how governments think they can dictate what internet users can and cannot see.
With respect to TikTok, parents need have some discipline and not give smart phones to their ten-year-olds. You might as well give them a crack pipe.
There's no reason,as far as I'm concerned, that we shouldn't have a choice in algorithms on social media platforms. I want to be able to pick an open source algorithm that i can understand the pros and cons of. Hell let me pick 5. Why not?
> the internet grew tremendously, encompassing the kinds of activities that did not exist in 1996
I guess that's one way to say that you never experienced the early internet. In three words: rotten dot com. Makes all the N-chans look like teenagers smoking on the corner, and Facebook et.al. look like toddlers in paddded cribs.
This will frankly hurt any and all attempts to host any content online, and if anyone can survive it, it will be the biggest corporations alone. Section 230 also protected ISPs and hosting companies (linode, Hetzer, etc) after all.
Their targeting may not be intentional, but will that matter? Are they willing to be jailed in a foreign country because of their perceived inaction?
"In other words, the fundamental issue here is not really whether big tech platforms should be regulated as speakers, as that's a misconception of what they do. They don't speak, they are middlemen."
Parasites.
I think a bigger issue in this case is the age. A 10-year old should not have access to TikTok unsupervised, especially when the ToS states the 13-year age threshold, regardless of the law’s opinion on moderation.
I think especially content for children should be much more severely restricted, as it is with other media.
It’s pretty well-known that age is easy to fake on the internet. I think that’s something that needs tightening as well. I’m not sure what the best way to approach it is though. There’s a parental education aspect, but I don’t see how general content on the internet can be restricted without putting everything behind an ID-verified login screen or mandating parental filters, which seems quite unrealistic.
Finally, it goes to end of global social media. jurisdiction cannot be use as a weapon. if you use it as a weapon. they don't hesitate use that to you as a weapon.
I hope this makes certain streaming platforms liable for the things certain podcast hosts say while they shovel money at and promote them above other content.
So under this new reading of the law, is it saying that AWS is still not liable for what someone says on reddit, but now reddit might be responsible for it?
It is amazing how people were programmed to completely forget the meaning of Section 230 over the years just by repetition of the stupidest propaganda.
Surely this will bubble up to the Supreme Court?
Once they've weighed in, we'll know if the "free ride" really is over, and if so what ride replaces it.
> Because TikTok’s “algorithm curates and recommends a tailored compilation of videos for a user’s FYP based on a variety of factors, including the user’s age and other demographics, online interactions, and other metadata,” it becomes TikTok’s own speech.
This is fascinating and raises some interesting questions about where the liability starts and stops i.e. is "trending/top right now/posts from following" the same as a tailored algorithm per user? Does Amazon become culpable for products on their marketplace? etc.
For good or for bad, this century's Silicon Valley was built on Section 230 and I don't foresee it disappearing any time soon. If anything, I suspect it will be supported by future/refined by legislation instead of removed. No one wants to be the person who legisliate away all online services...