Item — HN Reader

>That’s right: I teach courses in Python and Pandas. Never mind that the first is a programming language and the second is a library for data analysis in Python. Meta’s AI system noticed that I was talking about Python and Pandas, assumed that I was talking about the animals (not the technology), and banned me.

If this is true, that is insanely embarrassing for a company that purports to be a sophisticated programmatic advertising platform.

5 comments

I wanted to run ads for my plant nursery. Their bot suspended me before I can even run my first ads. They don't tell you the reason why and you have one chance to dispute. Then it's another automated response, again, not telling you the reason, you're permanently blocked after that. Their system make it feel like they actively punish legit but inexperienced business owner while letting all sort of scam running around in the wild.

What's wrong with talking about animals anyway? Truly bizarre.

Depends in what context you are talking about the animals.

Trade in animals is covered by a number of laws¹ intended to protect them, and fb has a blanket ban on advertising trading animals. This is a perfectly logical thing to declare wrong.

What has happened here, most likely, is that a right thing (selling courses about Python and Pandas, the technical things of that name not the animals) has been misidentified as that wrong thing, by an automated system that doesn't understand that two concepts can have the same names, with no practical way for the affected advertiser to appeal or even query the automated decision.

Basically this is a variant of the https://en.wikipedia.org/wiki/Scunthorpe_problem.

[1] Morals too, of course, but companies like FB work based on laws/regulations² rather than based on what is considered [im]moral.

[2] or based on working around laws/regulations!

That department should upgrade to a modern LLM which would do a better job than many humans in understanding the context of python and pandas.

There would still be need for some sort of appeal/correction process, which doesn't seem to exist in a useful state now and would be required no matter what automated system you put in place.

I think if the front line bot was an LLM, rather than what is probably regex, they would probably have more true positives and less false positives that would potentially allow them to have a real person handling the exceptions - then train the LLM on the exception for the next release.

I'm assuming something to do with the illegal trading of exotic/endangered species; mentioning pythons and pandas, alongside whatever wording was used to sell the course probably triggered the bot to flag it.

That's the only explanation I have.

Just guessing here, but if the senteces sounded like "how to use pandas to help you in work" AI could have decided that you're going to use a cute bear to lay bricks for you. Plus the "exotic animals", as someone else mentioned.

Not guessing, I used to work on that team: it’s based on keywords alone. The list of items is too big and changes too often to handle anything beyond that — more than a trillion edits per month.

I wouldn’t be surprised if you could sell endangered animals if you called them P4ndas.

Makes me wonder what's the worst possible phrasing

"Master Python and Pandas, make them work for you, attack new challanges, earn good money"

"How to use the innards of Python and Pandas"?

"If you find yourself fighting with Python and Pandas, I'll help you improve your technique"?

"Using Python and Pandas in the wild"?

"10 unexpected uses for Python and Pandas in the food and apparel industries"?

"Breaking all the rules for Python imports"?

"How to replace your whole ecosystem with just Python and Pandas"?

"Put Python to work on the family farm"?

I asked GPT-4:

There's a book called "Using Python and Pandas in the wild". What do you think it's about?

"Using Python and Pandas in the wild" likely refers to the application of the Python programming language and the Pandas library in real-world scenarios or practical use cases.

Even text-davinci-003 nails this (via OpenAI Playground).

Prompt:

> The newly-released book "Using Python and Pandas in the wild" discusses

Response:

> the application of Python and Pandas for data analysis, cleaning, and transformation. It focuses on the fundamentals of Pandas, how to use it to explore and manipulate data, and offer guidance for more advanced topics, such as building data visualizations and machine learning models. Readers will also get an overview of the best practices for working with time-series data, unsupervised learning, and natural language processing. The book is designed to be a comprehensive resource for anyone who wants to learn how to use Python and Pandas for data analysis.

> Makes me wonder what's the worst possible phrasing

I got some very strange looks when buying one particular book back in the days when physical books were the norm. It was only after I got home that I realised a book called “Python Cookbook” gives an entirely different impression to non-programmers.

It reminds of old Ruby anecdote. There was this guy on a Ruby conference with t-shirt with ":s*x" print, which for rubyists means "s*x symbol", but regular people will read it, well... colon s*x.

(Censored because I'm at work and I'm afraid of my VPN.)

Mmmm, I know rattlesnakes are said to be a pretty good meal. I wonder if pythons and boas taste any good. Those big snakes might make a delicious feast.

Containerized Python and Pandas for easier shipping!

“Minimize cost of your Python and Pandas deployment for international customers”

Perhaps you're not allowed to use Facebook for selling animals. Which would be sensible IMO

Selling pets should be fine?

There are many laws regarding pet ownership and for good reason, if not involving an invasive species there's also animal cruelty issues. Pandas and other exotic animals need specialized care which a typical vet will not be able to provide. And that's if they aren't dangerous. Then there's the puppy mills, which is a huge issue on its own.

The puppy mills aren't running ads, they're using Groups and doing their advertising on the comments sections of local animal shelters. It's grotesque.

A post will be like "Meet Billy, he's been here at the shelter for 2 years," get a few sympathy comments and be followed up with a "look, we have puppies for sale!"

All existing pandas are owned by the PRC so the chance of somebody selling a panda for real, vs selling some sort of panda themed toy/book/etc, is virtually nil. No human would make this mistake; their filter is braindead.

Pythons, on the other hand, are often (illegally) sold, and can be extremely difficult to eradicate as an invasive species. Florida's Everglades are in trouble because people buy Burmese pythons and release them when they grow too big to handle.

That's not to defend Facebook's approach to ad management whatsoever. I definitely agree that it is far too blunt a tool for the problem.

Pretending the problem doesn't exist, however, because one example is unlikely is just intellectually dishonest.

That is true that the python trade is somewhat problematic and is accordingly regulated, but there are also a lot of people who are legally selling pythons on facebook and evidently don't get banned for it. Search "facebook python breeder" and they're easy to find. My conclusion is that the filtering system is totally and inexplicably capricious. A braindead computer program making indefensible decisions that it probably can't even justify to facebook employees themselves, let alone to the people it's banning. Maybe mentioning pythons or pandas alone would be fine, but both together gets you classified as an exotic smuggler dealing in literal bears? Who knows. Probably not even facebook knows.

Depends on local laws.

Also it can be difficult to reliably distinguish between an individual passing on a pet, a properly licensed breeder selling animals, and a bad actor doing the same (selling exotic animals, being an unlicensed breeder in jurisdictions where this is not permitted (generally or for particular animals), etc.). For this reason many advertising companies simply have a blanket ban on advertising any animal trading. Being more selective potentially leaves them open to legal issues if the selection process makes a bad decision and lets something illegal through, and that risk is not worth taking in order to allow (and therefore take a cut from) the more legitimate trade.

Selling animals is highly regulated in much of the world, which is why you see “free animals but pay for the shots” kind of things here and there.

Just as long as you're not selling pandas, which is what the AI thought he was doing

You can talk about them on FB, just can't advertise trading them.

I think it'd be embarrassing if the tech was making this sort of mistake in a high % of cases.

Anecdotally, that's the only account I've read of this kind of error.

Not defending Facebook, they ought to deploy humans in the loop.

But saying the tech is embarrassing misses how machine learning works. It will always make mistakes, and the kind of mistakes that seem silly from a human perspective.

That's because, well, these systems aren't human.

>that's the only account I've read of this kind of error.

it's still anecdotal, but the author did mention that they went to FB/LinkedIn and they got a pretty swift answer. If nothing else, it sounds like it wasn't a unique case (or the author has a very tight knit group of tech isntructors).

What I mean is, there are certainly a lot more than 2 or 3 individuals or businesses that have ever advertised on FB for Python and Pandas (programming) stuff. Courses, recruiting, books, events, commercial libraries, etc.

If this kind of error was systematic, no one would be able to run these ads. We'd have certainly already seen a public outcry about it.

That doesn't seem to be the case, here (yet)?..

I did leave the possibility of "tight nit group of tech instructors".

With that said, it's not impossible for this to be a systematic issue considering (theoretically, I don't have formal evidence)

1. Facebook may not be the best place to reach out to the tech community

2. There may simply be too many positives (so, actual exotic animal trading) for it to be worth fixing the few false positives. And the author lacked the financial worth to properly fix the issue.

3. At the end of the day, the issue was not resolved. Leaving the potential next victim wanting to talk about Python laying in wake.

I subscribe to #2 personally.

Those mistakes are systematic.

I worked on the team that dealt with that, and you pretty much can’t mention exotic animals by name if you sell books, movies, fan art, etc.

Yes, indeed.

Sounds like speculation to me. Anything to back that up?

Clearly it’s speculation, this entire thread is speculation because Meta won’t actually tell anyone why they were banned.

The author might be speculating, but I worked on that team and his diagnostic is very likely exact.

Do you have any insights into how it might be resolved given that you think a blatant false positive like this is likely? Or is it a lost cause?

It’s a story in three steps:

1. The problem is that there is a staggering amount of “things” that can be either sold on Facebook/Instagram/WhatsApp or advertised for (it’s the same service): every film on Netflix, every item for sale on Etsy, eBay, Amazon, every game on a random retro console platform, etc. There are hundreds of billions of things. Some are duplicates (humanity barely makes that many things), but not many are on the same platform anyway. The real problem is that there are even more updates: trillions. When I was there, most large partners updated all their items daily because the API didn’t let them match items dynamically easily enough, and they believed updates granted them some magical recommendation properties. All those updates have to go through a staggering amount of processes, one by one: unauthorized items, spam filters, enlarging photos, transformers to figure out what it is so it can be advertised to the right audience, etc. Why not just item creation? Because it’s all too easy to upload a valid description and then update it to sell unicorn meat. There are miracles done to figure out which items are new and which are neutral changes (like price or inventory updates) or no changes at all. Still, those aren’t nearly as effective as offering the right tools to the bigger partners (like helping them figure out which items were actually updated on their side, before sending them through an expensive, convoluted pipeline on our end). That was my recommendation, which got lost in internal politics. But at the moment, whatever “trivial“ process you recommend, you genuinely have to add 12 zeros to its computational cost.

2. Assuming you resolve internal politics, prioritize sending new products (and genuine product updates) separately from inventory, and only receive billions of materially new descriptions every day, the company has the skill and resources to run proper filters and separate a course on Python with an illegal snake trade — but why? More than any other company, Meta demands a business case for everything, and being nice to 0.000_001% of their partners isn’t a good case. Again, I’m not exaggerating: the entire economy of several large countries goes through WhatsApp. Groceries, Restaurant supplies, Job offers, Prescription drugs, Taxi services, Spare garage parts, Police uniforms, Wedding planning, Army weapon procurement contracts, there is everything going through there. Reuven is a great teacher, but his classes are a speck in that world. “All digital goods not sold by the top 15 platforms” is a tiny sliver of a slice of that pie. So you have to figure out how many second-hand VHS resellers got kicked out for offering a lightly used copy of _The Hounds of Baskerville_ but also how many bakers making custom cakes have heard of someone getting kicked and didn’t even try advertising… With barely any trace on Meta for many of those and several orders of magnitude more actual spammers and people creating accounts by the dozen genuinely trying to sell endangered species. “Is the juice worth the squeeze?” gets weirdly negative when you have all the juice in the world already.

3. On top of that, your business case has to be worth the time for several of the best teams (read: expensive) at Meta: computer vision, spam detection, parsing the largest database Meta has, etc. More importantly, their filter has to offer an absolute guarantee that not a single pangolin can be advertised or sold on any of the services: the company breathes by the strongly held belief that any compromising screenshot will end up on the front page of the NYTimes for a month (because it absolutely will). Some YouTuber selling powdered sugar as “coke” as a prank for views? He has to be banned before he can fake a screenshot. Any less, and the PR team will have headaches.

Bonus: Among the things that Meta has learned they can’t rely on through similar crises is Unicode. There’s a big scandal that (among other failures in management) hinged on a bag-things-detector failing because one language has an alphabet that wasn’t compatible with Unicode, so the bag-things-detector failed silently. Can you build a system able to detect euphemisms similar to ‘oregano :wink: :wink:’ in Hokkien, or Gondi? That shouldn’t prevent Meta from having a system that recognizes English, Spanish, Hindi, French, Mandarin, etc., and works for 95% of the items, but you can never know if you might be missing something.

A heavy-handed system, full or blatant false positive scales better, doesn’t cost that much, and is cheaper to operate, with less risk of freak actual positive.

Thanks for the detailed answer. I guess these are unavoidable without investing a lot more money on it if you are aiming for basically reducing false negatives to zero at that scale. One thing that surprised me is that Yann LeCun frequently mentions transformers have been in prod for all parts of moderation on Meta for a while already but maybe they come into play at more critical problems or at later stages.

Moderation, yes definitely. There’s are a couple billion users, they post a handful of comments per day and rarely post anything long enough to justify applying a transformer, so “everything posted to Facebook/Instagram” is several orders of magnitude smaller. Barely an inconvenience.

Messenger is bigger, but hardly one writes anything long enough to be suspicious. WhatsApp isn’t moderated the same way.

What would be the alternative here? Meta bans people from selling live animals or bans them from selling programming courses?

I don't think it is stated anywhere that the ban was the result of the thing that was being advertised.

> This review found inauthentic behavior or violations of our Advertising Policies or Community Guidelines

There could be thousands of other things.

Sure would be nice if they'd actually lay out what rule(s) were broken then, I guess.

> There could be thousands of other things.

Sounds like speculation to me. Anything to back that up?