Somewhat related: open YouTube in an incognito window.
Search something rather innocuous like "news". Click the first thing then go back to the main page. You have to watch through the ad.
The amount of slop you will get suggested as soon as the algorithm warms up is nuts. The ad you are first served might also be slop. I mean AI generated video advertising overt bogus cures (a memorable one recently - "did you know the Bible has the answer to Alzheimer's?") over AI slop footage.
When I compare to my normal Youtube experience, it feels like a totally different app. I have never really experienced such a cheap illustration of the "Youtube rabbit hole" effect. The degree to which AI authored advertising is participating in this is imo pretty concerning if you have, in particular, older parents or relatives.
What's crazy is how long it takes to report something like this to a big company and actually have them take action on it. I remember 10 years ago I reported a Giphy ad, and it took them months to take action.
I'd be interested to know if the same amount of leniency shown here would be applied if we were talking about ads depicting violence against elected politicians.
That's a really common reaction to people first hearing about the delta between how society talks about the subject vs what they actually do about it. Its a bit like the fiction of the social safety net or strong institutions.
The truth is impunity is the rule, victims especially of CSA and CSAM, or revenge porn, or of Mark or Musk's companies generating naked images of their body have basically no power, and its an exception when someone who is an abuser AND has any amount of social standing is put away.
it can take awhile for things to work their way from NCMEC (national center for missing & exploited children) to the appropriate local agency, then have that agency do their investigation, get a warrant (if they can), etc. and that is assuming that there is even a cooperative jurisdiction for NCMEC to send the initial report to.
but, plenty of people do end up in prison, especially if its facebook/dropbox/gmail/etc... they just dont make the headlines (especially nationally) unless its especially heinous.
> but as Wired notes, it relies on "automated tools" as part of the process, which may have been how the more than 50 ads TTP discovered made it through.
Again, and again, and again, these technologies have proven ineffective to moderate content. User posted and otherwise. Until we have a legal mechanism in place to hold these companies meaningfully liable for this, it will not stop.
Every single one of these has the same damn list of excuses about cost of hiring human moderators, but I promise everyone reading this if you make the executives of a social network legally liable for distributing CSAM on their platforms, they will find all kinds of money to get human moderators really fucking quick.
The article also mentions that a ton of even worse ads were posted between when they contacted Meta and when they were ready to go live with the article, and some of the offending ads which were taken down had just been re-uploaded again and automatically approved.
Not only are their AI tools ineffective at what they claim they should be doing, they can't even be bothered to prevent removed ads from being re-uploaded later on.
I think people have been demanding this for ages but lobbying by social media companies against any change in the law is enormous.
As for the cost of content moderation, Meta has zero interest in any type of moderation to begin with. The cost is just a public-facing excuse. In Careless People, Wynn-Williams mentions that Facebook didn't do anything to stop anti rohingya posts in Myanmar because they did not have any fact checkers who could read Burmese. In other words, they did not have a single moderator in the damn country.
FB might or might not have a right to exist in someone's eyes, but the question remains: is 0.5c-like latency messaging across the world acceptable? Because facebook or not, given internets this is what you will end up with. People are people.
On the other hand, now try to ban that telecom. Gods will laugh.
Yeah and/but also the case it's not bidirectional enough
The eternal question of tech platforms - should we be responsible for what users post? (Ads are not user posted, so we can eliminated that factor. Even if posted by 'users' it's a separate pipeline) But with companies like Meta it doesn't even have to go that far - the philosophical conversation - because it's so hostile to users. People get banned for mild things all the time (apparently extra the case, since the 'pivot' to AI) while the absolutely horrible things you report never get taken down
> The eternal question of tech platforms - should we be responsible for what users post?
I would never say a platform has to be responsible for what a user posts, and to be clear, I mean either a user of the platform, or a user of the platform's ad tooling, or anything else. Anything that arrives via an upload button of some sort. However: If you as a platform then distribute what has been posted, yes, you are responsible for it. Your service is now broadcasting that content. If that content is harmful, you should be held responsible.
Who else conceivably could be? And if the response to this is "well we have way too much stuff coming in to moderate it" okay, then restrict how much stuff comes in. Require approval on individual posts or on accounts.
Like, this is such basic shit. If a newspaper printed CSAM on the front page, they would be arrested within the day. But because Facebook is real super big they just don't have to take care of their house? Ridiculous.
But they couldn't possibly do it at that the scale they're operating at. They couldn't hire enough people in the countries they operate, or know the language, or trust that who they do hire aren't corrupt or working against their interest. And they can't manually approve every ad, or make sure targeting isn't illegal or otherwise harmful.
---
This is sarcasm, by the way. Most decent people make sure what they're about to do won't harm other people and if would cause harm they wouldn't do it. But social media companies and ad networks aren't run by decent people. They're run by people who believe their business is entitled to infinite scale regardless of how many people are harmed along the way.
>But they couldn't possibly do it at that the scale they're operating at. They couldn't hire enough people in the countries they operate, or know the language, or trust that who they do hire aren't corrupt or working against their interest. And they can't manually approve every ad, or make sure targeting isn't illegal or otherwise harmful.
you say sarcasm, but i think this is literally true.
this isn't a defense of facebook, i think that their scale makes it actually impossible (while remaining a going concern). the result should be that they arent allowed to operate at a scale that they cant handle, instead of being able to brush it under the rug every ~6 months when an incident hits the news cycle.
the amount of content posted per second to facebook is unfathomable. a quick google (just for a rough order of magnitude) says ~65,000 posts per second. add in instagram (which is worse for CSAM) and its gotta be at least double that.
it would require millions on millions of employees, spread across ~every country. between the payroll, buildings and property, equipment, etc. i just dont think its possible at that scale.
(again, emphasizing that im not saying this as an excuse for facebook, but that i dont really think facebook/instagram should exist at all)
No, because most ads are created by automated systems (many of which are scam/spam ads) and Meta doesn't want to lose revenue by slowing the system down to human speed.
Facebook absolutely could remain a going concern while dealing with all of these problems, the problem is that they would be spending money to lose money.
One of the more perverse aspects of risk prevention is that riskier customers are often way more lucrative. It's kind of inherent to the notion that markets price risk into transactions. The thing is, illegal behavior is also just a particular kind of risk: if someone wants to sell nudify apps or scam old people, Facebook could conceivably be on the hook[0] for damages if they're caught. And in a sense, a risk premium is just a fancy kind of bribe: if Facebook won't take my ads because they think they're too risky, then I'll just pay more and more until Facebook accepts them.
With illegal behavior, we hope that we can make the punishment high enough that Facebook refuses any conceivable risk premium the perpetrators of that behavior would be willing to pay. The problem is that Facebook is really desperate for any increasing growth story, and when you're too big to fail, your growth story is to drip-feed enshittify your stack. Increasing tolerance for illegal behavior is downstream of this: you cut enforcement, tell them to automate more, they become easier to bypass... and you make more money. Because it turns out all that risky behavior... is only risky for normal people. Facebook can launder[1] the risk away and make a tidy profit.
[0] Do not tell me about CDA 230, I was there when the old magic was written.
[1] See also, HSBC turning themselves into a literal money laundering operation.
So you throw theZuck in the hoosegow. Now what? FB will continue printing money and doing the things it always has even if under new management. Why do you think it'll change with just throw theZuck in jail?
(This reminds me of, and not to be political. This is a technological, rather than political angle) There's a TED talk by Carole Cadwalladr about "Facebook's role in Brexit" in which she talks about the ads the Leave campaign ran. This is what is meant by Facebook's "breaking democracy", where campaigns don't have to go through the usual disclosure avenues
Are there reasons why Brexit might have been popular? Yes, in the sense I've heard about them. Should we then be blaming Facebook?
The problem is consolidated power. Carole herself said in her talk that it's not about Trump vs no Trump (sorry. It's been a bit since I've seen the talk. I shouldn't just paraphrase), it's about citizens' right to meaningfully choose in an information landscape
This is the problem with Facebook. It becomes an Everything App. Who cares about fraud (say, on marketplace, needless to say the site itself, the way it's screwed over businesses) on the platform when you can grow big? Let's say in this case, the actual case I'm commenting on - "Facebook didn't know". Well then that's the problem, isn't it? It's a company set up by someone of low moral reasoning (https://news.ycombinator.com/item?id=1692122) that doesn't set out to make a good product, something people will actually like. It's about dominating an industry and ads can be totally unvetted or scummy or scammy if it enables them to do this
Somewhat related: open YouTube in an incognito window.
Search something rather innocuous like "news". Click the first thing then go back to the main page. You have to watch through the ad.
The amount of slop you will get suggested as soon as the algorithm warms up is nuts. The ad you are first served might also be slop. I mean AI generated video advertising overt bogus cures (a memorable one recently - "did you know the Bible has the answer to Alzheimer's?") over AI slop footage.
When I compare to my normal Youtube experience, it feels like a totally different app. I have never really experienced such a cheap illustration of the "Youtube rabbit hole" effect. The degree to which AI authored advertising is participating in this is imo pretty concerning if you have, in particular, older parents or relatives.
All their fines are merely a cost of doing business, and they won't change their ways until the fines hurt.
So that's why they are lobbying for age verification, i.e ID verification?
Statistics like this is why stuff like Chat Control in the EU and banning private communications completely gets so much support.
What's crazy is how long it takes to report something like this to a big company and actually have them take action on it. I remember 10 years ago I reported a Giphy ad, and it took them months to take action.
Classy as usual.
I'd be interested to know if the same amount of leniency shown here would be applied if we were talking about ads depicting violence against elected politicians.
No, ads which are ANTI-pedophile aren't allowed.
Depends on the politician.
> and ran across Facebook, Threads, Messenger and Instagram.
So it's the SAME AD PLATFORM? Even in MESSENGER? My god. Not only is this whole thing horrible but it's doubly so
https://archive.ph/4LhoU
I didn't realize the original was behind a paywall.
grok is this you
Non-paywall alternative:
https://www.engadget.com/2231100/meta-apps-displayed-ads-tha...
36 million CSAM media removed in 2025? Why aren't millions of people in jail? Wtf?
That's a really common reaction to people first hearing about the delta between how society talks about the subject vs what they actually do about it. Its a bit like the fiction of the social safety net or strong institutions.
The truth is impunity is the rule, victims especially of CSA and CSAM, or revenge porn, or of Mark or Musk's companies generating naked images of their body have basically no power, and its an exception when someone who is an abuser AND has any amount of social standing is put away.
it can take awhile for things to work their way from NCMEC (national center for missing & exploited children) to the appropriate local agency, then have that agency do their investigation, get a warrant (if they can), etc. and that is assuming that there is even a cooperative jurisdiction for NCMEC to send the initial report to.
but, plenty of people do end up in prison, especially if its facebook/dropbox/gmail/etc... they just dont make the headlines (especially nationally) unless its especially heinous.
Bots, good luck finding the people responsible for them.
Also how much of the child exploitation material was created by the children themselves?
Teenagers who send each other nude photos are creating child exploitation material but you hardly want to put them jail for that.
VPNs, hacked accounts, no jurisdiction, etc., I'd presume.
Ah, yes. That's the Privacy is bad for Children argument. FB and Twitter never fudge with the appropriateness of their contents.
Because the overwhelming majority of these reports are for teens sexting each other.
Because then half the world leaders would be in jail and the other half would lose their excuse for building mass surveillance systems
> but as Wired notes, it relies on "automated tools" as part of the process, which may have been how the more than 50 ads TTP discovered made it through.
Again, and again, and again, these technologies have proven ineffective to moderate content. User posted and otherwise. Until we have a legal mechanism in place to hold these companies meaningfully liable for this, it will not stop.
Every single one of these has the same damn list of excuses about cost of hiring human moderators, but I promise everyone reading this if you make the executives of a social network legally liable for distributing CSAM on their platforms, they will find all kinds of money to get human moderators really fucking quick.
The article also mentions that a ton of even worse ads were posted between when they contacted Meta and when they were ready to go live with the article, and some of the offending ads which were taken down had just been re-uploaded again and automatically approved.
Not only are their AI tools ineffective at what they claim they should be doing, they can't even be bothered to prevent removed ads from being re-uploaded later on.
I think people have been demanding this for ages but lobbying by social media companies against any change in the law is enormous.
As for the cost of content moderation, Meta has zero interest in any type of moderation to begin with. The cost is just a public-facing excuse. In Careless People, Wynn-Williams mentions that Facebook didn't do anything to stop anti rohingya posts in Myanmar because they did not have any fact checkers who could read Burmese. In other words, they did not have a single moderator in the damn country.
FB might or might not have a right to exist in someone's eyes, but the question remains: is 0.5c-like latency messaging across the world acceptable? Because facebook or not, given internets this is what you will end up with. People are people.
On the other hand, now try to ban that telecom. Gods will laugh.
Yeah and/but also the case it's not bidirectional enough
The eternal question of tech platforms - should we be responsible for what users post? (Ads are not user posted, so we can eliminated that factor. Even if posted by 'users' it's a separate pipeline) But with companies like Meta it doesn't even have to go that far - the philosophical conversation - because it's so hostile to users. People get banned for mild things all the time (apparently extra the case, since the 'pivot' to AI) while the absolutely horrible things you report never get taken down
> The eternal question of tech platforms - should we be responsible for what users post?
I would never say a platform has to be responsible for what a user posts, and to be clear, I mean either a user of the platform, or a user of the platform's ad tooling, or anything else. Anything that arrives via an upload button of some sort. However: If you as a platform then distribute what has been posted, yes, you are responsible for it. Your service is now broadcasting that content. If that content is harmful, you should be held responsible.
Who else conceivably could be? And if the response to this is "well we have way too much stuff coming in to moderate it" okay, then restrict how much stuff comes in. Require approval on individual posts or on accounts.
Like, this is such basic shit. If a newspaper printed CSAM on the front page, they would be arrested within the day. But because Facebook is real super big they just don't have to take care of their house? Ridiculous.
Yeah sure, I think we basically agree
I'm just bemoaning the fact FB can't even pretend they care when they're so busy banning people for nothing
But they couldn't possibly do it at that the scale they're operating at. They couldn't hire enough people in the countries they operate, or know the language, or trust that who they do hire aren't corrupt or working against their interest. And they can't manually approve every ad, or make sure targeting isn't illegal or otherwise harmful.
---
This is sarcasm, by the way. Most decent people make sure what they're about to do won't harm other people and if would cause harm they wouldn't do it. But social media companies and ad networks aren't run by decent people. They're run by people who believe their business is entitled to infinite scale regardless of how many people are harmed along the way.
>But they couldn't possibly do it at that the scale they're operating at. They couldn't hire enough people in the countries they operate, or know the language, or trust that who they do hire aren't corrupt or working against their interest. And they can't manually approve every ad, or make sure targeting isn't illegal or otherwise harmful.
you say sarcasm, but i think this is literally true.
this isn't a defense of facebook, i think that their scale makes it actually impossible (while remaining a going concern). the result should be that they arent allowed to operate at a scale that they cant handle, instead of being able to brush it under the rug every ~6 months when an incident hits the news cycle.
Don’t they have like an 80% gross margin?
the amount of content posted per second to facebook is unfathomable. a quick google (just for a rough order of magnitude) says ~65,000 posts per second. add in instagram (which is worse for CSAM) and its gotta be at least double that.
it would require millions on millions of employees, spread across ~every country. between the payroll, buildings and property, equipment, etc. i just dont think its possible at that scale.
(again, emphasizing that im not saying this as an excuse for facebook, but that i dont really think facebook/instagram should exist at all)
Right, but the article isn't about posts, but about a paid ad campaign - surely those are started in numbers that could be moderated?
No, because most ads are created by automated systems (many of which are scam/spam ads) and Meta doesn't want to lose revenue by slowing the system down to human speed.
>Right, but the article isn't about posts, but about a paid ad campaign
the comment that started this thread says:
"these technologies have proven ineffective to moderate content. User posted and otherwise"
which is what i (and i assumed the parent to my comment) was replying to. i agree that paid ad campaigns alone is more tractable.
Facebook absolutely could remain a going concern while dealing with all of these problems, the problem is that they would be spending money to lose money.
One of the more perverse aspects of risk prevention is that riskier customers are often way more lucrative. It's kind of inherent to the notion that markets price risk into transactions. The thing is, illegal behavior is also just a particular kind of risk: if someone wants to sell nudify apps or scam old people, Facebook could conceivably be on the hook[0] for damages if they're caught. And in a sense, a risk premium is just a fancy kind of bribe: if Facebook won't take my ads because they think they're too risky, then I'll just pay more and more until Facebook accepts them.
With illegal behavior, we hope that we can make the punishment high enough that Facebook refuses any conceivable risk premium the perpetrators of that behavior would be willing to pay. The problem is that Facebook is really desperate for any increasing growth story, and when you're too big to fail, your growth story is to drip-feed enshittify your stack. Increasing tolerance for illegal behavior is downstream of this: you cut enforcement, tell them to automate more, they become easier to bypass... and you make more money. Because it turns out all that risky behavior... is only risky for normal people. Facebook can launder[1] the risk away and make a tidy profit.
[0] Do not tell me about CDA 230, I was there when the old magic was written.
[1] See also, HSBC turning themselves into a literal money laundering operation.
Zuckerberg needs to be arrested and imprisoned. These sociopaths aren't going to understand right and wrong on their own. It requires an intervention.
So you throw theZuck in the hoosegow. Now what? FB will continue printing money and doing the things it always has even if under new management. Why do you think it'll change with just throw theZuck in jail?
Court monitor that ensures compliance. It's not that hard.
Presumably the new CEO would like to stay out of jail and will fix things?
(This reminds me of, and not to be political. This is a technological, rather than political angle) There's a TED talk by Carole Cadwalladr about "Facebook's role in Brexit" in which she talks about the ads the Leave campaign ran. This is what is meant by Facebook's "breaking democracy", where campaigns don't have to go through the usual disclosure avenues
Are there reasons why Brexit might have been popular? Yes, in the sense I've heard about them. Should we then be blaming Facebook?
The problem is consolidated power. Carole herself said in her talk that it's not about Trump vs no Trump (sorry. It's been a bit since I've seen the talk. I shouldn't just paraphrase), it's about citizens' right to meaningfully choose in an information landscape
This is the problem with Facebook. It becomes an Everything App. Who cares about fraud (say, on marketplace, needless to say the site itself, the way it's screwed over businesses) on the platform when you can grow big? Let's say in this case, the actual case I'm commenting on - "Facebook didn't know". Well then that's the problem, isn't it? It's a company set up by someone of low moral reasoning (https://news.ycombinator.com/item?id=1692122) that doesn't set out to make a good product, something people will actually like. It's about dominating an industry and ads can be totally unvetted or scummy or scammy if it enables them to do this
Shut it all down.