> As we think about our role at Cloudflare in this developing market, it's not about protecting the status quo but instead helping catalyze a better business model for the future of Internet content creation. That means creating a level playing field. Ideally there should be lots of AI companies, large and small, and lots of content creators, large and small.
Not mentioned: there would be a single gatekeeper for the internet, Cloudflare.
The "level playing field" rhetoric reminds me so much of Apple talking about the App Store. This new internet business model is just the App Store, substituting websites for apps and Cloudflare for Apple. The system only works with some middleman between the AI companies and the content creators.
I see what you mean, and I am divided on the issue. On one hand, I don't think it's fair to have AI companies to freely scrape the world without fairly compensating content producers. On the other hand, adding more gatekeepers to the web ends up killing it.
Cloudflare is most active in pushing standards, and highlighting the issue of bots scraping web pages for free. That said, the other major CDNs (e.g. Akamai, Fastly) also have Bot Management functionality, so hopefully this is not a gatekeeping scenario, more of a standards-building scenario.
Are you implying it's going to fail in its goal as a competing browser, or that the competition is somehow irrelevant in a world that's forced to be slave to Google/Apple?
I'm implying that Ladybird will never grow to the size where it will have any influence on the internet. So this donation is a token gesture to making things better and will be unlikely to change anything.
I dont think Ladybird will ever surpass Firefox and to have any impact it would need to be at least as popular as firefox.
Even if you end up being right, which is not outside of the realm of possibility, it's still huge to get a boost on breaking the browser duopoly while ironing out longstanding issues with web standards.
> Not mentioned: there would be a single gatekeeper for the internet, Cloudflare.
Nothing in their idea challenges the underlying tech behind the internet. Anyone is free to compete in constructing a reverse proxy service with LLM-centric content controls similar to cloudflare, whether that’s AWS WAF or akamai or some new startup.
Nothing in Google's search monopoly challenges the underlying tech behind the internet. Anyone is free to complete...
From the stats I've seen, Cloudflare has an 80% market share for reverse proxy services. 20% of all websites use Cloudflare, 50% of the most popular websites globally. That's a dangerous amount of concentration, and it's the only reason Cloudflare can propose this new business model for the internet and be taken seriously.
Google’s monopoly is dangerous because they linked success in search to dominance in other areas, and especially the most popular web browser.
I wouldn’t recommend trusting any large company but so far Cloudflare doesn’t appear to be pulling a Google because they sell directly rather than to third parties. Google never charged for search so they ended up doing a reverse acquisition into DoubleClick to get advertisers to pay for the searches we do. Cloudflare does have a free tier but their paid services are decidedly not free and since they have serious competition in the CDN business, zero-trust, etc. they have the direct incentive not to screw their customers which Google lacked. I’d get worried if that ever changes.
> Google’s monopoly is dangerous because they linked success in search to dominance in other areas
That's precisely what's happening here: Cloudflare is leveraging its CDN dominance to become a kind of payment processor for the internet.
> they have serious competition in the CDN business
Do they? I just said they have 80% market share.
> they have the direct incentive not to screw their customers which Google lacked
Google Search is free service for users but a paid service for advertisers. The advertisers are Google's customers. Theoretically, Google has an incentive not to screw its customers, but practically they can, because of their search monopoly.
> Cloudflare is leveraging its CDN dominance to become a kind of payment processor for the internet
Seems like the moment is ripe for this move. In recent news Google partnered with stablecoins and traditional payment processors to create an agent-to-agent micropayment system (AP2)
> Nothing in Google's search monopoly challenges the underlying tech behind the internet. Anyone is free to compete...
But it's true? It's still true today. The only worrying part of the story is that google also makes browser and OS, which doesn't apply to Cloudflare.
The above comparison to App Store is even weirder / more ridiculous. App devs publish on App Store because App Store is pre-installed on every iPhone already, so it maximizes the number of users they can reach. Websites use Cloudflare to protect themselves, at the cost of reducing the number of users they can reach. The two situations are so different that "false equivalence" is an understatement.
> App devs publish on App Store because App Store is pre-installed on every iPhone already, so it maximizes the number of users they can reach.
This seems like a weird statement, because App Store is the only way of publishing apps on iPhone. The statement might make sense if you were talking about the Mac, on which App Store is pre-installed, but developers can still publish outside the App Store.
> Websites use Cloudflare to protect themselves, at the cost of reducing the number of users they can reach.
How does Cloudflare reduce the number of users they can reach?
So what? I should hate them for that? Cloudflare is really good at what it does. Nobody has to use cloudflare, but people who know what they’re doing choose cloudflare because the service they provide is worth the minuscule price they charge and it solves the massive abuse and performance problems that otherwise plague the internet.
Bing/msn.com failed to displace Google because Google was simply better, not because Google played dirty.
Whenever anyone says "Oh so I should hate company X" you know you're in for a bad, stilted argument that's going to defend some very narrowly defined thing.
Companies, by their nature, grow and take over things for the purposes of making money - tech companies moreso. We have seen companies overstep in the past.
Please don't stifle reasonable concerns with this sort of inflammatory rhetoric.
There's nothing reasonable about your concern. My original reply two levels up explains why: they have competitors and the service they provide is highly fungible.
Where did I mention hate? I don't care what emotions you feel or don't feel. The problem is the concentration of power in one company. That has nothing to do with emotion.
> Bing/msn.com failed to displace Google because Google was simply better, not because Google played dirty.
I don't think the courts agree with you about Google playing dirty. In any case, monopolies are inherently dangerous.
Substitute whatever adjective you want. You're spreading FUD about the cloudflare boogeyman while ignoring the fact that they have well funded competitors and have no technical mechanism whereby they could lock anyone into their so-called reverse proxy monopoly.
It could be interesting to build a small startup that identifies hate speech on Twitter, Threads, Blue Sky, and other platforms.
I envision a UI that displays the message and, in a sidebar, lists what aspects of the message classify it as hate speech. Then, like a spam filter, you could decide to block the message.
How would that work short of them buying most of the fiber in the world? A natural monopoly is a big problem for ISPs but they don’t do last-mile and they don’t appear to be anywhere close to being able to manipulate thr transit market.
Also weird that they think reddit is something other than ragebait. I avoid reddit as much as I can, because everytime I go there the reddit feed is full of some of the worst ragebait on the internet.
> Not mentioned: there would be a single gatekeeper for the internet, Cloudflare.
Cloudflare is already a monopoly though. From what I can tell, what they are saying here, besides proclaiming their continued existence, is that they and AI companies and content creators need the internet to exist.
And what they are building is on top of the 402 response, which anyone can use? So you could use that without using any CDN at all, without too much development cost?
They are all platforms: news companies, publishing houses, ISPs, recording labels, YouTube, Google, Meta, TikTok, Windows, App Store, Play Store, Amazon.. and as platforms they squeeze both the layer above and the one below, their customers and their providers.
You know what that looks like? rent. It looks like home owners setting rent as high as the market can bear, until it becomes almost no advantage at all to participate.
It sure looks like Cloudflare wants to be the platform for AI royalty collections. But creators, customers, sellers - they are all hostage to the platforms they can't avoid. That is where the money sink is.
Sorry, not exactly monopoly. Probably too big to fail, from what I gather, though. I don't know enough to know whether there is anyone else that could realistically compete with their scale.
My point was simply that their size is unrelated to this proposition, outside the fact they are big enough to be taken seriously.
Is there currently a more open alternative to Cloudflare? I would assume that people don't really use it until they have significant traffic from all different parts of the world?
In which case for the self-host people we can just pick a decent CDN?
I've done a few tests with CF hosted webapps... overall it's pretty good experience, and for my usage has been free, despite opting into a paid account level, I haven't exceeded the free usage.
For mostly-static or static-site content it's pretty nice all around. I've not gotten into the SQLite service so much though, which I might and seems interesting.. there's also Turso to consider as an alternative option... Not to mention Deno, Fastly and other similar options.
Kind of hoping to see something similar start to gain traction to support wasm backend systems similarly.
This is beyond their DDoS/Proxy protection, but worth considering as part of what they offer. There's a lot there to like.
> The "level playing field" rhetoric reminds me so much of Apple talking about the App Store.
To be fair to apple, many billion dollar businesses have been built for and distributed by the app store. Think about all people employed to work on mobile apps, all those people going to local taco truck for lunch owned by an immigrant, that immigrant buying a nice christmas present for their kid, etc.
> To be fair to apple, many billion dollar businesses have been built for and distributed by the app store.
This sentence is missing something essential:
Many billion dollar businesses have been built for iPhone and distributed by the app store.
iPhone is the platform, whereas App Store is merely the exclusive method of software distribution as dictated by Apple. Contrast with the Apple Mac, where the App Store is not the exclusive method of software distribution.
"Let's give money to the content creators" is exactly what big incumbents want. They want to farm you, the viewer, for big profits, and give the content creators a few pennies in return. Meanwhile they hoover up the lion's share of the profit. We've seen this over and over again. It's rent-seeking.
"Content creators" are part of the problem. Generating endless "content", which isn't very useful or valuable, and creates too much "content supply", which devalues it. This then creates a giant "soup" of content that viewers drown in, trying to find some content that isn't as identically useless as all the rest. But the big incumbents love it, because they use this content soup to collect money from - you guessed it - ad companies. (Those same companies they don't want to make ad-blockers against...)
So now that search is dead, they need to find a new way to drink from the ad-dollar faucet. At first it'll be "pay content creators from AI subscription money", but then AI will be offered "free with ads", and then later "paid with ads". And nobody's going to stop it, because everyone is "happy": the viewer gets their free crap (with ads), the content creators get a few pennies, the big companies rake in billions, and ad companies continue their "industrial welfare" by pouring their excess profits into this whole system as ad-dollars. The great commoditization of eyeballs continues.
Had a similar thought, yeah, Spotify is like this. I don't know what the numbers are, but a very, very large majority of musicians make very little money off of Spotify (Weird Al Yankovic said he made enough in a year to buy dinner for himself).
On the other hand, Youtube is pretty good. You can make some money off of youtube viewings, but what it's really great for is building an audience. Musicians know that Youtube is a good tool for building a successful music career.
As for devaluing content with oversaturation, yeah, agree with this. Ironically, media is both the bread and butter of the internet, but it has also become devalued with the shear amount of it. Still, this isn't a problem of the media companies. Content creators can either chose to engage and or not, and hope they breakthrough.
And you kind of said it, "The great commoditization of eyeballs continues." Money makes the world go round, and nothing is free. If we want a vibrant internet, companies have got to find a way to make money (although, monopolies aren't a good thing, but paying for good content and services seems like a good idea).
Btw, to all that are reading this, actually, found out the that Weird Al's music makes a lot more from Spotify than buying himself a sandwich - His rights holders get paid by spotify. These middlemen then take a cut and pay Weird Al. But on top of this, since Weird Al does parodies, the musicians he parodies also take a share (Michael Jackson, Nirvana...).
A question: Why do musicians put their music on Spotify if they make so little of it.
If the answer is that it's the labels who owns the rights and puts the music there, then the next question is why do the labels do it if Spotify pays so little.
If the answer is that Spotify pays decently to the labels, then isn't all of this just the labels exploiting the artists as usual?
Am not an expert at Spotify, but for at least the musicians i know, they self-publish these days, so it's really not about the labels. For us, Spotify is just another way of getting our music out there. Yeah, we can build a very small audience who'll follow our music, but more over, it's just a way to point interested producers, venues, musicians or listeners to our music.
And actually Spotify hasn't been profitable until 2024. It seems like the problem is streaming music is not a great business model. They pay a tons to labels and (big name) artists, but only get a set subscription fee from its listeners.
It's a the lesser of two evils, because without it, people would go back to bittorrenting music or using things like the Apple Store, where you don't have to buy an entire album, you can just buy only the songs you like for a couple bucks.
While I'm partial to your cynic interpretation — after all Cloudlfare is a for-profit US business —, the model isn't that far from what Jaron Lanier had proposed in ’Who Owns The Future?‘ (2010).
In the book he explains how these micropayments would be sent out to all the websites you visit, etc. And you can have a built-in mechanism to combat the "star system" that tech automatically enables, where a few people get the most money.
Lanier's proposal was that ISPs take charge of this issue, but perhaps Cloudflare can make it work. The less cynical interpretation would be that later they federate the system or even turn into a non-profit.
I don't think every website you visit should be paid. Who says this random website even deserves payment? This would incentivize people to use tricks to redirect traffic to their websites (which already happens..). Imagine a world where, in order to step into a retail store and browse the merchandise, you had to pay $0.25. Payment systems are already a nightmare, this would add headaches. And just because it's a non-profit or federated doesn't mean anything; show me a non-profit and I'll show you a hierarchical organization with a rich board, poor workers, and problematic outcomes.
People need to think long and hard about why they think there should be unlimited stuff on the internet. Have you looked around? It's mostly crap. Random scribblings by opinionated amateurs, exhibitionists shilling snake-oil, "products" like online games or unlimited videos designed to sell your personal information for a profit, "news" that is 100% gossip, and an unlimited number of echo chambers. We need less incentive for all this crap, not more. But ads are never going away, so it's moot.
Everyone seems to be dismissing this without reading it. As a content publisher, I am very interested in any proposal that results in me getting residual payments from AI scrapers. That would indeed be a new internet business model.
This is also being attempted by RSL with their “Crawler Authentication Protocol” (https://rslstandard.org/guide/web-crawlers) for demanding proof of licensing from scrapers and RSL Collective (https://rslcollective.org) for providing the licensing itself. The missing piece there is the ability to detect scrapers with high accuracy without punishing regular browsing humans.
As an engineer who works at a medium-large publishing company, am also interested in this direction.
I have fully read Cloudflare's proposal, as I have RSL, and L402.
In its current state, RSL is not a full solution, but can be. The spec does not cover payments. Its basically just a new token type, which crawlers would register for to crawl your content, and predefined responses for more detailed states. But nothing around payments, yet.
Cloudflare's solution is basically the same as L402. This is very sad for me, because their attempt is basically one of locking down the ecosystem in order to create a monopoly.
If they were really about creating a fair internet where "pay per crawl" is possible (it already was with L402), they would openly support L402, implement it for their gateway, and try to push the industry towards it. But no, they just want the biggest piece of the pie; a pie that doesn't even exist yet.
As a content consumer, I'm also hoping to be part of the ecosystem. I already use Patreon a lot as "AdBlock absolution", but it doesn't fix the market dynamics. Major content platforms tend to stagnate or worsen over time, because they prefer to sell impressions to advertisers than a good product to consumers.
I'm really excited by this post. I love the vision. especially the vision of using what is already known about the gaps and what is not known in having content created to fill those gaps. There are definitely some gaps in LLM knowledge.
It always bothers me a little when people describe themselves as a content publisher or content creator. Why content? Why not say you are a short film maker or writer or musician or videographer or journalist or painter or photographer or ...?
In the past, your work has been merely content to the people who are looking for bits -- any bits -- to place between and around the stuff they really care about: ads. These days I suppose your work may be seen by some as grist for the AI mill.
By calling your work content you are acceding to the idea that your work is only valuable in other contexts. It isn't! I bet it's something you made with care and thought.
Or maybe I misread and you really are a content publisher. You're the person running the CMS who is looking for other people's work to monetize. If so, nevermind.
As a philosopher, I'd guess it's frequently because more concrete terms like "blogger" or "forum commenter" or "video game player" somehow lend even less credibility than the generic terminology.
I think as a publisher, you should be worried that Cloudflare is taking the first step towards the 2025 version of what “rights holders” have done to the movie and entertainment industry.
Imagine trying to create and distribute a movie without the backing of your local distributor/broadcasting cartel. Only instead of gating the movie theatre, Cloudflare is the single access to all things web. Also, based in the US, so good luck producing content their government doesn’t like.
I like the IDEA of residual payments for content on paper. Brave browser tried this, but it didn't get enough traction from publishers of note to be viable. But I think, at least in theory, the idea of me as a consumer having some money attached to my eyeballs for the person/people who created the content I'm seeking out is a good one. Paywalls are annoying and awful, and unfortunately they seem to encourage data hording and selling practices that are unsavory. But I like the model of simply opening the browser, visiting a page, and the creators getting some royalty from that. The system we have now is too exploitative - it extracts more value that it creates.
I mean, you shouldn't need to read a wall of text to understand what a business model is.
It should be front and center, communicated clearly, and easy to understand. If I was an investor, the lack of clarity after reading a few paragraphs would concern me.
It definitely isn't an aerated Linkedin post full of single-sentence paragraphs and emoji, but it's also certainly not a "wall of text". It's what we used to call a "blog post", it has very readable paragraphs broken up into short titled sections.
> As a content publisher, I am very interested in any proposal that results in me getting residual payments from AI scrapers.
So you use a public resource and presumably like the upside that comes with sharing on it, but you want to limit uses now that someone has found one that you don't like, so you're fine with degrading that public resource?
You could always share things privately or behind a paywall if you don't want them available publicly. But people seem to want to have their cake and eat it too.
I get why a hosting provider would want to limit crawlers to save bandwidth. The "creator" angle is just greed.
> So you use a public resource and presumably like the upside that comes with sharing on it, but you want to limit uses now that someone has found one that you don't like, so you're fine with degrading that public resource?
Can you say more? what is the "public resource" I'm using as a content publisher? In that role I see myself more as the provider of a public resource (my content), not a user.
Your point is that pepole leaving some produce on the side of the road along with a money box should accept that some people loot all the produce in a 100km radius without paying, and resell it?
You can put images up on a public website, but you still retain copyright & can control the images' use. This is no different from any kind of digital content...
And Cloudflare is trying to legitimise a “business model” resting on the assumption that it’s ok for AI vealers to scrape content in the first place. “Opt-in” is not in Silicon Valley vocabulary.
I read that differently: we’re starting from the current world where anyone publishing online is seeing expensive traffic volumes from companies whose goal is not to send user traffic back. Cloudflare seem to be arguing that site owners should easily be able to charge for agent access independently from regular user access, but since that is always under the control of the site owner it seems like the opposite of your “opt-in” complaint.
As a musician, yeah, don't agree with this. It'd be great to find a way practical way to get content creators compensated, because with the internet, almost all media is becoming free (or near free), and it's hard to survive.
And, the idea being proposed is a version of what you're talking about (paywall). Not allowing crawlers to scrap your site unless they compensate you is a form of paywall, just for crawlers...
...But, I do agree with one aspect you're saying about greed, Cloudflare is probably being self serving in this. An open standard seems like it'd be a better solution than having one company control content payment.
I hope that the next big shift will be to undo the asymmetry that crept into the internet quickly after it first became popular. Let us host stuff at home. Let us run odd and strange and great systems wherever we are. Undo the cloud, undo the capture of the net – I'm old enough to remember when we just had a bunch of boxen under a desk somewhere, and it was pretty great.
I've been thinking lately how nice it was when the Internet had lots of phpBB-like forums spread around. Each one ran by a different server with it's own rules. None of them required your ID or phone number to join. No one really knew who anyone else was in real life. If you got tired of the people one forum you had several similar forums on different sites you could go to. It was fun seeing the kinds of things people would share in that environment. Everyone had the power to be seen, heard and disregarded in a short span of time.
I think of these things like they only exist in the past but I'm sure these places are still around only I don't go to them as much now. Probably because all the useful stuff these days exist in mainly reddit and the stackexchanges.
I too miss the days of phpBB an IPB forums. Circle.so has come the closest to replicating some of that interface / vibe, but as expected, their pricing has crept up beyond a reasonable point for small projects / communities. The answer seems to be something self hosted.
The problem is always discovery. There are lots and lots of little sites out there but without a search engine or directory you don’t know about them. Even back in the day Yahoo had a directory. And once you need such an aggregator, it will naturally create a choke point.
You can still do this, it all still works, technically. You will have a spam/malware/security issues that wouldn't have been an issue back in the day. You will also have discoverability issues - but that hasn't actually changed, if it's just for you or your friends.
This is one of the core usecases of cloudflared (https://github.com/cloudflare/cloudflared)! Without having a static IP address or dynamic DNS, you can establish a connection between your machine in a private network with Cloudflare. And once a resource is on the Cloudflare network, you can pretty much do anything with it, including routing your visitors to that server and using the full Cloudflare stack including its CDN, firewalls, and Anti-Bot protections.
This project does exactly the opposite of what the GP was talking about. We need to selfhost to be independent of Cloudflare, not dive even deeper into it. cloudflared exists to embrace and extinguish and absolutely fuck that and everything that comes from Cloudflare.
Don't just dream. We can choose to use the ISP that says "yes", or at least a less firm "no". We can vote with our wallets to create a market opening for the ISP that says "yes".
In general, most ISP have a usage agreement posted which prohibits excessive client server traffic.
This is why most NOCs enforce an asymmetric firewall bandwidth limit for download focused customers, and install colo Google/CDN rack appliances.
Most ISP are just the modern retooled Cable company business. Try to stream video off your home platform on a normal service port, and you will hit the caps pretty quickly. =3
> Unless you believe that content creators should work for free, or that they are somehow not needed anymore — both of which are naive assumptions
Realistically, what was the benefit of the ad-driven model? From my point of view, most of the highly-valuable information came from the various forums, wikis and personal sites, on all of which people would publish the information for free. Ads were largely used to cover hosting costs of the large forums and wikis, but the content creators saw not a single dime.
Over the last 15 years, the search results have shifted from this, to instead become 100% news articles and SEO spam, all with a lot of fluff and very little substance, tons of ads, pop-ups, autoplay videos and subscription walls. All of this is well funded thanks to the business model, but for what? I can't imagine anyone sitting through all that crap to get to what they were looking for. There's a reason people would tell others to add "reddit" to the search query only a few years back, and even that's becoming less worthwhile.
Is this really what we want to preserve? Big publications and their interests?
> Realistically, what was the benefit of the ad-driven model? ...Is this really what we want to preserve? Big publications and their interests?
They are proposing a scheme by which AI scrapers would have to pay for the content they scrape, which could replace the ad-driven model and be viable for more creators.
After AI scrape the ad free content, they'll probably turn around and show end users the AI summary with Ads. It never ends. The problem is a much deeper flaw at the heart of capitalism. Enshitification of everything.
> We have a pretty good mathematical representation of human knowledge...
> Swiss Cheese — there's a lot of cheese, but there's also a lot of holes.
Such hubris! Human knowledge is not like Swiss (mostly known with a few gaps). Human knowledge is like a high-dimensional fractal broccoli: a stem of hard-won knowledge surrounded by a vast frontier of unknown. The hardest part of pushing on the frontier is deciding where to push. Deciding what to research requires taste.
In short: we'll have the machines tell the creators what to create.
> You could imagine an AI company suggesting back to creators that they need more created about topics they may not have enough content about. Say, for example, the carrying capacity of unladened swallows because they know their subscribers of a certain age and proclivity are always looking for answers about that topic. The very pruning algorithms the AI companies use today form a roadmap for what content is worth enough to not be pruned but paid for.
Valid point / not a bad idea. One flaw might be that (successful and high quality) content creators are accustomed outsized rewards for their work. Ads pay very, very well for people creating content viewed by a lot of people.
Would AI companies be willing to match that (even 50% match)? If not, we might just end up with low paid copyrighters / ghost writers churning out large amounts of content for LLMs in subject areas where they don't necessarily have expertise.
I think Patreon-style backing has been proving out the viability of payment based creators. Technology Connections is largely funded by Patreon but puts out free content all the time. My guess is that once x402 becomes more standardized that Youtube, Substack, and other SaaS can add in paywall functionality which can help fund the free content for creators.
Creating something on demand instead of growing it autonomously already made everything non-authentic for the most part. Automating this would make everything totally fake and homogenized, and also devalue the data in the process. "Regularization" and filling gaps is a mistake, they should reward the originality instead.
"Make no mistake, the Internet has never been free."
Honestly, this sounds like a Mozilla press release. The spin is absurd
Before ISPs existed, someone else paid. Universities, governments, selected corporations, etc.
After ISPs (post-UUNet), internet subscribers have always had to pay
Who finances the data collection, i.e., the transfer of data from/about subscribers to the so-called "tech" company for commercial exploitation. Hint: It is not the so-called "tech" companies
When I first accessed the internet, and later the www, the term "creators" was never used to describe people who used it
Methinks "creators" is SillyCon Valley speak for "unpaid independent contractors". Why not use the term "producers"
The gigantic websites comprised of other peoples' work, today's so-called "platforms", are not "creators", they produce nothing. Despite billions of dollars at their disposal, and vast amounts of published material, their production costs are small. This is because production costs are outsourced to unpaid independent contractors. The material the "platforms" publish does not belong to them, they are only third party intermediaries (middlemen)
In the original internet, people publishing on the internet via websites produced the website content themselves
Today, "platforms" are not creating anything except a baited trap for people using the internet, whether they are "creators" (producers) or consumers, where internet users can be spied on for advertising purposes. And the targets of this surveillance and subsequent targeted advertising will even pay for sending the surveillance data collected to the interlopers
Today's "creators" may be paid (if they are lucky, like winning a lottery) or not (the other 99%, their free contributions serve as bait). But "make no mistake", it is not the third party intermediary "platforms" that pay "creators", it is advertisers, but only after the "platforms" (middlemen) take a cut
SillyCon Valley calls this an "ecosystem". But today's internet, cf. the original one, IMHO looks more like a toxic waste dump
Something that stuck out on this was the notion of the LLMs as models of human knowledge, and the notion that we could "see the gaps". There's been an interesting debate* in journalism over the concept of neutrality - what it means to "just present the facts," and whether that's actually a useful service, and what are actual facts, and how do you define a fact (it's gotten rather epistemological), and that's just for things that actually happened in the real world. I think what's not taken into account here is how much of the world is not able to be summarized into One Universal Truth - how much of what we look for is actually the preference of the author ("what are the best new albums this year?" "how do I cook chicken?"), or an intellectual synthesis of a subject (basically anything written by that acoup guy), or just some entertainment or random digression on a topic. I think this is part of the crux of the argument behind the artists vs LLMs debate (you know, aside from all the economic exploitation) - even the act of writing a summary is a creative act when performed by a human that generates something new, whereas an LLM is, as far as we can tell from both the mechanics and the actual output, a remix of existing content. I think our appetite for the latter is less than we think it is.
I don't fundamentally think that Cloudflare's view here is _wrong_ - by and large, when I google what internal temperature to cook pork to so I don't die, I'm not looking for an opinion or someone's life story - but I think I'm much more interested in how we create the weird niches that create the "knowledge" for the blender. Cloudflare alludes to it with the talk about Reddit and whatnot, but I'd love to know what the plan is to actually create and nurture those communities where people can really get weird into whatever topic they're interested in. Right now we're all sort of just ghettoized into various Facebook communities or whatever, but recreating an actual vibrant communal internet where people can find their weirdo subcultures and actually negotiate on some kind of reasonable footing with the LLM scrapers (who were the social network people, who were the search engine people, who have always been the ad people) would be a genuine improvement to the internet.
That discussion also makes me worry that they may try to use LLMs or LLM-based metrics to measure the size of the gap as a proxy for value of the content.
The landlord of the marketplace should probably not dabble in the appraisal of products, whether for factuality or value.
> As a rough and simplistic sketch, think of it as some number of dollars per AI company’s monthly active users going into a collective pool to be distributed out to content creators based on what most fills in the holes in the cheese.
This is the old "blockchains will create new niches for creators" / "rent-seeking is Good Actually" argument, barely reheated. Total imagination bankruptcy from C-suite.
I think providers of so-called "answer engines" will read the writing on the wall and find new ways to support content creators in order to keep their databases fresh, relevant, and centered on human perspectives. I also think that to see how the economics will play out, you only need to look at similarly centralized systems like Apple's App Store and Google's AdSense.
Because the providers will act as a single tunnel that all content passes through before reaching the end user, the tolls they collect will be large. So, I don't doubt that there will still be opportunities for content creators to earn money as answer engines siphon off more and more of the web's traffic, but expect those opportunities to be broadly low-paying, falling decidedly in the "side hustle" category.
AI providers will want to incentivize content creation. There will still be a glut of ready providers, and little reason for providers to make anything but small, nominal payments.
I don't know if this is going to work, but at least they are skating to where the puck is going to be.
The new model will be something like:
1. A content creator creates a web site and uses Cloudflare.
2. AI companies pay Cloudflare to allow them to scrape content.
3. Cloudflare gives a cut to the content creator.
4. Users pay AI companies and get their questions answered.
A few observations/predictions:
* If this works, there will be competitors to Cloudflare (AWS, Microsoft, etc.) who will offer better terms to content creators. Content creators can then (easily) switch to whichever reverse-proxy has the best terms.
* Media companies will transform into Cloudflare competitors, aggregating content and monetizing by selling to AI. Their pitch will be that the content will be more curated than Cloudflare. Their brands might survive if the AIs pass the source of the content all the way to the user. For example, the AI says something like, "According to a BBC contributor....". Otherwise, media brands will no longer be known to consumers (only AI companies will care).
* If this works, AI companies will try to cut out the middle-man by building their own ecosystem of content creators.
* As more and more people get their answers directly from AI, it will be easier to sell content directly to AI companies. I.e., instead of publishing something on the open web and relying on Robots.txt to protect your content, you will sell content straight to the AI company. NOTE: If this happens, then the only way this will scale is if the AI itself decides which content it wants to buy for the next training run.
* At the limit, the web and everything about it basically disappears. Everyone gets their content directly from an AI and never visits a web site directly. Therefore, web sites disappear and all that's left is the HTTP protocol, which is used by AI clients to talk to the AI cloud.
Charge site owners to protect the site from aggressive crawlers, charge crawlers/agents to bypass the protection. Like a good old fashioned protection racket
Paying a security guard isn't considered a protection racket, while paying a member of the mob so that nothing happens to my store is considered a protection racket.
Cloudflare has controls already in their dashboard for controlling whether LLMs should be denied responses when querying your site. How they intend to broker payments and selective access isn’t really clear, but you can stop giving your content away for free if you’d like.
Yeah, I was waiting for "how are they going to shove AI into this?" and boom there it is halfway down the page. No content here (obvious problem: how does paying Reddit Inc to allow scraping result in money going to the person who created the content? shrug). Just more AI hype to get investor cash before the bubble bursts. Woopie.
Edit: Re-read it and just noticed this:
> assuming the AI companies are willing to step up, support the ecosystem, and pay for the content that is the most valuable to them
Lolllllll. Y'all are gonna love my pitch for an airline startup. It begins, "assuming we can turn off gravity..."
If Cloudflare demonstrates it can reliably distinguish AI scraper traffic from human traffic, then they will be able to force AI companies to step up, no assumption necessary. Still a big if, but (this one clause you quoted notwithstanding) it doesn't read to me like they're relying on the goodwill of AI companies.
I think they all know where it is heading and it is massively less money for everyone. Every technical breakthrough doesn't mean more profit for everyone and nor should it. Sometimes it's just a dishwasher or washing machine, something that frees up the bulk of humanity from something that is drudgery. Whether we actually get those benefits from AI remains to be seen.
Its cloudflare trying to enshittify the internet with micro transactions[0] and take their N% cut (of course it will start at like 2% but ask any uber driver how thats going).
The problem is the arguments they make for why this should happen are quite compelling, especially to those running sites (you'll see plenty of complaints on this forum about it), but theres also a large group of people who think information / code / data should be "free" (see open source code/maps/anything you can think off). So really its just a moral debate that will be lost in the interest of profit (which is ya know good n bad, if AI companies did more caching we probably wouldn't need this, but here we are).
Micropayments aren't hard. Customers simply don't want to use them. Tons of newspapers and magazines have tried offering single articles for sale for $1 or 50c, and all have failed. People just don't want to use micro payments. They want to pay a monthly subscription.
The only hard thing about micropayments is that "payments" is a worldwide monopoly and the incumbents don't want it to exist.
Now, the problem with micropayments gated sites is that you are trying to gate the exact thing that attracts people, so don't expect this business model to work.
It would like exactly how it is today. It's not hard to find someone not charging anything for the same piece of content, or not charging anything for another type of content that fills the same void.
using conventional methods small transactions are eaten by service fees. This means microtransactions require a different way to pay which is friction and people are apathetic especially when what're you doing is trying to get them to pay for things.
As a researcher, I like the notion that there could be some incentive or mechanism to create new and original content, investigate new things, or conduct research in general. I'll group all of those under the umbrella of new content. I don't think things will work out as ideally as described here, but I can appreciate the sentiment and hope that it works out positively.
That said, the mental model that this article uses only recognizes that new content can fill in "holes" (interpolation). It also can expand the boundaries in new directions (extrapolation). That is a different and harder problem. If you distribute money to people "based on what most fills in the holes in the cheese", you really aren't expanding the boundaries of human knowledge as much as you are strengthening existing knowledge. They need to take boundary expansion into account here.
I also recognize that we know where the holes or boundaries are in many fields. They are "known unknowns". But this proposal does not take into account "unknown unknowns" --- things that we do not even realize that we don't know yet. It's going to be harder to incentive research into unknown unknowns when we don't even know what they are yet.
> why did Cloudflare never build an ad blocker despite many requests?
Many of us who use an ad blocker don't actually have a huge problem with ads. It's the tracking we object to.
I don't want Cloudflare to build an ad blocker, I want them to make a tracker blocker. Make surveillance so difficult and expensive and unreliable that companies stop doing it.
Unfortunately ads and tracking go hand in hand: without tracking, targeted ads are much less effective. If everything else stays the same, you'll just get more, bigger, and flashing ads to compensate for the inefficiency, like what you see today on smaller sites without sophisticated user data.
As an user of the internet, I wish cloudflare didn’t exist. They are in most sites I visit, they add a couple of seconds of delay always. They can shut down entire ranges of ips if the government says so (no matter if some sites hosted in such ranges are legitimate).
It sucks big time to live in an internet with cloudflare
Every website which gets a little bit of traffic or reputation will be hit with massive DDoS attacks. Your option as a webmaster is to use something like Cloudflare, or your content will become unavailable to everyone.
You can block all websites using Cloudflare if you think it sucks.
DDOS mitigation costs are coming down. If you're running a decently sized service and dont have any enemies that want to pay constant rates to shut you down then _should_ be fine. You can even temporarily use cloudflare if you're getting hit particularly hard then remove it when it passes. 100% uptime is a meme and not required for most websites.
Given CloudFlare's direction, I won't be surprised if they start offering "anti adblock as a service" soon. Detecting it is still rather simple, and add in some punishments such as temporary IP blocking, and it would be rather effective.
(Note that everything CloudFlare talked about in this blog post also applies to adblock users, not just AI agents.)
The golden age of the Internet is not where people do it for money, or for views. That way lies clickbait and content farms. The golden age of the Internet is one where people share information because they want to.
I've probably spent 400+ hours a year sharing content because I wanted to. However the wanted to, was partially because of the good feeling i got from knowing I was helping semi-specific people and then later meeting those people. It's sparked numerous interesting friendships and discussions. That latter part will no longer happen as the AI sits in the middle and becomes the known source. It has massively put me off creating more content. I wasn't creating content to generically help humanity move forward, I did it to help people similar to me facing particular situations. As that becomes diluted, the incentives won't hold to the same degree.
I recently listened to the interview between Ben Thompson and Cloudflare's CEO Matthew Prince and couldn't shake the feeling of a company that tries hard to repeatedly hit gold with the business model of being the gatekeeper of the plumbing of the web, with the promise of "the world out there is very dangerous".
The only thing I see happening is agreements between the biggest players and the content brokers, squeezing out others who’d like to enter. At least just now anyone can theoretically scrape the web for useful info.
The bigger problem I see coming down the line, given the current business model for the internet, is what happens when investors need a return. When the LLM only provides one “correct” answer, the only way to make money from advertisers is to influence that answer. At least just now we are presented with information which we evaluate ourselves, though even this is subject to manipulation.
"With some things there's been clear progress: when we launched in 2010 less than 10 percent of the Internet was encrypted, today well over 95 percent is encrypted. We're proud of the role we played in making that happen."
Yeah right, most of it is now proxied in cleartext through them!
I don't think they actually know what the future business model is. No one really knows. What people are presenting is iterative models. That's fine but the model will be quite different. Ok we'll need subscriptions and to pay creators but I believe if everything is crypto and stablecoin based it's all going to be pay per view or pay per query because you know a lot of people just want that specific content from a specific source because of reputation or because that's what they like. I'm not paying for 5 different agents for that bespoke experience just like I'm not paying for NYT, WSJ and multiple other publications because it is insanity to price up what should effectively be pay per article. So maybe on the backend creators get a royalty kick back just like the music industry but on the consumer side I definitely think beyond subscriptions with the advent of crypto wallets we're going towards micro transactions for everything.
Ok the the toxic nature of the internet and social media in general and what has become of our digital age, totally agree that it's rage and click bait. I wrote something to that effect here https://github.com/micro/mu/issues/27. But I personally don't think we're going to directly interact with agents the whole time. They will exist, they will be somewhere in the middle layers, there might even be a chat interface that replaces search queries with answers but I think the whole web as a whole and social media needs a rethink. Ads as a business model has to die, even though clearly it won't and we need to shift our attention elsewhere.
I like that they are considering this problem. I really dont like what they have decided to focus on. Having 'answer engines' as the core with content creators filling in the gaps like some sort of task rabbit workers sounds horrible. I dont want gaps filled by people who are doing it for money. I want people that actually care enough to organically create the content.
The problem isnt oh no people wont be viewing ads and thus creators wont get paid. The problem is "why arent people who view these websites willing to pay?". So many subscriptions are like $5 a month but if we consider how much a creator is paid for showing you an ad its like 0.0005 cents. I'd rather pay that everytime i view a page than look at ads or have cloudflare pay people to create the content openAI thinks I want.
Funding the content we consume should be cheap considering how many people are there consuming it and how cheap it is these days to serve content.
Do people think of CF as a leader in terms of solutions that are "open, collaborative, standardized, and shared across many organizations"? My impression is that their open source work is mostly Cloudflare-specific client libraries and the occasional passion project from their engineers. Quiche may be a counter example, but it's a rare exception.
Examples:
Pingora claims to be battle-tested, but I have a hard time believing that it's to the same level of quality as whatever Cloudflare runs internally. https://github.com/cloudflare/pingora/issues/601
Small parts of Oxy were open sourced as "foundations" but the repo gives off the impression of a checkbox for someone rather than a serious commitment to building CF's own services on top of it — not "open, collaborative, standardized, and shared across many organizations".
> Make no mistake, the Internet has never been free. There's always been a reward system that transferred value from consumers to creators and, in doing so, filled the Internet with content. Had the Internet not had that reward system it wouldn't be nearly as vibrant as it is today.
This is an attempt to rewrite history. Back in the day, we stood up servers at our own expense and filled them with content for free, for nothing other than the fun of it. In fact, the "vibrancy" of the internet appears to be inversely correlated to the number of people using it to generate a profit.
They're not saying what you think either, there's no reward system or transfer of value in free content, offered for free. It doesn't matter if it's hosted at someone's own expense if it's done for nothing other than the fun of it.
I never implied there was a reward system or value transfer (unless you count the feel-good reward of sharing knowledge with others and the value of doing so).
I said that putting things on the Internet is not, and has never been, "free".
And I think that's what Cloudflare is saying too.
You're free to interpret it differently (as was OP) but I think you're wrong. ;-)
CloudFlare is definitely implying there's always been a reward systems and value transfer at play, that's there in the quoted text and what the OP is complaining about
So why can't websites, do like any other "store". They can make you agree to pay for something but not enforce payment. Like couldn't I say, here's my article. If you read it you agree to pay me $1. Now a lot of people might read it and it might not be worth my time to go and collect $1 from each of them, just like Walmart isn't going to tackle you about eating a grape in the produce aisle. But if you want to make a business that's about exploiting Walmart's non-enforcement of free grapes then they are going to stop you.
I guess this is the 'meat' of the article. Interesting that they regard reddit as a 'niche' quirky corner of the internet. To me they are well over the peak of their enshittification journey with mainly bots and overzealous mods and pay-to-play accounts. Just like digg and others before something else will take their place soon enough as the place where enthusiasts share valuable content before businesses poke their nose in and we repeat the cycle again.
> What's most interesting is what content companies are getting the best deals. It's not the ragebait headline writers. It's not the news organizations writing yet another take on what's going on in politics. It's not the spammy content farms full of drivel. Instead, it's Reddit and other quirky corners that best remind us of the Internet of old. For those of you old enough, think back to the Internet not of the last 15 years but of the last 35. We’ve lost some of what made that early Internet great, but there are indications that we might finally have the incentives to bring more of it back.
> It seems increasingly likely that in our future, AI-driven Internet — assuming the AI companies are willing to step up, support the ecosystem, and pay for the content that is the most valuable to them — it’s the creative, local, unique, original content that’ll be worth the most. And, if you’re like us, the thing you as an Internet consumer are craving more of is creative, local, unique, original content. And, it turns out, having talked with many of them, that’s the content that content creators are most excited to create.
I wonder if that’s a purposeful framing to avoid saying that content a large and diverse group of humans has created is very valuable. Think about how much money the Facebook data set would be worth to OpenAI. But cloudflare likely wants to avoid the privacy nightmare taking attention away from what they’re doing.
Monetization models that intentionally create scarcity are scummy, including the one proposed in the article. The existing intellectual property economy is scummy. But we can fix this.
Imagine a future where:
The Internet rewards passionate creators more than instrumental alpha-seekers. This is key. But how?
We publish everything for free under reciprocal licenses like the AGPL and CC-BY-SA, not permissive licenses like the MIT and CC-BY. Those who take from the pile must add to it, or at least not claim it as exclusively their own.
Creators are rewarded by the populace via direct payments, like how streamers are rewarded. People decide what's valuable and what's slop, not Big Brother.
This creates an incentive structure where:
Those who make software and content because they love it will have more resources to make more quality stuff.
Those who make software and content for the money but actually hate what they do will find something else to do because companies will pay them less.
There are two necessary conditions that must be met, that we can work on right now:
We recognize artificial scarcity as evil, so we can enforce reciprocal licenses socially and bypass the broken legal apparatus.
We use multiple competing payment platforms to pay creators and receive payments, so there is incentive for these platforms to get better.
> Imagine a future business model of the Internet that doesn't reward traffic-generating ragebait but instead rewards those content creators that help fill in the holes in our collective metaphorical cheese. That will involve some portion of the subscription fees AI companies collect, and some portion of the revenue from the ads they'll inevitably serve, going back to content creators who most enrich the collective knowledge.
Reminds me of that bit of Snow Crash, where the freelancers sell data to the Library of Congress.
My concern is Cloudflare will implement this plan in a way that makes it very profitable for large players and absolutely kills any new entrants to the market in the future.
I can’t help but feel we are headed for less sharing rather than more, and “private peering” between those with content vaults and those who wish to access it.
I read this as: ads have been great, and have helped to pay all content creators (even the small creators). But now, Answer Engines need certain content that's missing: and it's the largest content creators that are making deals with the Answer Engines.
What CF didn't address is how the smallest content creators are going to be "making deals" with the Answer Engines. They aren't.
The needed inovations for any potential new business model of the internet are not technical.
Even if some system of making crawlers pay for a content access, how could you be sure it's never used without your permission? It's not like DRM solved internet piracy.
And even if a technical solution were possible, it's natural to see this level of cynicism of how an internet giant would handle it.
People generally have the sense that the internet is a free for all, and compensating creators is optional (i.e. I don't have to load the ads on your website if I don't want to, your ads are malware, your ads are scams, it's my eyes, etc. etc.).
I'm not sure why there would be an expectation from most users that AI should compensate creators.
100% agree with statement of problem. Only partly on-board with the possible solutions. There are many other ways, and most likely not a single solution to fit all.
Kudos to CF for pointing out the dips in the road ahead. A big, juicy problem to tackle.
Why would humans fill the gaps, and not AI? The current media platforms algorithmically curate content. I suspect the logical conclusion of that is to use AI to generate content on-demand, perfectly tailored to what the user is most likely to find engaging.
Colour me cynical mauve, but you would have to pry the current business model out of advertisers' cold dead hands.
What is most likely to happen is the so called Answer Engines will embed advertising into their results -- except in a more insidious, subtle, hard to detect and filter out manner.
The Open Letter reads naive at best, asking us to imagine an Internet where creators are rewarded for "filling the holes in human knowledge". We all know that is not what sells, and that the opposite of this will continue to inundate the infosphere.
> What is most likely to happen is the so called Answer Engines will embed advertising into their results -- except in a more insidious, subtle, hard to detect and filter out manner.
This is the part of the new AI paradigm that concerns me the most, and I think you are correct.
If "pay for placement" (or worse "legislate for placement") in LLM training becomes a thing, then we lose all transparency as these biases get baked into the knowledge set, and users have no way of knowing where and when they get applied.
If AI companies continue to grow and be successful, and if they continue to depend on scraping human-written content to train their models, and if any tech can gain enough adoption to prevent AI from doing so without paying (and without hurting non-scraper readers), then creators will get paid. That is what the claim seems to be here. You can say it's a long shot, but it's not fundamentally at odds with the existing legible incentives.
People like the current model too, because you can access most(ish) content without ads or subscription if you ad-block.
This discussion is going to be rife with the pot calling the kettle black as people who have blocked every ad for the last 15 years call out AI for not compensating creators...
Say AI company A pays and gets access, trains a model-A. Now come AI company B, C, D.. and they train on synthetic data from model A. Do they need to license the original content anymore? (morally and practically)
>Make no mistake, the Internet has never been free. There's always been a reward system that transferred value from consumers to creators and, in doing so, filled the Internet with content.
Jesus Christ I knew when I clicked on this link it was going to be corporate propaganda which pushes a specific narrative favorable to their business model, but this is just wrong.
It's one thing to say that online content has always relied on having somebody to foot the bill for infrastructure if the content can't offset that overhead itself but they're just lying as if none of those forums, IRC rooms and crazy geocities/angelfire webrings ever existed. As if there is nothing on the internet today which exists for its own sake rather than as the outcome of a creator's profit-motive.
A whole new SEO style race to the bottom, as an arms race between quality and exploiting monetization begins, with cloudflare arbitraging every step of every interaction. Sounds great! Let's enshittify everything, and give total control to a private company who's more than happy to grant undue power to governments and bureaucrats to oppress and harass and exploit citizens, as long as they make a buck from every click and harvest every bit of data traversing every network on the planet.
”What's most interesting is what content companies are getting the best deals. It's not the ragebait headline writers. It's not the news organizations writing yet another take on what's going on in politics. It's not the spammy content farms full of drivel. Instead, it's Reddit and other quirky corners that best remind us of the Internet of old.”
I don’t know if I see a company extracting rent from other people as a win…
We need something like tracking cookies for model inputs and outputs.
Creators should be able to trace and quantify exactly what data of theirs was fed into the grift machine and be reimbursed directly by the grift machine custodian each time their data is used to generate new output.
A middleman who collects a giant chunk of creator royalties, for data that will be used perpetually by the grift machine... that sounds like a bad deal.
AI drives search, search leads to vapid content, vapid content has ads, big incumbents get lots of money for serving ads. Only difference is Google is now an AI engine rather than a search engine. There will be a few more large rent-seeking "platforms" that will make billions to prop it all up. The internet will be filled with complete crap that is inescapable.
Ah, it's the same playbook as their "free SSL for everyone" move years ago - take something that should be a commodity, offer it for "free," but make sure you're the chokepoint. Except this time they're trying to do it with the entire payment layer of the future web. Nice gig if you can get it.
Personally, I’m of the view that the internet is like a bulletin board: If you post your content publicly then it can be read publicly. Paywalls and required log-ins are perfectly OK.
I think what Cloudflare is advocating for is preservation of a “public+Ads” bundle where ads are viewed by a human. So they want to disentangle a computer reading something (the core premise of the internet) from a human putting eyes on it.
I wouldn’t mind see a world in which more content is private and the creator can decide how to monetize. It feels like Cloudflare stepping in to say public != public is an overreach.
* Assuming AI companies will pay for content instead of just stealing it
* Assuming they'll pay enough to sustain a business, even though they're incentivized and positioned to pay as little as possible
* Assuming AI companies will give a shit about trustworthy content, as the moat (at least right now) for developing an LLM is too huge for them to seriously worry about competitors. And even if they did, it'd likely result in consolidation rather than innovation
* Assuming the AI companies (or anyone) would even be able to distinguish credible human content from AI generated slop
I don't see this working out. Props to cloud flare marketing though. This advertisement generated a good amount of discussion.
> Had the Internet not had that reward system it wouldn't be nearly as vibrant as it is today.
> What's most interesting is what content companies are getting the best deals. It's not the ragebait headline writers. It's not the news organizations writing yet another take on what's going on in politics. It's not the spammy content farms full of drivel. Instead, it's Reddit and other quirky corners that best remind us of the Internet of old.
Removing all the vibrant parts is long due, apparently :')
How grateful should we be to cloudflare that they are able to help us make the internet a fairer place? I for one am indebted that they want to hold the keys and guard the doors.
Big News : company who'se competition makes money from traffic thinks traffic should be free
I'm always fascinated by companies capacity to make deregulation the answer to every problem, included deregulation.
Neoliberalism never fails to make me cringe, the idea that any value comes as a direct consequence of the business model is not surprising coming from tech companies but still.
> So how will the business model work? [...] Imagine a future business model of the Internet that doesn't reward traffic-generating [...] but instead rewards those content creators [...]. That will involve some portion of the subscription fees AI companies collect, and some portion of the revenue from the ads they'll inevitably serve, going back to content creators ...
This isn't revolutionary and won't remove the biggest issues with the internet, This is basically how youtube remunerates content creators, that counts AI using your data as a "view" or remunerable unit of consumption.
AI companies will never agree to remunerate anyone without a fight. They spent billions on their AI models and want ROI.
Remember how youtube demonetises content with not 'advertiser friendly' content ? is that how the whole internet works now ? can i get removed from the internet if Trump affiliated CEOs hate me ?
> Our conversations with the leading AI companies nearly all acknowledge that they have a responsibility to give back to the ecosystem and compensate content creators. Confirming this, the largest publishers are reporting they're having much more constructive conversations about licensing their content to those AI companies.
So, pretty transparently, you make content, that content is owned by a publisher, that publisher gives that content to an AI company, the AI company pays back the publisher (how things already work so far), and then the publisher aggrees to give some of it back to the creators. How does cloudflare plan to promise this ? they mention "collaboration" so non binding agreements, as i assume OpenAI won't agree to pay fines to cloudflare for not paying you the 20 bucks you think you deserve for your content.
At no point do they even entertain the idea that you should be remunerated based on how often AI serves your content, or that this remuneration should be mendatory.
This letter would create incredible opportunities for censorship and abuse. Just a few days after Trump makes companies pull Jimmy and Stephen, Cloudflare propose a new internet, controlled by them, centralized in america, where Trump, or any future president could just order Wikipedia to be removed or demonetized. No thanks
> Hardin threw himself back in the chair. “You know, that's the most interesting part of the whole business. I admit that I thought his Lordship a most consummate donkey when I first met him – but it turned out that he is an accomplished diplomat and a most clever man. I took the liberty of recording all his statements.” ... “When Houk, after two days of steady work, succeeded in eliminating meaningless statements, vague gibberish, useless qualifications—in short all the goo and dribble—he found he had nothing left. Everything canceled out. Lord Dorwin, gentlemen, in five days of discussion didn't say one damned thing, and said it so that you never noticed.”
The headings tell you nothing, and doesn't seem to have much to do with the paragraphs under them.
You have to go 2/3rds of the way before you hit this:
> some number of dollars per AI company’s monthly active users going into a collective pool to be distributed out to content creators based on what most fills in the holes
and this:
> You could imagine an AI company suggesting back to creators that they need more created about topics they may not have enough content about.
Did the author not try to compose this thought in their head before they wrote it down? Did the author (or at least one editor) read this before publishing? I wouldn't get away with a sentence like this in 11th grade, and I'm not even from a natively English speaking country.
This seems completely disconnected from the reality that's been unfolding before all of us for the past few years:
> it seems potentially we're on the cusp of a new, better, and maybe healthier Internet business model
Repetition
> It may seem impossibly idealistic today, but the good news is...
Killing the open internet is generally a good thing. Large companies and hostile nation states benefit from the open internet massively, while providing none of it back. The Chinese intranet is not accessible to EU/NA scrapers, but they can read all of our scientific journals. Facebook posts aren’t freely available for you to scrape, but llama is trained on obscure usenet posts and the entire comment history of reddit and hackernews. North Korea has their own linux distribution. Etc.
If the open internet is already dead (and it is already dead), it’s better to accept that reality and silo off the good parts behind paywalls so that people can get paid, rather than to let bad people benefit massively from it while they build their walled gardens. This has been a long time coming.
> As we think about our role at Cloudflare in this developing market, it's not about protecting the status quo but instead helping catalyze a better business model for the future of Internet content creation. That means creating a level playing field. Ideally there should be lots of AI companies, large and small, and lots of content creators, large and small.
Not mentioned: there would be a single gatekeeper for the internet, Cloudflare.
The "level playing field" rhetoric reminds me so much of Apple talking about the App Store. This new internet business model is just the App Store, substituting websites for apps and Cloudflare for Apple. The system only works with some middleman between the AI companies and the content creators.
I see what you mean, and I am divided on the issue. On one hand, I don't think it's fair to have AI companies to freely scrape the world without fairly compensating content producers. On the other hand, adding more gatekeepers to the web ends up killing it.
This feels like a lose-lose situation.
Cloudflare is most active in pushing standards, and highlighting the issue of bots scraping web pages for free. That said, the other major CDNs (e.g. Akamai, Fastly) also have Bot Management functionality, so hopefully this is not a gatekeeping scenario, more of a standards-building scenario.
In some positive news today, Cloudflare is sponsoring the Ladybird browser effort:
https://news.ycombinator.com/item?id=45332860
That's not nothing.
Its a token gesture. Ladybird is not a serious disruption.
How could you be so confident about such a thing?
Are you implying it's going to fail in its goal as a competing browser, or that the competition is somehow irrelevant in a world that's forced to be slave to Google/Apple?
I'm implying that Ladybird will never grow to the size where it will have any influence on the internet. So this donation is a token gesture to making things better and will be unlikely to change anything.
I dont think Ladybird will ever surpass Firefox and to have any impact it would need to be at least as popular as firefox.
Likely related on getting influece in order to enforce the gatekeeping standards, unfortunately.
Even if you end up being right, which is not outside of the realm of possibility, it's still huge to get a boost on breaking the browser duopoly while ironing out longstanding issues with web standards.
That feels like a bog basic false choice fallacy.
> That feels like a bog basic false choice fallacy.
Only if you somehow believe you have a choice, or even a say.
That is a lot of hands... Are you AI generated? lol =3
Two _is_ more than the average, but not by that much...
> On the other hand, adding more gatekeepers to the web ends up killing it.
LOL, what?
> Not mentioned: there would be a single gatekeeper for the internet, Cloudflare.
Nothing in their idea challenges the underlying tech behind the internet. Anyone is free to compete in constructing a reverse proxy service with LLM-centric content controls similar to cloudflare, whether that’s AWS WAF or akamai or some new startup.
Nothing in Google's search monopoly challenges the underlying tech behind the internet. Anyone is free to complete...
From the stats I've seen, Cloudflare has an 80% market share for reverse proxy services. 20% of all websites use Cloudflare, 50% of the most popular websites globally. That's a dangerous amount of concentration, and it's the only reason Cloudflare can propose this new business model for the internet and be taken seriously.
Google’s monopoly is dangerous because they linked success in search to dominance in other areas, and especially the most popular web browser.
I wouldn’t recommend trusting any large company but so far Cloudflare doesn’t appear to be pulling a Google because they sell directly rather than to third parties. Google never charged for search so they ended up doing a reverse acquisition into DoubleClick to get advertisers to pay for the searches we do. Cloudflare does have a free tier but their paid services are decidedly not free and since they have serious competition in the CDN business, zero-trust, etc. they have the direct incentive not to screw their customers which Google lacked. I’d get worried if that ever changes.
> Google’s monopoly is dangerous because they linked success in search to dominance in other areas
That's precisely what's happening here: Cloudflare is leveraging its CDN dominance to become a kind of payment processor for the internet.
> they have serious competition in the CDN business
Do they? I just said they have 80% market share.
> they have the direct incentive not to screw their customers which Google lacked
Google Search is free service for users but a paid service for advertisers. The advertisers are Google's customers. Theoretically, Google has an incentive not to screw its customers, but practically they can, because of their search monopoly.
> Cloudflare is leveraging its CDN dominance to become a kind of payment processor for the internet
Seems like the moment is ripe for this move. In recent news Google partnered with stablecoins and traditional payment processors to create an agent-to-agent micropayment system (AP2)
https://cloud.google.com/blog/products/ai-machine-learning/a...
> Nothing in Google's search monopoly challenges the underlying tech behind the internet. Anyone is free to compete...
But it's true? It's still true today. The only worrying part of the story is that google also makes browser and OS, which doesn't apply to Cloudflare.
The above comparison to App Store is even weirder / more ridiculous. App devs publish on App Store because App Store is pre-installed on every iPhone already, so it maximizes the number of users they can reach. Websites use Cloudflare to protect themselves, at the cost of reducing the number of users they can reach. The two situations are so different that "false equivalence" is an understatement.
> The only worrying part of the story is that google also makes browser and OS, which doesn't apply to Cloudflare.
Well actually: https://blog.cloudflare.com/supporting-the-future-of-the-ope...
> App devs publish on App Store because App Store is pre-installed on every iPhone already, so it maximizes the number of users they can reach.
This seems like a weird statement, because App Store is the only way of publishing apps on iPhone. The statement might make sense if you were talking about the Mac, on which App Store is pre-installed, but developers can still publish outside the App Store.
> Websites use Cloudflare to protect themselves, at the cost of reducing the number of users they can reach.
How does Cloudflare reduce the number of users they can reach?
> https://blog.cloudflare.com/supporting-the-future-of-the-ope...
Yeah and if it succeeds (while unlikely) it proves Google-style monopoly isn't that bad and permanent.
> because App Store is the only way of publishing apps on iPhone
You're going to be really surprised to find out iOS porn game is a thing in Asia :)
But that's not my point. My point was Cloudflare and AppStore are very different things.
> How does Cloudflare reduce the number of users they can reach?
Any barrier would affect the number of users you reach. Even just a capcha or +200ms loading time.
So what? I should hate them for that? Cloudflare is really good at what it does. Nobody has to use cloudflare, but people who know what they’re doing choose cloudflare because the service they provide is worth the minuscule price they charge and it solves the massive abuse and performance problems that otherwise plague the internet.
Bing/msn.com failed to displace Google because Google was simply better, not because Google played dirty.
Whenever anyone says "Oh so I should hate company X" you know you're in for a bad, stilted argument that's going to defend some very narrowly defined thing.
Companies, by their nature, grow and take over things for the purposes of making money - tech companies moreso. We have seen companies overstep in the past.
Please don't stifle reasonable concerns with this sort of inflammatory rhetoric.
There's nothing reasonable about your concern. My original reply two levels up explains why: they have competitors and the service they provide is highly fungible.
> So what? I should hate them for that?
Where did I mention hate? I don't care what emotions you feel or don't feel. The problem is the concentration of power in one company. That has nothing to do with emotion.
> Bing/msn.com failed to displace Google because Google was simply better, not because Google played dirty.
I don't think the courts agree with you about Google playing dirty. In any case, monopolies are inherently dangerous.
> Where did I mention hate?
Substitute whatever adjective you want. You're spreading FUD about the cloudflare boogeyman while ignoring the fact that they have well funded competitors and have no technical mechanism whereby they could lock anyone into their so-called reverse proxy monopoly.
It could be interesting to build a small startup that identifies hate speech on Twitter, Threads, Blue Sky, and other platforms.
I envision a UI that displays the message and, in a sidebar, lists what aspects of the message classify it as hate speech. Then, like a spam filter, you could decide to block the message.
https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=1084806... https://arxiv.org/abs/2405.01577 https://arxiv.org/abs/2401.03346
[dead]
But everything in their idea challenges the idea of the Internet as a public-ish good.
> Not mentioned: there would be a single gatekeeper for the internet, Cloudflare.
I don't get it. Can you point out where Cloudflare gains the power to prevent Google or any other company from doing the same?
https://en.wikipedia.org/wiki/Natural_monopoly
It's not impossible. But it's hard.
How would that work short of them buying most of the fiber in the world? A natural monopoly is a big problem for ISPs but they don’t do last-mile and they don’t appear to be anywhere close to being able to manipulate thr transit market.
> How would that work short of them buying most of the fiber in the world?
Did Google need that to dominate search?
No, and they didn’t have a natural monopoly there either.
I'm not sure I'd agree.
https://www.promarket.org/2017/05/09/google-close-natural-mo...
Also weird for their shining example to be Reddit. Where the actual creators there would not get a dime, presumably
Also weird that they think reddit is something other than ragebait. I avoid reddit as much as I can, because everytime I go there the reddit feed is full of some of the worst ragebait on the internet.
I think that's true of any algorithmic feed. The individual niche communities are still pretty good I find. YMMV of course
> Not mentioned: there would be a single gatekeeper for the internet, Cloudflare.
Cloudflare is already a monopoly though. From what I can tell, what they are saying here, besides proclaiming their continued existence, is that they and AI companies and content creators need the internet to exist.
And what they are building is on top of the 402 response, which anyone can use? So you could use that without using any CDN at all, without too much development cost?
They are all platforms: news companies, publishing houses, ISPs, recording labels, YouTube, Google, Meta, TikTok, Windows, App Store, Play Store, Amazon.. and as platforms they squeeze both the layer above and the one below, their customers and their providers.
You know what that looks like? rent. It looks like home owners setting rent as high as the market can bear, until it becomes almost no advantage at all to participate.
It sure looks like Cloudflare wants to be the platform for AI royalty collections. But creators, customers, sellers - they are all hostage to the platforms they can't avoid. That is where the money sink is.
How is Cloudflare a monopoly as opposed to just being popular? The CDN, security, edge compute, etc. markets all have multiple popular competitors.
Sorry, not exactly monopoly. Probably too big to fail, from what I gather, though. I don't know enough to know whether there is anyone else that could realistically compete with their scale.
My point was simply that their size is unrelated to this proposition, outside the fact they are big enough to be taken seriously.
Is there currently a more open alternative to Cloudflare? I would assume that people don't really use it until they have significant traffic from all different parts of the world?
In which case for the self-host people we can just pick a decent CDN?
I've done a few tests with CF hosted webapps... overall it's pretty good experience, and for my usage has been free, despite opting into a paid account level, I haven't exceeded the free usage.
For mostly-static or static-site content it's pretty nice all around. I've not gotten into the SQLite service so much though, which I might and seems interesting.. there's also Turso to consider as an alternative option... Not to mention Deno, Fastly and other similar options.
Kind of hoping to see something similar start to gain traction to support wasm backend systems similarly.
This is beyond their DDoS/Proxy protection, but worth considering as part of what they offer. There's a lot there to like.
an American company controlling the internet
> The "level playing field" rhetoric reminds me so much of Apple talking about the App Store.
To be fair to apple, many billion dollar businesses have been built for and distributed by the app store. Think about all people employed to work on mobile apps, all those people going to local taco truck for lunch owned by an immigrant, that immigrant buying a nice christmas present for their kid, etc.
> To be fair to apple, many billion dollar businesses have been built for and distributed by the app store.
This sentence is missing something essential:
Many billion dollar businesses have been built for iPhone and distributed by the app store.
iPhone is the platform, whereas App Store is merely the exclusive method of software distribution as dictated by Apple. Contrast with the Apple Mac, where the App Store is not the exclusive method of software distribution.
And of all companies Cloudflare. People, research their business practices...
"Let's give money to the content creators" is exactly what big incumbents want. They want to farm you, the viewer, for big profits, and give the content creators a few pennies in return. Meanwhile they hoover up the lion's share of the profit. We've seen this over and over again. It's rent-seeking.
"Content creators" are part of the problem. Generating endless "content", which isn't very useful or valuable, and creates too much "content supply", which devalues it. This then creates a giant "soup" of content that viewers drown in, trying to find some content that isn't as identically useless as all the rest. But the big incumbents love it, because they use this content soup to collect money from - you guessed it - ad companies. (Those same companies they don't want to make ad-blockers against...)
So now that search is dead, they need to find a new way to drink from the ad-dollar faucet. At first it'll be "pay content creators from AI subscription money", but then AI will be offered "free with ads", and then later "paid with ads". And nobody's going to stop it, because everyone is "happy": the viewer gets their free crap (with ads), the content creators get a few pennies, the big companies rake in billions, and ad companies continue their "industrial welfare" by pouring their excess profits into this whole system as ad-dollars. The great commoditization of eyeballs continues.
Had a similar thought, yeah, Spotify is like this. I don't know what the numbers are, but a very, very large majority of musicians make very little money off of Spotify (Weird Al Yankovic said he made enough in a year to buy dinner for himself).
On the other hand, Youtube is pretty good. You can make some money off of youtube viewings, but what it's really great for is building an audience. Musicians know that Youtube is a good tool for building a successful music career.
As for devaluing content with oversaturation, yeah, agree with this. Ironically, media is both the bread and butter of the internet, but it has also become devalued with the shear amount of it. Still, this isn't a problem of the media companies. Content creators can either chose to engage and or not, and hope they breakthrough.
And you kind of said it, "The great commoditization of eyeballs continues." Money makes the world go round, and nothing is free. If we want a vibrant internet, companies have got to find a way to make money (although, monopolies aren't a good thing, but paying for good content and services seems like a good idea).
Btw, to all that are reading this, actually, found out the that Weird Al's music makes a lot more from Spotify than buying himself a sandwich - His rights holders get paid by spotify. These middlemen then take a cut and pay Weird Al. But on top of this, since Weird Al does parodies, the musicians he parodies also take a share (Michael Jackson, Nirvana...).
He got 80 million streams of his music in 2023, which paid out $386k to his rights holders according to a Spotify payout calculator. Not great, but not bad. https://musically.com/2023/12/01/weird-al-yankovics-spotify-...
A question: Why do musicians put their music on Spotify if they make so little of it.
If the answer is that it's the labels who owns the rights and puts the music there, then the next question is why do the labels do it if Spotify pays so little.
If the answer is that Spotify pays decently to the labels, then isn't all of this just the labels exploiting the artists as usual?
Am not an expert at Spotify, but for at least the musicians i know, they self-publish these days, so it's really not about the labels. For us, Spotify is just another way of getting our music out there. Yeah, we can build a very small audience who'll follow our music, but more over, it's just a way to point interested producers, venues, musicians or listeners to our music.
And actually Spotify hasn't been profitable until 2024. It seems like the problem is streaming music is not a great business model. They pay a tons to labels and (big name) artists, but only get a set subscription fee from its listeners.
It's a the lesser of two evils, because without it, people would go back to bittorrenting music or using things like the Apple Store, where you don't have to buy an entire album, you can just buy only the songs you like for a couple bucks.
While I'm partial to your cynic interpretation — after all Cloudlfare is a for-profit US business —, the model isn't that far from what Jaron Lanier had proposed in ’Who Owns The Future?‘ (2010).
In the book he explains how these micropayments would be sent out to all the websites you visit, etc. And you can have a built-in mechanism to combat the "star system" that tech automatically enables, where a few people get the most money.
Lanier's proposal was that ISPs take charge of this issue, but perhaps Cloudflare can make it work. The less cynical interpretation would be that later they federate the system or even turn into a non-profit.
I don't think every website you visit should be paid. Who says this random website even deserves payment? This would incentivize people to use tricks to redirect traffic to their websites (which already happens..). Imagine a world where, in order to step into a retail store and browse the merchandise, you had to pay $0.25. Payment systems are already a nightmare, this would add headaches. And just because it's a non-profit or federated doesn't mean anything; show me a non-profit and I'll show you a hierarchical organization with a rich board, poor workers, and problematic outcomes.
People need to think long and hard about why they think there should be unlimited stuff on the internet. Have you looked around? It's mostly crap. Random scribblings by opinionated amateurs, exhibitionists shilling snake-oil, "products" like online games or unlimited videos designed to sell your personal information for a profit, "news" that is 100% gossip, and an unlimited number of echo chambers. We need less incentive for all this crap, not more. But ads are never going away, so it's moot.
Everyone seems to be dismissing this without reading it. As a content publisher, I am very interested in any proposal that results in me getting residual payments from AI scrapers. That would indeed be a new internet business model.
This is also being attempted by RSL with their “Crawler Authentication Protocol” (https://rslstandard.org/guide/web-crawlers) for demanding proof of licensing from scrapers and RSL Collective (https://rslcollective.org) for providing the licensing itself. The missing piece there is the ability to detect scrapers with high accuracy without punishing regular browsing humans.
As an engineer who works at a medium-large publishing company, am also interested in this direction.
I have fully read Cloudflare's proposal, as I have RSL, and L402.
In its current state, RSL is not a full solution, but can be. The spec does not cover payments. Its basically just a new token type, which crawlers would register for to crawl your content, and predefined responses for more detailed states. But nothing around payments, yet.
Cloudflare's solution is basically the same as L402. This is very sad for me, because their attempt is basically one of locking down the ecosystem in order to create a monopoly.
If they were really about creating a fair internet where "pay per crawl" is possible (it already was with L402), they would openly support L402, implement it for their gateway, and try to push the industry towards it. But no, they just want the biggest piece of the pie; a pie that doesn't even exist yet.
It seems there are multiple initiatives. Have you also read this one?
https://iabtechlab.com/announcing-content-monetization-proto...
Maybe people have agendas, or maybe they have different perspectives that should be reconciled?
> without punishing regular browsing humans.
As a content consumer, I'm also hoping to be part of the ecosystem. I already use Patreon a lot as "AdBlock absolution", but it doesn't fix the market dynamics. Major content platforms tend to stagnate or worsen over time, because they prefer to sell impressions to advertisers than a good product to consumers.
I'm really excited by this post. I love the vision. especially the vision of using what is already known about the gaps and what is not known in having content created to fill those gaps. There are definitely some gaps in LLM knowledge.
> As a content publisher
It always bothers me a little when people describe themselves as a content publisher or content creator. Why content? Why not say you are a short film maker or writer or musician or videographer or journalist or painter or photographer or ...?
In the past, your work has been merely content to the people who are looking for bits -- any bits -- to place between and around the stuff they really care about: ads. These days I suppose your work may be seen by some as grist for the AI mill.
By calling your work content you are acceding to the idea that your work is only valuable in other contexts. It isn't! I bet it's something you made with care and thought.
Or maybe I misread and you really are a content publisher. You're the person running the CMS who is looking for other people's work to monetize. If so, nevermind.
As a philosopher, I'd guess it's frequently because more concrete terms like "blogger" or "forum commenter" or "video game player" somehow lend even less credibility than the generic terminology.
I think as a publisher, you should be worried that Cloudflare is taking the first step towards the 2025 version of what “rights holders” have done to the movie and entertainment industry.
Imagine trying to create and distribute a movie without the backing of your local distributor/broadcasting cartel. Only instead of gating the movie theatre, Cloudflare is the single access to all things web. Also, based in the US, so good luck producing content their government doesn’t like.
I like the IDEA of residual payments for content on paper. Brave browser tried this, but it didn't get enough traction from publishers of note to be viable. But I think, at least in theory, the idea of me as a consumer having some money attached to my eyeballs for the person/people who created the content I'm seeking out is a good one. Paywalls are annoying and awful, and unfortunately they seem to encourage data hording and selling practices that are unsavory. But I like the model of simply opening the browser, visiting a page, and the creators getting some royalty from that. The system we have now is too exploitative - it extracts more value that it creates.
I mean, you shouldn't need to read a wall of text to understand what a business model is.
It should be front and center, communicated clearly, and easy to understand. If I was an investor, the lack of clarity after reading a few paragraphs would concern me.
It definitely isn't an aerated Linkedin post full of single-sentence paragraphs and emoji, but it's also certainly not a "wall of text". It's what we used to call a "blog post", it has very readable paragraphs broken up into short titled sections.
These days all content has to be in the form of a catchy 6 word tweet or 10 second reel to keep people's attention.
There was no subway surfers video in the top how can we be expected to concentrate.
I read the entire article and I have no clue how this new business model will come to be. Something about filling holes im the cheese…
> As a content publisher, I am very interested in any proposal that results in me getting residual payments from AI scrapers.
So you use a public resource and presumably like the upside that comes with sharing on it, but you want to limit uses now that someone has found one that you don't like, so you're fine with degrading that public resource?
You could always share things privately or behind a paywall if you don't want them available publicly. But people seem to want to have their cake and eat it too.
I get why a hosting provider would want to limit crawlers to save bandwidth. The "creator" angle is just greed.
> So you use a public resource and presumably like the upside that comes with sharing on it, but you want to limit uses now that someone has found one that you don't like, so you're fine with degrading that public resource?
Can you say more? what is the "public resource" I'm using as a content publisher? In that role I see myself more as the provider of a public resource (my content), not a user.
Maybe they mean the alphabet?
Your point is that pepole leaving some produce on the side of the road along with a money box should accept that some people loot all the produce in a 100km radius without paying, and resell it?
You can put images up on a public website, but you still retain copyright & can control the images' use. This is no different from any kind of digital content...
And Cloudflare is trying to legitimise a “business model” resting on the assumption that it’s ok for AI vealers to scrape content in the first place. “Opt-in” is not in Silicon Valley vocabulary.
I read that differently: we’re starting from the current world where anyone publishing online is seeing expensive traffic volumes from companies whose goal is not to send user traffic back. Cloudflare seem to be arguing that site owners should easily be able to charge for agent access independently from regular user access, but since that is always under the control of the site owner it seems like the opposite of your “opt-in” complaint.
As a musician, yeah, don't agree with this. It'd be great to find a way practical way to get content creators compensated, because with the internet, almost all media is becoming free (or near free), and it's hard to survive.
And, the idea being proposed is a version of what you're talking about (paywall). Not allowing crawlers to scrap your site unless they compensate you is a form of paywall, just for crawlers...
...But, I do agree with one aspect you're saying about greed, Cloudflare is probably being self serving in this. An open standard seems like it'd be a better solution than having one company control content payment.
I hope that the next big shift will be to undo the asymmetry that crept into the internet quickly after it first became popular. Let us host stuff at home. Let us run odd and strange and great systems wherever we are. Undo the cloud, undo the capture of the net – I'm old enough to remember when we just had a bunch of boxen under a desk somewhere, and it was pretty great.
I've been thinking lately how nice it was when the Internet had lots of phpBB-like forums spread around. Each one ran by a different server with it's own rules. None of them required your ID or phone number to join. No one really knew who anyone else was in real life. If you got tired of the people one forum you had several similar forums on different sites you could go to. It was fun seeing the kinds of things people would share in that environment. Everyone had the power to be seen, heard and disregarded in a short span of time.
I think of these things like they only exist in the past but I'm sure these places are still around only I don't go to them as much now. Probably because all the useful stuff these days exist in mainly reddit and the stackexchanges.
I too miss the days of phpBB an IPB forums. Circle.so has come the closest to replicating some of that interface / vibe, but as expected, their pricing has crept up beyond a reasonable point for small projects / communities. The answer seems to be something self hosted.
The problem is always discovery. There are lots and lots of little sites out there but without a search engine or directory you don’t know about them. Even back in the day Yahoo had a directory. And once you need such an aggregator, it will naturally create a choke point.
You can still do this, it all still works, technically. You will have a spam/malware/security issues that wouldn't have been an issue back in the day. You will also have discoverability issues - but that hasn't actually changed, if it's just for you or your friends.
This is one of the core usecases of cloudflared (https://github.com/cloudflare/cloudflared)! Without having a static IP address or dynamic DNS, you can establish a connection between your machine in a private network with Cloudflare. And once a resource is on the Cloudflare network, you can pretty much do anything with it, including routing your visitors to that server and using the full Cloudflare stack including its CDN, firewalls, and Anti-Bot protections.
This project does exactly the opposite of what the GP was talking about. We need to selfhost to be independent of Cloudflare, not dive even deeper into it. cloudflared exists to embrace and extinguish and absolutely fuck that and everything that comes from Cloudflare.
> Let us host stuff at home.
We can dream. ISP says NO!
Don't just dream. We can choose to use the ISP that says "yes", or at least a less firm "no". We can vote with our wallets to create a market opening for the ISP that says "yes".
I have 2 options and both are garbage :/
My ISP gives me v6 and lets me turn off their firewall. That's a start
In general, most ISP have a usage agreement posted which prohibits excessive client server traffic.
This is why most NOCs enforce an asymmetric firewall bandwidth limit for download focused customers, and install colo Google/CDN rack appliances.
Most ISP are just the modern retooled Cable company business. Try to stream video off your home platform on a normal service port, and you will hit the caps pretty quickly. =3
Do you want more people to view your content?
If no, you can still do that today.
If yes, then imagine the other 1,000,000 people who think likewise. That was how we got where we are today.
Who is stopping you from doing any of this?
[dead]
> Unless you believe that content creators should work for free, or that they are somehow not needed anymore — both of which are naive assumptions
Realistically, what was the benefit of the ad-driven model? From my point of view, most of the highly-valuable information came from the various forums, wikis and personal sites, on all of which people would publish the information for free. Ads were largely used to cover hosting costs of the large forums and wikis, but the content creators saw not a single dime.
Over the last 15 years, the search results have shifted from this, to instead become 100% news articles and SEO spam, all with a lot of fluff and very little substance, tons of ads, pop-ups, autoplay videos and subscription walls. All of this is well funded thanks to the business model, but for what? I can't imagine anyone sitting through all that crap to get to what they were looking for. There's a reason people would tell others to add "reddit" to the search query only a few years back, and even that's becoming less worthwhile.
Is this really what we want to preserve? Big publications and their interests?
> Realistically, what was the benefit of the ad-driven model? ...Is this really what we want to preserve? Big publications and their interests?
They are proposing a scheme by which AI scrapers would have to pay for the content they scrape, which could replace the ad-driven model and be viable for more creators.
How is this better for creators than ads?
Ads are ugly and invasive and bloated and slow.
After AI scrape the ad free content, they'll probably turn around and show end users the AI summary with Ads. It never ends. The problem is a much deeper flaw at the heart of capitalism. Enshitification of everything.
> We have a pretty good mathematical representation of human knowledge...
> Swiss Cheese — there's a lot of cheese, but there's also a lot of holes.
Such hubris! Human knowledge is not like Swiss (mostly known with a few gaps). Human knowledge is like a high-dimensional fractal broccoli: a stem of hard-won knowledge surrounded by a vast frontier of unknown. The hardest part of pushing on the frontier is deciding where to push. Deciding what to research requires taste.
In short: we'll have the machines tell the creators what to create.
> You could imagine an AI company suggesting back to creators that they need more created about topics they may not have enough content about. Say, for example, the carrying capacity of unladened swallows because they know their subscribers of a certain age and proclivity are always looking for answers about that topic. The very pruning algorithms the AI companies use today form a roadmap for what content is worth enough to not be pruned but paid for.
Valid point / not a bad idea. One flaw might be that (successful and high quality) content creators are accustomed outsized rewards for their work. Ads pay very, very well for people creating content viewed by a lot of people.
Would AI companies be willing to match that (even 50% match)? If not, we might just end up with low paid copyrighters / ghost writers churning out large amounts of content for LLMs in subject areas where they don't necessarily have expertise.
I think Patreon-style backing has been proving out the viability of payment based creators. Technology Connections is largely funded by Patreon but puts out free content all the time. My guess is that once x402 becomes more standardized that Youtube, Substack, and other SaaS can add in paywall functionality which can help fund the free content for creators.
Creating something on demand instead of growing it autonomously already made everything non-authentic for the most part. Automating this would make everything totally fake and homogenized, and also devalue the data in the process. "Regularization" and filling gaps is a mistake, they should reward the originality instead.
"The machines" in this scenario are being informed by what the humans are asking for though..
They're not just making it up..
It means they will request faster horses, though.
"Make no mistake, the Internet has never been free."
Honestly, this sounds like a Mozilla press release. The spin is absurd
Before ISPs existed, someone else paid. Universities, governments, selected corporations, etc.
After ISPs (post-UUNet), internet subscribers have always had to pay
Who finances the data collection, i.e., the transfer of data from/about subscribers to the so-called "tech" company for commercial exploitation. Hint: It is not the so-called "tech" companies
When I first accessed the internet, and later the www, the term "creators" was never used to describe people who used it
Methinks "creators" is SillyCon Valley speak for "unpaid independent contractors". Why not use the term "producers"
The gigantic websites comprised of other peoples' work, today's so-called "platforms", are not "creators", they produce nothing. Despite billions of dollars at their disposal, and vast amounts of published material, their production costs are small. This is because production costs are outsourced to unpaid independent contractors. The material the "platforms" publish does not belong to them, they are only third party intermediaries (middlemen)
In the original internet, people publishing on the internet via websites produced the website content themselves
Today, "platforms" are not creating anything except a baited trap for people using the internet, whether they are "creators" (producers) or consumers, where internet users can be spied on for advertising purposes. And the targets of this surveillance and subsequent targeted advertising will even pay for sending the surveillance data collected to the interlopers
Today's "creators" may be paid (if they are lucky, like winning a lottery) or not (the other 99%, their free contributions serve as bait). But "make no mistake", it is not the third party intermediary "platforms" that pay "creators", it is advertisers, but only after the "platforms" (middlemen) take a cut
SillyCon Valley calls this an "ecosystem". But today's internet, cf. the original one, IMHO looks more like a toxic waste dump
Something that stuck out on this was the notion of the LLMs as models of human knowledge, and the notion that we could "see the gaps". There's been an interesting debate* in journalism over the concept of neutrality - what it means to "just present the facts," and whether that's actually a useful service, and what are actual facts, and how do you define a fact (it's gotten rather epistemological), and that's just for things that actually happened in the real world. I think what's not taken into account here is how much of the world is not able to be summarized into One Universal Truth - how much of what we look for is actually the preference of the author ("what are the best new albums this year?" "how do I cook chicken?"), or an intellectual synthesis of a subject (basically anything written by that acoup guy), or just some entertainment or random digression on a topic. I think this is part of the crux of the argument behind the artists vs LLMs debate (you know, aside from all the economic exploitation) - even the act of writing a summary is a creative act when performed by a human that generates something new, whereas an LLM is, as far as we can tell from both the mechanics and the actual output, a remix of existing content. I think our appetite for the latter is less than we think it is.
I don't fundamentally think that Cloudflare's view here is _wrong_ - by and large, when I google what internal temperature to cook pork to so I don't die, I'm not looking for an opinion or someone's life story - but I think I'm much more interested in how we create the weird niches that create the "knowledge" for the blender. Cloudflare alludes to it with the talk about Reddit and whatnot, but I'd love to know what the plan is to actually create and nurture those communities where people can really get weird into whatever topic they're interested in. Right now we're all sort of just ghettoized into various Facebook communities or whatever, but recreating an actual vibrant communal internet where people can find their weirdo subcultures and actually negotiate on some kind of reasonable footing with the LLM scrapers (who were the social network people, who were the search engine people, who have always been the ad people) would be a genuine improvement to the internet.
* read "hellish swamp diving affair"
That discussion also makes me worry that they may try to use LLMs or LLM-based metrics to measure the size of the gap as a proxy for value of the content.
The landlord of the marketplace should probably not dabble in the appraisal of products, whether for factuality or value.
> As a rough and simplistic sketch, think of it as some number of dollars per AI company’s monthly active users going into a collective pool to be distributed out to content creators based on what most fills in the holes in the cheese.
This is the old "blockchains will create new niches for creators" / "rent-seeking is Good Actually" argument, barely reheated. Total imagination bankruptcy from C-suite.
I think providers of so-called "answer engines" will read the writing on the wall and find new ways to support content creators in order to keep their databases fresh, relevant, and centered on human perspectives. I also think that to see how the economics will play out, you only need to look at similarly centralized systems like Apple's App Store and Google's AdSense.
Because the providers will act as a single tunnel that all content passes through before reaching the end user, the tolls they collect will be large. So, I don't doubt that there will still be opportunities for content creators to earn money as answer engines siphon off more and more of the web's traffic, but expect those opportunities to be broadly low-paying, falling decidedly in the "side hustle" category.
AI providers will want to incentivize content creation. There will still be a glut of ready providers, and little reason for providers to make anything but small, nominal payments.
I don't know if this is going to work, but at least they are skating to where the puck is going to be.
The new model will be something like:
1. A content creator creates a web site and uses Cloudflare.
2. AI companies pay Cloudflare to allow them to scrape content.
3. Cloudflare gives a cut to the content creator.
4. Users pay AI companies and get their questions answered.
A few observations/predictions:
* If this works, there will be competitors to Cloudflare (AWS, Microsoft, etc.) who will offer better terms to content creators. Content creators can then (easily) switch to whichever reverse-proxy has the best terms.
* Media companies will transform into Cloudflare competitors, aggregating content and monetizing by selling to AI. Their pitch will be that the content will be more curated than Cloudflare. Their brands might survive if the AIs pass the source of the content all the way to the user. For example, the AI says something like, "According to a BBC contributor....". Otherwise, media brands will no longer be known to consumers (only AI companies will care).
* If this works, AI companies will try to cut out the middle-man by building their own ecosystem of content creators.
* As more and more people get their answers directly from AI, it will be easier to sell content directly to AI companies. I.e., instead of publishing something on the open web and relying on Robots.txt to protect your content, you will sell content straight to the AI company. NOTE: If this happens, then the only way this will scale is if the AI itself decides which content it wants to buy for the next training run.
* At the limit, the web and everything about it basically disappears. Everyone gets their content directly from an AI and never visits a web site directly. Therefore, web sites disappear and all that's left is the HTTP protocol, which is used by AI clients to talk to the AI cloud.
But what is this new business model, and how do we transition to it? There's so little actual detail there.
Charge site owners to protect the site from aggressive crawlers, charge crawlers/agents to bypass the protection. Like a good old fashioned protection racket
That's not what protection racket means. Cloudflare isn't the one building LLMs and scraping websites.
I guess it would be if Cloudflare was operating the bots.
huh? they can block the bots
Usually the protection racket (https://en.wikipedia.org/wiki/Protection_racket) is when the people who will cause the destruction charge to prevent it from happening.
Paying a security guard isn't considered a protection racket, while paying a member of the mob so that nothing happens to my store is considered a protection racket.
What do you call it when the same security guard also gets paid by the attackers to let them in?
Charge crawlers and pay site owners, and take a cut of course
Thanks for the tl;dr . Why is this blog post so long? Will they charge per token?
I think they're alluding to their already-announced scheme which will charge parasitic crawlers an entry fee for safe passage past their filters.
https://www.cloudflare.com/en-gb/paypercrawl-signup/
Cloudflare has controls already in their dashboard for controlling whether LLMs should be denied responses when querying your site. How they intend to broker payments and selective access isn’t really clear, but you can stop giving your content away for free if you’d like.
You create content and gate it behind Cloudflare.
Cloudflare blocks AI scrapers unless they pay the toll.
Yeah, I was waiting for "how are they going to shove AI into this?" and boom there it is halfway down the page. No content here (obvious problem: how does paying Reddit Inc to allow scraping result in money going to the person who created the content? shrug). Just more AI hype to get investor cash before the bubble bursts. Woopie.
Edit: Re-read it and just noticed this:
> assuming the AI companies are willing to step up, support the ecosystem, and pay for the content that is the most valuable to them
Lolllllll. Y'all are gonna love my pitch for an airline startup. It begins, "assuming we can turn off gravity..."
If Cloudflare demonstrates it can reliably distinguish AI scraper traffic from human traffic, then they will be able to force AI companies to step up, no assumption necessary. Still a big if, but (this one clause you quoted notwithstanding) it doesn't read to me like they're relying on the goodwill of AI companies.
Waiting for the day when I can cash out my reddit karma for USD
A bunch of vague nonsense, par for the course around the current hysteria of AI.
Nobody at the head of these large tech firms has a clue about where this technology is heading.
I think they all know where it is heading and it is massively less money for everyone. Every technical breakthrough doesn't mean more profit for everyone and nor should it. Sometimes it's just a dishwasher or washing machine, something that frees up the bulk of humanity from something that is drudgery. Whether we actually get those benefits from AI remains to be seen.
Its cloudflare trying to enshittify the internet with micro transactions[0] and take their N% cut (of course it will start at like 2% but ask any uber driver how thats going).
The problem is the arguments they make for why this should happen are quite compelling, especially to those running sites (you'll see plenty of complaints on this forum about it), but theres also a large group of people who think information / code / data should be "free" (see open source code/maps/anything you can think off). So really its just a moral debate that will be lost in the interest of profit (which is ya know good n bad, if AI companies did more caching we probably wouldn't need this, but here we are).
[0] https://blog.cloudflare.com/introducing-ai-crawl-control/
Seems like a cheap attempt to insert yet another middleman (Cloudflare) to content monetization while virtue signaling.
I wonder how the web would look like if one could click "pay 1 cent to continue".
Maybe content would become better? Maybe it would make one think "Hmm... one moment, is this something I want to read or am I just doomscrolling?".
Micropayments aren't hard. Customers simply don't want to use them. Tons of newspapers and magazines have tried offering single articles for sale for $1 or 50c, and all have failed. People just don't want to use micro payments. They want to pay a monthly subscription.
$1 is not a micropayment. It's a regular payment.
The only hard thing about micropayments is that "payments" is a worldwide monopoly and the incumbents don't want it to exist.
Now, the problem with micropayments gated sites is that you are trying to gate the exact thing that attracts people, so don't expect this business model to work.
> Why are micropayments so hard?
Regulation.
It would like exactly how it is today. It's not hard to find someone not charging anything for the same piece of content, or not charging anything for another type of content that fills the same void.
If HN charged 1 cent per day and you could pay with a simple click, you would not come here anymore?
Where would you go?
using conventional methods small transactions are eaten by service fees. This means microtransactions require a different way to pay which is friction and people are apathetic especially when what're you doing is trying to get them to pay for things.
> Why are micropayments so hard?
Because Visa & Mastercard are greedy dicks?
As a researcher, I like the notion that there could be some incentive or mechanism to create new and original content, investigate new things, or conduct research in general. I'll group all of those under the umbrella of new content. I don't think things will work out as ideally as described here, but I can appreciate the sentiment and hope that it works out positively.
That said, the mental model that this article uses only recognizes that new content can fill in "holes" (interpolation). It also can expand the boundaries in new directions (extrapolation). That is a different and harder problem. If you distribute money to people "based on what most fills in the holes in the cheese", you really aren't expanding the boundaries of human knowledge as much as you are strengthening existing knowledge. They need to take boundary expansion into account here.
I also recognize that we know where the holes or boundaries are in many fields. They are "known unknowns". But this proposal does not take into account "unknown unknowns" --- things that we do not even realize that we don't know yet. It's going to be harder to incentive research into unknown unknowns when we don't even know what they are yet.
> why did Cloudflare never build an ad blocker despite many requests?
Many of us who use an ad blocker don't actually have a huge problem with ads. It's the tracking we object to.
I don't want Cloudflare to build an ad blocker, I want them to make a tracker blocker. Make surveillance so difficult and expensive and unreliable that companies stop doing it.
Unfortunately ads and tracking go hand in hand: without tracking, targeted ads are much less effective. If everything else stays the same, you'll just get more, bigger, and flashing ads to compensate for the inefficiency, like what you see today on smaller sites without sophisticated user data.
As an user of the internet, I wish cloudflare didn’t exist. They are in most sites I visit, they add a couple of seconds of delay always. They can shut down entire ranges of ips if the government says so (no matter if some sites hosted in such ranges are legitimate).
It sucks big time to live in an internet with cloudflare
Every website which gets a little bit of traffic or reputation will be hit with massive DDoS attacks. Your option as a webmaster is to use something like Cloudflare, or your content will become unavailable to everyone.
You can block all websites using Cloudflare if you think it sucks.
DDOS mitigation costs are coming down. If you're running a decently sized service and dont have any enemies that want to pay constant rates to shut you down then _should_ be fine. You can even temporarily use cloudflare if you're getting hit particularly hard then remove it when it passes. 100% uptime is a meme and not required for most websites.
Given CloudFlare's direction, I won't be surprised if they start offering "anti adblock as a service" soon. Detecting it is still rather simple, and add in some punishments such as temporary IP blocking, and it would be rather effective.
(Note that everything CloudFlare talked about in this blog post also applies to adblock users, not just AI agents.)
The golden age of the Internet is not where people do it for money, or for views. That way lies clickbait and content farms. The golden age of the Internet is one where people share information because they want to.
Source: https://1gn15.com/cloudflare
I've probably spent 400+ hours a year sharing content because I wanted to. However the wanted to, was partially because of the good feeling i got from knowing I was helping semi-specific people and then later meeting those people. It's sparked numerous interesting friendships and discussions. That latter part will no longer happen as the AI sits in the middle and becomes the known source. It has massively put me off creating more content. I wasn't creating content to generically help humanity move forward, I did it to help people similar to me facing particular situations. As that becomes diluted, the incentives won't hold to the same degree.
I recently listened to the interview between Ben Thompson and Cloudflare's CEO Matthew Prince and couldn't shake the feeling of a company that tries hard to repeatedly hit gold with the business model of being the gatekeeper of the plumbing of the web, with the promise of "the world out there is very dangerous".
https://stratechery.com/2025/an-interview-with-cloudflare-fo...
The only thing I see happening is agreements between the biggest players and the content brokers, squeezing out others who’d like to enter. At least just now anyone can theoretically scrape the web for useful info.
The bigger problem I see coming down the line, given the current business model for the internet, is what happens when investors need a return. When the LLM only provides one “correct” answer, the only way to make money from advertisers is to influence that answer. At least just now we are presented with information which we evaluate ourselves, though even this is subject to manipulation.
"With some things there's been clear progress: when we launched in 2010 less than 10 percent of the Internet was encrypted, today well over 95 percent is encrypted. We're proud of the role we played in making that happen."
Yeah right, most of it is now proxied in cleartext through them!
I don't think they actually know what the future business model is. No one really knows. What people are presenting is iterative models. That's fine but the model will be quite different. Ok we'll need subscriptions and to pay creators but I believe if everything is crypto and stablecoin based it's all going to be pay per view or pay per query because you know a lot of people just want that specific content from a specific source because of reputation or because that's what they like. I'm not paying for 5 different agents for that bespoke experience just like I'm not paying for NYT, WSJ and multiple other publications because it is insanity to price up what should effectively be pay per article. So maybe on the backend creators get a royalty kick back just like the music industry but on the consumer side I definitely think beyond subscriptions with the advent of crypto wallets we're going towards micro transactions for everything.
Ok the the toxic nature of the internet and social media in general and what has become of our digital age, totally agree that it's rage and click bait. I wrote something to that effect here https://github.com/micro/mu/issues/27. But I personally don't think we're going to directly interact with agents the whole time. They will exist, they will be somewhere in the middle layers, there might even be a chat interface that replaces search queries with answers but I think the whole web as a whole and social media needs a rethink. Ads as a business model has to die, even though clearly it won't and we need to shift our attention elsewhere.
I like that they are considering this problem. I really dont like what they have decided to focus on. Having 'answer engines' as the core with content creators filling in the gaps like some sort of task rabbit workers sounds horrible. I dont want gaps filled by people who are doing it for money. I want people that actually care enough to organically create the content.
The problem isnt oh no people wont be viewing ads and thus creators wont get paid. The problem is "why arent people who view these websites willing to pay?". So many subscriptions are like $5 a month but if we consider how much a creator is paid for showing you an ad its like 0.0005 cents. I'd rather pay that everytime i view a page than look at ads or have cloudflare pay people to create the content openAI thinks I want.
Funding the content we consume should be cheap considering how many people are there consuming it and how cheap it is these days to serve content.
Middleman as a Service!
Do people think of CF as a leader in terms of solutions that are "open, collaborative, standardized, and shared across many organizations"? My impression is that their open source work is mostly Cloudflare-specific client libraries and the occasional passion project from their engineers. Quiche may be a counter example, but it's a rare exception.
Examples:
Pingora claims to be battle-tested, but I have a hard time believing that it's to the same level of quality as whatever Cloudflare runs internally. https://github.com/cloudflare/pingora/issues/601
https://blog.cloudflare.com/introducing-oxy/ was not open source.
Small parts of Oxy were open sourced as "foundations" but the repo gives off the impression of a checkbox for someone rather than a serious commitment to building CF's own services on top of it — not "open, collaborative, standardized, and shared across many organizations".
> Make no mistake, the Internet has never been free. There's always been a reward system that transferred value from consumers to creators and, in doing so, filled the Internet with content. Had the Internet not had that reward system it wouldn't be nearly as vibrant as it is today.
This is an attempt to rewrite history. Back in the day, we stood up servers at our own expense and filled them with content for free, for nothing other than the fun of it. In fact, the "vibrancy" of the internet appears to be inversely correlated to the number of people using it to generate a profit.
But "at our own expense" is not free, is it? In the end someone has always paid the bills for things being on the Internet.
That's the "never been free" in that statement.
I don't think they're saying what you think they're saying.
They're not saying what you think either, there's no reward system or transfer of value in free content, offered for free. It doesn't matter if it's hosted at someone's own expense if it's done for nothing other than the fun of it.
I never implied there was a reward system or value transfer (unless you count the feel-good reward of sharing knowledge with others and the value of doing so).
I said that putting things on the Internet is not, and has never been, "free".
And I think that's what Cloudflare is saying too.
You're free to interpret it differently (as was OP) but I think you're wrong. ;-)
CloudFlare is definitely implying there's always been a reward systems and value transfer at play, that's there in the quoted text and what the OP is complaining about
How can you virtuously reminisce about the heydays of the open internet and then vaguely allude to gatekeeping the internet?
So why can't websites, do like any other "store". They can make you agree to pay for something but not enforce payment. Like couldn't I say, here's my article. If you read it you agree to pay me $1. Now a lot of people might read it and it might not be worth my time to go and collect $1 from each of them, just like Walmart isn't going to tackle you about eating a grape in the produce aisle. But if you want to make a business that's about exploiting Walmart's non-enforcement of free grapes then they are going to stop you.
I guess this is the 'meat' of the article. Interesting that they regard reddit as a 'niche' quirky corner of the internet. To me they are well over the peak of their enshittification journey with mainly bots and overzealous mods and pay-to-play accounts. Just like digg and others before something else will take their place soon enough as the place where enthusiasts share valuable content before businesses poke their nose in and we repeat the cycle again.
> What's most interesting is what content companies are getting the best deals. It's not the ragebait headline writers. It's not the news organizations writing yet another take on what's going on in politics. It's not the spammy content farms full of drivel. Instead, it's Reddit and other quirky corners that best remind us of the Internet of old. For those of you old enough, think back to the Internet not of the last 15 years but of the last 35. We’ve lost some of what made that early Internet great, but there are indications that we might finally have the incentives to bring more of it back.
> It seems increasingly likely that in our future, AI-driven Internet — assuming the AI companies are willing to step up, support the ecosystem, and pay for the content that is the most valuable to them — it’s the creative, local, unique, original content that’ll be worth the most. And, if you’re like us, the thing you as an Internet consumer are craving more of is creative, local, unique, original content. And, it turns out, having talked with many of them, that’s the content that content creators are most excited to create.
I wonder if that’s a purposeful framing to avoid saying that content a large and diverse group of humans has created is very valuable. Think about how much money the Facebook data set would be worth to OpenAI. But cloudflare likely wants to avoid the privacy nightmare taking attention away from what they’re doing.
Monetization models that intentionally create scarcity are scummy, including the one proposed in the article. The existing intellectual property economy is scummy. But we can fix this.
Imagine a future where:
The Internet rewards passionate creators more than instrumental alpha-seekers. This is key. But how?
We publish everything for free under reciprocal licenses like the AGPL and CC-BY-SA, not permissive licenses like the MIT and CC-BY. Those who take from the pile must add to it, or at least not claim it as exclusively their own.
Creators are rewarded by the populace via direct payments, like how streamers are rewarded. People decide what's valuable and what's slop, not Big Brother.
This creates an incentive structure where:
Those who make software and content because they love it will have more resources to make more quality stuff.
Those who make software and content for the money but actually hate what they do will find something else to do because companies will pay them less.
There are two necessary conditions that must be met, that we can work on right now:
We recognize artificial scarcity as evil, so we can enforce reciprocal licenses socially and bypass the broken legal apparatus.
We use multiple competing payment platforms to pay creators and receive payments, so there is incentive for these platforms to get better.
> Imagine a future business model of the Internet that doesn't reward traffic-generating ragebait but instead rewards those content creators that help fill in the holes in our collective metaphorical cheese. That will involve some portion of the subscription fees AI companies collect, and some portion of the revenue from the ads they'll inevitably serve, going back to content creators who most enrich the collective knowledge.
Reminds me of that bit of Snow Crash, where the freelancers sell data to the Library of Congress.
My concern is Cloudflare will implement this plan in a way that makes it very profitable for large players and absolutely kills any new entrants to the market in the future.
I can’t help but feel we are headed for less sharing rather than more, and “private peering” between those with content vaults and those who wish to access it.
I read this as: ads have been great, and have helped to pay all content creators (even the small creators). But now, Answer Engines need certain content that's missing: and it's the largest content creators that are making deals with the Answer Engines.
What CF didn't address is how the smallest content creators are going to be "making deals" with the Answer Engines. They aren't.
The needed inovations for any potential new business model of the internet are not technical.
Even if some system of making crawlers pay for a content access, how could you be sure it's never used without your permission? It's not like DRM solved internet piracy.
And even if a technical solution were possible, it's natural to see this level of cynicism of how an internet giant would handle it.
People generally have the sense that the internet is a free for all, and compensating creators is optional (i.e. I don't have to load the ads on your website if I don't want to, your ads are malware, your ads are scams, it's my eyes, etc. etc.).
I'm not sure why there would be an expectation from most users that AI should compensate creators.
100% agree with statement of problem. Only partly on-board with the possible solutions. There are many other ways, and most likely not a single solution to fit all.
Kudos to CF for pointing out the dips in the road ahead. A big, juicy problem to tackle.
Why would humans fill the gaps, and not AI? The current media platforms algorithmically curate content. I suspect the logical conclusion of that is to use AI to generate content on-demand, perfectly tailored to what the user is most likely to find engaging.
An AI is stuck inside a chip, it cannot go out in the real world and get information. It has to get it from humans.
Colour me cynical mauve, but you would have to pry the current business model out of advertisers' cold dead hands.
What is most likely to happen is the so called Answer Engines will embed advertising into their results -- except in a more insidious, subtle, hard to detect and filter out manner.
The Open Letter reads naive at best, asking us to imagine an Internet where creators are rewarded for "filling the holes in human knowledge". We all know that is not what sells, and that the opposite of this will continue to inundate the infosphere.
> What is most likely to happen is the so called Answer Engines will embed advertising into their results -- except in a more insidious, subtle, hard to detect and filter out manner.
This is the part of the new AI paradigm that concerns me the most, and I think you are correct.
If "pay for placement" (or worse "legislate for placement") in LLM training becomes a thing, then we lose all transparency as these biases get baked into the knowledge set, and users have no way of knowing where and when they get applied.
If AI companies continue to grow and be successful, and if they continue to depend on scraping human-written content to train their models, and if any tech can gain enough adoption to prevent AI from doing so without paying (and without hurting non-scraper readers), then creators will get paid. That is what the claim seems to be here. You can say it's a long shot, but it's not fundamentally at odds with the existing legible incentives.
People like the current model too, because you can access most(ish) content without ads or subscription if you ad-block.
This discussion is going to be rife with the pot calling the kettle black as people who have blocked every ad for the last 15 years call out AI for not compensating creators...
Say AI company A pays and gets access, trains a model-A. Now come AI company B, C, D.. and they train on synthetic data from model A. Do they need to license the original content anymore? (morally and practically)
>Make no mistake, the Internet has never been free. There's always been a reward system that transferred value from consumers to creators and, in doing so, filled the Internet with content.
Jesus Christ I knew when I clicked on this link it was going to be corporate propaganda which pushes a specific narrative favorable to their business model, but this is just wrong.
It's one thing to say that online content has always relied on having somebody to foot the bill for infrastructure if the content can't offset that overhead itself but they're just lying as if none of those forums, IRC rooms and crazy geocities/angelfire webrings ever existed. As if there is nothing on the internet today which exists for its own sake rather than as the outcome of a creator's profit-motive.
Title is: Cloudflare’s 2025 Annual Founders’ Letter
(https://hn.algolia.com/?dateRange=pastWeek&page=0&prefix=tru...)
Title was changed, and then changed back? Use a colon or something if really insist on "New Internet Business Model" being part of the submission
A whole new SEO style race to the bottom, as an arms race between quality and exploiting monetization begins, with cloudflare arbitraging every step of every interaction. Sounds great! Let's enshittify everything, and give total control to a private company who's more than happy to grant undue power to governments and bureaucrats to oppress and harass and exploit citizens, as long as they make a buck from every click and harvest every bit of data traversing every network on the planet.
What could possibly go wrong?
”What's most interesting is what content companies are getting the best deals. It's not the ragebait headline writers. It's not the news organizations writing yet another take on what's going on in politics. It's not the spammy content farms full of drivel. Instead, it's Reddit and other quirky corners that best remind us of the Internet of old.”
I don’t know if I see a company extracting rent from other people as a win…
We need something like tracking cookies for model inputs and outputs.
Creators should be able to trace and quantify exactly what data of theirs was fed into the grift machine and be reimbursed directly by the grift machine custodian each time their data is used to generate new output.
A middleman who collects a giant chunk of creator royalties, for data that will be used perpetually by the grift machine... that sounds like a bad deal.
I wonder how well this will play out when we look back in 10 years.
AI drives search, search leads to vapid content, vapid content has ads, big incumbents get lots of money for serving ads. Only difference is Google is now an AI engine rather than a search engine. There will be a few more large rent-seeking "platforms" that will make billions to prop it all up. The internet will be filled with complete crap that is inescapable.
So, same as it is today.
Ah, it's the same playbook as their "free SSL for everyone" move years ago - take something that should be a commodity, offer it for "free," but make sure you're the chokepoint. Except this time they're trying to do it with the entire payment layer of the future web. Nice gig if you can get it.
Personally, I’m of the view that the internet is like a bulletin board: If you post your content publicly then it can be read publicly. Paywalls and required log-ins are perfectly OK.
I think what Cloudflare is advocating for is preservation of a “public+Ads” bundle where ads are viewed by a human. So they want to disentangle a computer reading something (the core premise of the internet) from a human putting eyes on it.
I wouldn’t mind see a world in which more content is private and the creator can decide how to monetize. It feels like Cloudflare stepping in to say public != public is an overreach.
Sounds good, except it's assuming a lot.
* Assuming AI companies will pay for content instead of just stealing it
* Assuming they'll pay enough to sustain a business, even though they're incentivized and positioned to pay as little as possible
* Assuming AI companies will give a shit about trustworthy content, as the moat (at least right now) for developing an LLM is too huge for them to seriously worry about competitors. And even if they did, it'd likely result in consolidation rather than innovation
* Assuming the AI companies (or anyone) would even be able to distinguish credible human content from AI generated slop
I don't see this working out. Props to cloud flare marketing though. This advertisement generated a good amount of discussion.
> Had the Internet not had that reward system it wouldn't be nearly as vibrant as it is today.
> What's most interesting is what content companies are getting the best deals. It's not the ragebait headline writers. It's not the news organizations writing yet another take on what's going on in politics. It's not the spammy content farms full of drivel. Instead, it's Reddit and other quirky corners that best remind us of the Internet of old.
Removing all the vibrant parts is long due, apparently :')
How grateful should we be to cloudflare that they are able to help us make the internet a fairer place? I for one am indebted that they want to hold the keys and guard the doors.
I fucking despise paid Medium and Substack articles so looks like I’m gonna hate this new internet
Big News : company who'se competition makes money from traffic thinks traffic should be free
I'm always fascinated by companies capacity to make deregulation the answer to every problem, included deregulation. Neoliberalism never fails to make me cringe, the idea that any value comes as a direct consequence of the business model is not surprising coming from tech companies but still.
> So how will the business model work? [...] Imagine a future business model of the Internet that doesn't reward traffic-generating [...] but instead rewards those content creators [...]. That will involve some portion of the subscription fees AI companies collect, and some portion of the revenue from the ads they'll inevitably serve, going back to content creators ...
This isn't revolutionary and won't remove the biggest issues with the internet, This is basically how youtube remunerates content creators, that counts AI using your data as a "view" or remunerable unit of consumption.
AI companies will never agree to remunerate anyone without a fight. They spent billions on their AI models and want ROI.
Remember how youtube demonetises content with not 'advertiser friendly' content ? is that how the whole internet works now ? can i get removed from the internet if Trump affiliated CEOs hate me ?
> Our conversations with the leading AI companies nearly all acknowledge that they have a responsibility to give back to the ecosystem and compensate content creators. Confirming this, the largest publishers are reporting they're having much more constructive conversations about licensing their content to those AI companies.
So, pretty transparently, you make content, that content is owned by a publisher, that publisher gives that content to an AI company, the AI company pays back the publisher (how things already work so far), and then the publisher aggrees to give some of it back to the creators. How does cloudflare plan to promise this ? they mention "collaboration" so non binding agreements, as i assume OpenAI won't agree to pay fines to cloudflare for not paying you the 20 bucks you think you deserve for your content.
At no point do they even entertain the idea that you should be remunerated based on how often AI serves your content, or that this remuneration should be mendatory.
This letter would create incredible opportunities for censorship and abuse. Just a few days after Trump makes companies pull Jimmy and Stephen, Cloudflare propose a new internet, controlled by them, centralized in america, where Trump, or any future president could just order Wikipedia to be removed or demonetized. No thanks
TL;DR: Cloudflare wants to become the toll gate operator for the internet and AI bots.
tldr, surely this article needs an ai summary at the top.
> Hardin threw himself back in the chair. “You know, that's the most interesting part of the whole business. I admit that I thought his Lordship a most consummate donkey when I first met him – but it turned out that he is an accomplished diplomat and a most clever man. I took the liberty of recording all his statements.” ... “When Houk, after two days of steady work, succeeded in eliminating meaningless statements, vague gibberish, useless qualifications—in short all the goo and dribble—he found he had nothing left. Everything canceled out. Lord Dorwin, gentlemen, in five days of discussion didn't say one damned thing, and said it so that you never noticed.”
- "Foundation", Isaac Asimov
SHCP (The Salvor Hardin Compression Protocol)
Like a script from the pen of Tobias Fünke.
The headings tell you nothing, and doesn't seem to have much to do with the paragraphs under them.
You have to go 2/3rds of the way before you hit this:
> some number of dollars per AI company’s monthly active users going into a collective pool to be distributed out to content creators based on what most fills in the holes
and this:
> You could imagine an AI company suggesting back to creators that they need more created about topics they may not have enough content about.
Did the author not try to compose this thought in their head before they wrote it down? Did the author (or at least one editor) read this before publishing? I wouldn't get away with a sentence like this in 11th grade, and I'm not even from a natively English speaking country.
This seems completely disconnected from the reality that's been unfolding before all of us for the past few years:
> it seems potentially we're on the cusp of a new, better, and maybe healthier Internet business model
Repetition
> It may seem impossibly idealistic today, but the good news is...
then again:
> It may seem impossible...
A nothing burger. Stating the obvious.
Immediate negative gut reaction to this - this is a disgusting stance from the CEO of a DDoS protection company trying to gatekeep an open internet.
Killing the open internet is generally a good thing. Large companies and hostile nation states benefit from the open internet massively, while providing none of it back. The Chinese intranet is not accessible to EU/NA scrapers, but they can read all of our scientific journals. Facebook posts aren’t freely available for you to scrape, but llama is trained on obscure usenet posts and the entire comment history of reddit and hackernews. North Korea has their own linux distribution. Etc.
If the open internet is already dead (and it is already dead), it’s better to accept that reality and silo off the good parts behind paywalls so that people can get paid, rather than to let bad people benefit massively from it while they build their walled gardens. This has been a long time coming.
> you’re still undermining the business model of those news sources.
Fantastic! I wish I could undermine their clickbait business model even more..
> But there’s reason for optimism
You mean Cloudflare being investigated for antitrust?