If the forecasting models were so good that people were actually consistently beating prediction markets, they wouldn't be starting startups to be selling it.
And even if it is good enough, once you're shelling out thousands of dollars a year in research costs, does that give you any remaining alpha?
You got it (I was going to say "nailed it" but that's becoming an LLM marker).
This is exactly what I feel about a lot of the paid investment advice out there. Compounding can make any decent alpha worth a ton, to the point that these people would be investment bankers and not advisors if they knew what they were talking about.
> And even if it is good enough, once you're shelling out thousands of dollars a year in research costs, does that give you any remaining alpha?
That's precisely why you would want to make a startup to get investment now rather than self-fund and bootstrap. That alpha isn't going to last forever, especially because everyone has access to the frontier LLMs, which keep getting better, and will eventually beat your fancy harness or specialized finetune.
And also, perhaps more importantly, so you can start developing an alternative to prediction markets and become the new PM; as Scott notes, with superforecaster AI, it's unclear why you really need Kalshi or Manifold or anyone else, with all their fees and overhead. Leave them to the degens, and carve off the socially useful part to do much more efficiently - tokens are cheaper than transactions! This is the big prize, but you need to start now before someone else does it better or commoditizes it.
But if these "superforecasters" can forecast just about anything accurately—which seems to be the claim—then they're not limited to any one claim, or even a small subset of them. They can place bets on everything and expect to win most of the time.
I asked the guy who turned $35 into $2 million in seven months on Kalshi whether, in another seven months, he would be able to 100,000x his money a second time to $200 billion. Unsurprisingly, he said no - there’s only so much easy money on Kalshi, and his AI had already taken it all (also, other people with similar AIs are starting to fight him for it!)
So this stuff is only as useful with a steady supply of people making less intelligent bets. You don’t have to sell me on it being bullshit, but you are.
You're attacking a strawman. No one is claiming that you can pull off that multiplier at arbitrary amounts arbitrary amounts of times. And 7 months is plenty of calendar time for those arbitrages to disappear, given the attention on the area and the rapid rate of development. (Warren Buffett can't pull off his early trades now either, doesn't mean he was stupid or grifting in taking early investment.)
Agree, to an extent. The link certainly smells like an ad. If the predictions are on lower volume markets (i.e. no institutional investors with actual quants behind) the compounding power stops. Also that means it can't beat institutional investors. So the reasonable option is to monetize what you can.
He's been interested in forecasting since a long time before there was any money in it, and still hyped Metacalculus (free research project) and Manifold (play money) because he is interested in the subject. If it smells like an ad it is because your ad sense is faulty.
If you had a tool that would be capable of writing all software better than a human, why would you price it by the word (token)? Why wouldn’t you take a percentage?
(This is one of the more interesting questions that came out of Alex Karp’s televised borderline psychotic break rant the other day and it has stuck in my mind even though he is clearly unstable)
The reason is, customer might care a lot more about the prediction than you do, or anyone else does.
For example, your customer might really care a lot about some niche prediction like the number of car break-ins in Walmart parking lots. In practice you won’t have sufficient liquidity in a prediction market to actually profit off of that prediction. But a security company might really want to know the answer to it.
If a model demonstrates being good by winning on prediction markets I may want to ask the same model questions which are too niche to have markets. That said I don't know if niche questions are enough of a revenue stream to be interesting for model providers.
Perhaps they may enforce a knowledge cut-off for information retrieval and price the service based on how recent the cut-off is, and also use the cut-off as a way to guard their advantage on the markets.
Doesn't this argument prove too much? Why does AlphaSense sell their company research instead of using it to trade themselves? Why do people work on open source time series forecasting packages instead of quietly using them to trade?
Wait, the original blog post was about competing with top-tier guessers using “AI.” I wonder if there’s more money to be made on the flip side: find the people making dumb bets, at scale, and take the free money they are offering, haha.
The market of trading is smaller than the market of business. I don't think most people have ever actually tried trading for real. The frequency with which you can put in a buy/sell order and nothing happens. Then you raise/lower the price several times and nothing still happens because nobody wants to buy or sell at a price that makes you money is much higher than you would imagine. If you've never tried it yourself.
That said, the very concept of selling such information means that it would eliminate any edge and become zero profit anyway.
I can't tell you the number of times of put in an order on SPY,QQQ,SPX when I found favorable statistical analysis and I could not get filled at any price below the estimated profit potential. Those are some of the most liquid products in the word. I'm not talking pennies either.
Futures are the only thing for me this has never happened on. However no retail broker I'm aware of allows trading futures though API.
Almost by definition, once AI forecasters are in the market, they won't (all) be beating the market.
But why evaluate AI forecasters by beating the market? Do we evaluate deep learning by whether hedge funds make money from it in the markets? These things have far, far more utility outside of finance.
But Scott's point is more: why even have markets? Once you have the superforecasting available on the questions you care about, why do you need to publish it for everyone to also react to?
I mean he explicitly at the end says there will be a new era of prediction markets:
"This, then, is my prediction for the AI superforecaster future: for basic questions, your off-the-shelf AI chatbot will be able to offer opinionated probabilities superior to those of any human. For more controversial or bias-laden questions, a new era of prediction markets will smooth over differences in brand and model and efficiently aggregate all AIs’ opinions."
But one reason that you would publish it for everyone else to react was exactly what I said, because if you can show people that a superhuman AI believes a certain outcome is more likely, you impact whether or not that certain outcome is more likely. Which feels like a flaw in this version of the future. That in fact you could overcorrect and make the markets less efficient.
Given a powerful enough AI with enough outlays (eg. a Playwright instant), can an AI cheat by influencing the real world and self-fulfil its prediction?
Eg. Is corp ABC's stock price going to go down by 10% in 30 days?
Orchestrate a disinformation campaign on the 29th day to tank ABC's stock price.
In theory but in this specific example you can make a lot more money on the stock market instead if it's that doable.
In general you should judge different prediction markets differently and read the fulfilment criteria though - some are very ripe for abuse, AI or no, while others are much more ironclad.
So this person was known for thoughtful blog posts, but has since become a total AI booster. It's probably a gonzo-like effect through the various subcultures he aims to explore, but it paints a very bad picture of them. Imagine if Louis Theroux became a scientologist.
For Pete’s sake lets outlaw prediction markets already instead of hyping them in crappy greed-posts like this one
I am embarrassed this “rationalist, I’m so much smarter than you, so I know better” asshole Scott Alexander hasn't been skewered yet for promotion of gambling (prediction markets) that people like Trumps son are making millions of on
Andreesen is full speed ahead on 1984-minority-report surveillance besides all the unethical crypto investments geared toward swindling normies. Anything that looks like it would turn a profit was fair investment for a16z-crypto, no matter the pump and dumping that was going on. His latest interview on the Joe Rogan show was all about how harmful the anti-flock-camera people are, trying to prevent police from solving crimes ! Imagine !
What a strange and scoldy thing to say. I find speaking in terms of concretere predictions the best way to discuss things. It bypasses hyper moralists and scoldy type of people (your post reminds me of such) and directly jumps to truth.
I don't think there's good evidence that it directly jumps to truth. The aim of betting on a prediction market is to earn money. This incentivizes you to bet on whatever you believe is the most profitable, which is disproportionately going to be whatever you have insider information about, whatever you can influence the outcome of, and whatever you can most effectively fool other people into taking the wrong side of.
If the forecasting models were so good that people were actually consistently beating prediction markets, they wouldn't be starting startups to be selling it.
And even if it is good enough, once you're shelling out thousands of dollars a year in research costs, does that give you any remaining alpha?
You got it (I was going to say "nailed it" but that's becoming an LLM marker).
This is exactly what I feel about a lot of the paid investment advice out there. Compounding can make any decent alpha worth a ton, to the point that these people would be investment bankers and not advisors if they knew what they were talking about.
There was an old ad for eTrade that basically said that: https://m.youtube.com/watch?v=dFQ_W4p1XKY
Same with get rich quick books.
With one exception : if it teaches you how to put out get rich quick content (books or otherwise).
Shouldn't it be generating that content and making the money for itself? Why does it need the human at all?
> And even if it is good enough, once you're shelling out thousands of dollars a year in research costs, does that give you any remaining alpha?
That's precisely why you would want to make a startup to get investment now rather than self-fund and bootstrap. That alpha isn't going to last forever, especially because everyone has access to the frontier LLMs, which keep getting better, and will eventually beat your fancy harness or specialized finetune.
And also, perhaps more importantly, so you can start developing an alternative to prediction markets and become the new PM; as Scott notes, with superforecaster AI, it's unclear why you really need Kalshi or Manifold or anyone else, with all their fees and overhead. Leave them to the degens, and carve off the socially useful part to do much more efficiently - tokens are cheaper than transactions! This is the big prize, but you need to start now before someone else does it better or commoditizes it.
> I met an AI superforecaster startup founder who told me his AI had turned $35 into $2 million on Kalshi over seven months
Ok… assuming you can’t use that $2mil for some reason, simply take out a $10k line of credit and you’ll have $571 million in 7 months.
If you have the 2mil, congrats you’re 7 months away from $114B - you’re now one of the top 20 richest people in the world.
If this was truely the money printing machine they are saying it is, they would not be talking about it.
Any given trade also has a capacity. A mispricing opportunity can only absorb so much investment before it's no longer mispriced.
A particular trade that can 2x $20k won't be able to do the same for $20 billion.
It's why RenTech capped their Medallion Fund and closed it to outside investment.
If there's only a billion dollars sloshing around on Kalshi, you can't expect to put $1 trillion into bets and take $2 trillion out.
But if these "superforecasters" can forecast just about anything accurately—which seems to be the claim—then they're not limited to any one claim, or even a small subset of them. They can place bets on everything and expect to win most of the time.
This doesn't pass the smell test.
Stock market’s got plenty to spare
from the article:
So this stuff is only as useful with a steady supply of people making less intelligent bets. You don’t have to sell me on it being bullshit, but you are.
1. The liquidity is not infinite to compound that easily.
2. The alpha dries up with more players, even in the year or whatever since that founder started.
You're attacking a strawman. No one is claiming that you can pull off that multiplier at arbitrary amounts arbitrary amounts of times. And 7 months is plenty of calendar time for those arbitrages to disappear, given the attention on the area and the rapid rate of development. (Warren Buffett can't pull off his early trades now either, doesn't mean he was stupid or grifting in taking early investment.)
So you agree this stuff isn’t useful?
Agree, to an extent. The link certainly smells like an ad. If the predictions are on lower volume markets (i.e. no institutional investors with actual quants behind) the compounding power stops. Also that means it can't beat institutional investors. So the reasonable option is to monetize what you can.
He's been interested in forecasting since a long time before there was any money in it, and still hyped Metacalculus (free research project) and Manifold (play money) because he is interested in the subject. If it smells like an ad it is because your ad sense is faulty.
If you had a tool that would be capable of writing all software better than a human, why would you price it by the word (token)? Why wouldn’t you take a percentage?
(This is one of the more interesting questions that came out of Alex Karp’s televised borderline psychotic break rant the other day and it has stuck in my mind even though he is clearly unstable)
The reason is, customer might care a lot more about the prediction than you do, or anyone else does.
For example, your customer might really care a lot about some niche prediction like the number of car break-ins in Walmart parking lots. In practice you won’t have sufficient liquidity in a prediction market to actually profit off of that prediction. But a security company might really want to know the answer to it.
If a model demonstrates being good by winning on prediction markets I may want to ask the same model questions which are too niche to have markets. That said I don't know if niche questions are enough of a revenue stream to be interesting for model providers.
Perhaps they may enforce a knowledge cut-off for information retrieval and price the service based on how recent the cut-off is, and also use the cut-off as a way to guard their advantage on the markets.
Doesn't this argument prove too much? Why does AlphaSense sell their company research instead of using it to trade themselves? Why do people work on open source time series forecasting packages instead of quietly using them to trade?
They do. One guy with at least 7 figure PnL is a well known forecaster, SemioticRivalry.
Only news article I found on that suggests the guy is mostly just market making against bets he thinks are dumb? No mention of AI.
Wait, the original blog post was about competing with top-tier guessers using “AI.” I wonder if there’s more money to be made on the flip side: find the people making dumb bets, at scale, and take the free money they are offering, haha.
The market of trading is smaller than the market of business. I don't think most people have ever actually tried trading for real. The frequency with which you can put in a buy/sell order and nothing happens. Then you raise/lower the price several times and nothing still happens because nobody wants to buy or sell at a price that makes you money is much higher than you would imagine. If you've never tried it yourself.
That said, the very concept of selling such information means that it would eliminate any edge and become zero profit anyway.
That only happens if you're trading very illiquid stocks.
I can't tell you the number of times of put in an order on SPY,QQQ,SPX when I found favorable statistical analysis and I could not get filled at any price below the estimated profit potential. Those are some of the most liquid products in the word. I'm not talking pennies either.
Futures are the only thing for me this has never happened on. However no retail broker I'm aware of allows trading futures though API.
Yes, but have you considered the performance of a few lucky outliers? Catch 'em fast before they regress to the mean!
I used AI to forecast that AI forecasters won't beat the market. We can move on now.
Almost by definition, once AI forecasters are in the market, they won't (all) be beating the market.
But why evaluate AI forecasters by beating the market? Do we evaluate deep learning by whether hedge funds make money from it in the markets? These things have far, far more utility outside of finance.
This new forecasting industry is pushed by very dirty people.
It feels like Scott is taking the bull case here - but like “perfect” information will pervert markets in bizarre ways.
As soon as the prediction market says “this path here has a 25% chance of curing cancer” all sorts of money is moved away from other things.
It will absolutely cause political outcomes not just predict them.
And then of course there’s the cheating element. Anything that’s feasible to change the outcome.
Maybe this just contributes to efficient markets? Or maybe the continued quests towards utopia have dystopic externalities.
But Scott's point is more: why even have markets? Once you have the superforecasting available on the questions you care about, why do you need to publish it for everyone to also react to?
I mean he explicitly at the end says there will be a new era of prediction markets:
"This, then, is my prediction for the AI superforecaster future: for basic questions, your off-the-shelf AI chatbot will be able to offer opinionated probabilities superior to those of any human. For more controversial or bias-laden questions, a new era of prediction markets will smooth over differences in brand and model and efficiently aggregate all AIs’ opinions."
But one reason that you would publish it for everyone else to react was exactly what I said, because if you can show people that a superhuman AI believes a certain outcome is more likely, you impact whether or not that certain outcome is more likely. Which feels like a flaw in this version of the future. That in fact you could overcorrect and make the markets less efficient.
Is there a future where AI takes all of these no-value arbitrage games to their limit and there is no longer a market for this type of behavior?
Prediction markets are just the new ICO, NFT, SPAC etc scam/gambling(scambling?).
Play if you want, and call yourself a genius if you win, but only with money you can afford to lose.
Given a powerful enough AI with enough outlays (eg. a Playwright instant), can an AI cheat by influencing the real world and self-fulfil its prediction?
Eg. Is corp ABC's stock price going to go down by 10% in 30 days?
Orchestrate a disinformation campaign on the 29th day to tank ABC's stock price.
In theory but in this specific example you can make a lot more money on the stock market instead if it's that doable.
In general you should judge different prediction markets differently and read the fulfilment criteria though - some are very ripe for abuse, AI or no, while others are much more ironclad.
This is not even dumb. Unbelievable coming from act....
So this person was known for thoughtful blog posts, but has since become a total AI booster. It's probably a gonzo-like effect through the various subcultures he aims to explore, but it paints a very bad picture of them. Imagine if Louis Theroux became a scientologist.
For Pete’s sake lets outlaw prediction markets already instead of hyping them in crappy greed-posts like this one
I am embarrassed this “rationalist, I’m so much smarter than you, so I know better” asshole Scott Alexander hasn't been skewered yet for promotion of gambling (prediction markets) that people like Trumps son are making millions of on
https://www.theguardian.com/us-news/ng-interactive/2026/may/...
We elected con man so we're becoming a swindler nation. In a just world the Andreesen David Sacks types would be in jail stripped of all their money.
Just out of curiosity, what is your legitimate criticism of Adreessen and Sacks?
Andreesen is full speed ahead on 1984-minority-report surveillance besides all the unethical crypto investments geared toward swindling normies. Anything that looks like it would turn a profit was fair investment for a16z-crypto, no matter the pump and dumping that was going on. His latest interview on the Joe Rogan show was all about how harmful the anti-flock-camera people are, trying to prevent police from solving crimes ! Imagine !
What a strange and scoldy thing to say. I find speaking in terms of concretere predictions the best way to discuss things. It bypasses hyper moralists and scoldy type of people (your post reminds me of such) and directly jumps to truth.
I don't think there's good evidence that it directly jumps to truth. The aim of betting on a prediction market is to earn money. This incentivizes you to bet on whatever you believe is the most profitable, which is disproportionately going to be whatever you have insider information about, whatever you can influence the outcome of, and whatever you can most effectively fool other people into taking the wrong side of.
I kinda like Taleb’s take on forecasters; get another job.
[dead]
Tell me you don't understand Taleb without telling me you don't understand Taleb.