I wonder how big the Pro model is that Google is using behind the scenes to train these smaller ones.
Going on baseless speculation, the lack of accompanying pro models with these flash releases either means: 1) the model is too big to be economical, 2) google doesn't have the compute to serve the big model, 3) their big model has too many alignment issues to serve to the public.
edit: looks like benchmarks are up on https://artificialanalysis.ai/models/gemini-3-6-flash. It's solidly middle-of-pack. However, if you want to be most fair to flash, look at the intelligence vs time per task and intelligence vs outputspeed benchmarks. This is a very fast model.
edit 2: I use antigravity from time to time and in my experience, 3.5 flash is an underrated model, so long as you know what it's good for. It's very good at frontend (much better than gpt 5.5) and it's fast, so it's a great tool for iteration. I expect 3.6 to be no different.
It's also very possible that they know their big model underperforms chatgpt 5.6 and fable by too much, so they are focusing on what they can get wins in like speed instead.
That's the only explanation that makes sense. If it was frontier but cost or compute were limiting factors, they'd release it at an obscene price for the bragging rights. Google doesn't care that much about alignment, and I don't think it's likely to be significantly different than 3.5 anyway. The only reason it would need to be soft-canceled is if it's terrible, and has to end up in a ditch like Llama 4 to avoid shareholder panic.
3.5 pro was clearly a miss. It should have been in prod mid may, not MIA in late July. The brain drain at deep mind is a clear indicator that the people who know the most think that they can’t stay at the frontier.
Antigravity NEEDED to be game-changing. Without the stream of data that Claude, Codex, and Cursor enjoy there is little chance of getting an effective reinforcement learning loop. For the first time in its history, GOOG is at a meaningful data disadvantage, and apparently a cultural one as well.
I don’t think this is necessarily true, did we all forget how much Google cared about alignment that their AI wasn’t able to render a white polar bear?
That and/or the business case isn’t as clear when serving enormous models? You’re constantly stuck in a red queen’s race where your profitability window is increasingly measured in weeks because the Chinese are right behind you.
For small models (which are probably distilled from their big ones) you can serve them economically all the time and not hemorrhage money.
> For small models (which are probably distilled from their big ones) you can serve them economically all the time and not hemorrhage money.
For smaller models, you're competing with DeepSeek V4 Flash. (Which I think is a 284B A13B?) Subjectively, this feels about as smart as Sonnet 4.5, give or take. And it costs $0.09/$0.18 on Open Router, compared to $1/$5 for the latest Claude Haiku. See https://openrouter.ai/deepseek/deepseek-v4-flash#providers The developer antirez of Redis fame uses this as a local coding model.
DeepSeek did some extremely clever research on hybrid attention to get the prices that low, reducing per-user context cache sizes dramatically.
So, no, when it comes to low-price models, the US models probably can't sustain their current margins there, either.
My view then was they are optimising the models for inference ability on their own hardware AND use cases, which is often speed and time to first token.
They've somehow seemed to end up with terrible compute shortages, which again is surprising given how good Google is at infra deployments AND have their own hardware. From rumors out there they are turning down enterprise deals for Gemini because they don't have the compute.
The problem is they're falling further and further behind on frontier class on coding especially, and since I wrote that article it's got even worse with open weights models undercutting them on price AND intelligence.
They include an LLM response with every single Google search, whether it is warranted or not. That scale is, my guess, many orders of magnitude higher than what OpenAI and Anthropic serve. And for Google none of these are paid interactions since their LLMs do not (YET) insert ads into the responses.
So my guess is that Google will continue having compute shortages until the Gemini enshittification starts.
I don't think so. According to some very basic research there are around 8bn searches a day, or 250bn a month.
Let's assume Google serves AI overviews on every SERP (they don't) and don't cache them (they do, afiak).
And let's assume that each AI overview is 2000 tokens (blended input/output), that's 500T tokens a month.
It's rumoured that anthropic is serving somewhere close to 10Q tokens a month.
Now it may be that AI overviews uses vastly more tokens than that per search, but I doubt it based on speed to render the overview.
My very rough napkin math on this is that maybe AI overviews is consuming 100T tokens/month max (after adjusting for caching and SERPs that don't have them), which would be 1% of Anthropic token volume.
There was some recent reporting that a July release of the Pro model got pushed back for exactly that reason. Its performance was not good compared to the OpenAI/Anthropic big models. They are having a lot of problems with posttrain.
When it is cheaper, and the "lower quality" model is adequate for the task at hand.
Plenty of problems have a low(er) skill/intelligence floor, anyone who uses the dual-mode agent paradigm (plan, then act) figures out the second phase can be completed by a less capable model. Even when disregarding costs - speed is important here because the agent can rapidly iterate without human supervision, based on compiler errors, lint and test failures
A coding agent driven by a large LLM can delegate smaller tasks to a faster model. For example searching through the codebase for references, examples, or established patterns. They are treated as tools and don't pollute the main agent's context.
It would be a shame if they cannot beat Kimi K3 or Qwen3.8 Max, both of which are claimed to be Fable-like. If that is true, it will be [or would be] the first time a major American lab falls behind a Chinese competitor.
Google can't compete with China, neither can Meta. Only two labs in the US can keep chucking billions at the frontier race. Everyone else has a real business to run.
China can keep up because it's cheaper to run a frontier lab there. They also have more researchers and a stronger cultural inclination for this sort of thing. And I guess the business case in China doesn't have to work as well as it does in the US.
> China can keep up because it's cheaper to run a frontier lab there
Not sure if this is what you meant, but their training runs are significantly cheaper. This was one of the big shockers from the Deepseek R1 paper. US foreign policy has helped to ensure that the Chinese are compute constrained, so they literally cannot buy the most expensive and powerful training rigs.
This has led to a steady drumbeat of innovations which are not revolutionary on their own but stack together to make things much more efficient.
> focusing on what they can get wins in like speed instead
Speed as a differentiator has always been Google's thing. They (used to?) show the microseconds it took to query & rank web-scale search results. Chrome, notoriously, focused on speed at the expense of resource use. The very many efforts to efficiently speed up Android & its runtime since its inception, and so on...
> their big model underperforms chatgpt 5.6
Possible but TFA claims:
We have started our most ambitious pre-training run yet, for Gemini 4 ...
I wonder if the broad use of AI overviews on Google search results is having an impact. Maybe the numbers make it more profitable to use their compute on several billion searches a day rather than selling API access.
It’s a small multilingual embedding model designed for things like search, RAG, and semantic similarity. It supports a fairly large context window and is designed to run efficiently on a CPU in a GPU starved world.
The interesting part is that it builds on BitNet, using ternary weights of -1, 0, and 1 instead of the usual floating-point weights. That should make indexing and searching large amounts of text much cheaper without giving up too much accuracy.
AI overview is just a summarization of the top 2-3 results. Of course at Google scale that will still need a ton of compute, but the requirement for generating an overview is many orders of magnitude lower than asking the same question in Gemini.
It seems like there are some credible rumors that Google is actually winning in terms of actually building models that work and don't lose money- between how they're able to price them, the TPU advantage and their capex advantage (being able to raise debt + just having a lot of cash - well I said not lose money... more like not go bankrupt).
From the outside they look like they're behind in terms of frontier models, but I think they might be the best positioned to not go out of business when the bubble pops.
Also look at the fact that they've been able to deploy AI-assisted search at google scale. It must be another order of magnitude larger (at least) than the model deployments for OpenAI and Anthropic.
Of course unless you're inside Google it's impossible to know for sure.
In terms of open models, Gemma 4 beats the pants off everything else to the point that paying for APIs becomes hard to justify. Qwen has the meme-share for coding, but it feels much less well rounded. I have no doubt that Google have both the infrastructure and the expertise to curb stomp everyone else, should they resolve in earnest to do so.
Lest we forget, "Attention is All You Need" came from Google.
It also came directly from the university of Toronto, and the university of Toronto seeded all American frontier labs (including Grok (why do you think they could start so fast))
Interesting, glad to hear. We have gemma4 at work, and I was considering localhosting qwen, but gemma4 is so far behind the Opus and Fable I have at home that I've decided to hold off for another model release.
"We have a company provided Toyota at work but it is so far behind the Ferrari I rent at home that I've decided to hold off for another model release."
At 20x the parameter count I should hope GLM beats Gemma! But is it 20x better? Expertise is demonstrated, not by making big models, but by making small ones. Bigger isn't better if you can't run it at all.
It's rumored that Gemini 3.5 flash has a >50% margin, and I'd imagine 3.6 flash is even higher.
I do not think OpenAI or Anthropic are actively chasing margins - though, Anthropic is supposed to be profitable on some form of non-GAAP accounting...
I suspect Google isn't really interested in seeing how far it can get dragged into a race of selling dollars for $0.25, and is more interested to see if it can stay in the race selling $0.50 for a dollar - when everyone else is losing or barely breaking even.
google has to make money. flash is awesome. you can run it free on their infra and the performance and latency is excellent for what you wait and pay for right now, with great perf per watt. every person in the world going to google.com runs it. every query. its far larger than free gpt, localhost qween and what not.
it's their pro that isn't awesome at all. in fact, their pro kinda suck now that everyone else woke up.
It kind of doesn't make sense though, because typically a large org like Google can afford to crush competitors on pricing. They could probably even go toe to toe with chinese model pricing for years without feeling it.
Maybe they don't want to price war with the other labs so they can comfortably maintain healthy margins on selling them compute?
But maybe the TPU advantage is in inference? That's what I assume because the number of compute cycles are going to be all in inference vs training. So they could train on GPUs if they want.
"As of October [2025], OpenAI's compute margins reached 70%, up from 52% at the end of 2024 and double the rate in January 2024, [The Information] said, citing a person familiar with the figures."
As for Anthropic, the rumors I remember seeing for their API margins were more like 85-90%, but I don't have a reference at hand for those. But once you know the API is wildly profitable and the subscriptions are roughly break-even and not even a big slice of their income, all of the investment makes a lot more sense.
It says the original report was in the Information, which I can't see, but I'm skeptical that they includes the training cost? And how much that changes the figure?
Pre-trains take a huge chunk of your compute offline, incurring both an raw expense (24/7 max power for all training clusters) and an opportunity cost (could have sold excess compute during that time). They also don't come with any great guarantees, as lots of techniques look good on small scale and crumble or plateau once scaled.
I think it's 2. I frequently get told there's no capacity for Pro and the query is answered by Flash with extended thinking. And tbh it's hard to tell the difference between the two, especially if you're not coding with it.
My hunch is Google is trying to integrate a fast and relatively cheap AI across search and every other surface of their product suite. And for that objective, a model that can move faster while being accurate and cheap enough is more important to them than producing a frontier class heavyweight model.
Google somehow managed to snatch defeat from the jaws of success with their AI products.
They literally forced me and my company out of Antigravity by phasing out AI Ultra subscription without any proper product follow-up. Antigravity IDE cannot even have poweruser subscriptions now from Google Workspace an Gemini Enterprise Agent Platform cannot be attached to Antigravity IDE.
Gemini Enterprise Agent Platform has an incredibly abysmal setup process, and if I want to limit spending per-user I have to create projects per user. The fact that you cannot activate Anthropic models on it if the billing still has free credits is almost a joke.
I was a big proponent of Google and Gemini, but they left us reeling with their abrupt product decisions. Forced us to buy $200 subscriptions directly from Anthropic/OpenAI.
Also a big proponent of Google and Gemini, but their stubbornness in artificially splitting their consumer and enterprise products is extremely annoying. It's pretty weird that I have access to more powerful tools when using my personal Google account compared to my corporate Google Workspace account.
It's literally the meme of the MS org chart pointing guns at each other.
The GCP team wants their slice, the other team wants some otjer slice, and so on. Everyone wants some crap for their promotion package.
It's no wonder Meta has shit the bed even worse.
It's also why Google still releases actually decent, useful models despite the product being such a hilarious mess. A lot of the time Gemini models have actually been better as production LLMs as part of LLM-based production applications than OpenAI and Anthropic models when it comes to the complete cost:quality:latency:adherence picture. And they still are. We have products in prod that use Gemini because they're better than any other model at the specific task. But we wouldn't dare use it for anything coding related, or even just as productivity tool to rely on, because as a consumer product it's a joke.
> The GCP team wants their slice, the other team wants some otjer slice, and so on. Everyone wants some crap for their promotion package.
I got a Google One plan for Gemini, but it came bundled with YT Premium lite, and that somehow made it impossible to renew YT Premium for 30 days. I suspect different teams stealing customers from each other.
Google also gave away 1 year Gemini plans with Pixel phones that either did not work at all for existing Google One users or messed up subscriptions by downgrading your account to worse plan, or making your existing paid time shorter if you been on cheaper plan or recently changee countries. Etc.
Like when you try to give Google money they try to squeeze you as much as possible.
At the same time you can get 5 time more limits for free just by registering 10 free Google accounts.
Exactly my experience. I'm building an AI document-extraction platform, so I had to benchmark a bunch of models — on the cost:quality:latency:adherence picture, flash wins hands down for structured extraction. (Caveat: I've only tested the three US labs and Mistral.). So like u said, totally viable in prod for a relatively static tool. Didn't build the tool suite with gemini, but if you use service mode it currently mainly runs on flash.
Haven't done any serious coding work with the flash models though — but I'm seeing more and more HN comments from people who seem to have picked it up for that in the last couple of months.
As someone who's been using Workspace as a personal email account for over a decade this has been such a struggle forever. Just lots of odd limitations to feature sets all over the place.
When they swapped Google Assistant for Gemini as the default voice provider in Android Auto it was so annoying. My wife's non-work space account can get Gemini to do the normal things like play music and what not, but my Workspace one can't do much of anything at all. I can talk about nearly any random topic with it, but getting it to change the playlist, nah, can't help you there.
It's no surprise to me to see them fumble actually supporting a lot of the consumer features of Gemini into Workspace.
I'm in the same situation. But I was shocked discovering it goes both ways: many new Gemini functionalities are only accessible using a consumer account instead of a Workspace account. Also, Gemini is now the only major AI assistant with no support for MCP connectors. Instead of adding this to the core product, like ChatGPT and Claude did, somebody at Google decided that it was smarter to add this fundamental feature to a new product instead: for enterprises this is Gemini Enterprise (which is a product completely different from the Gemini app); for consumers this the new Gemini Spark agent (meaning that you can use MCP within Spark but not within a "non-agentic" chat)... It's clear to me this a symptom of Google shipping their org chart, which is a disaster from a product perspective.
Microsoft’s Copilot products are a very similar situation where the enterprise and consumer (and GitHub) features only make sense if you think about the org chart. Both companies need stronger top down product thinking.
This drove me bonkers. You can enable play music (etc) in Android Auto for workspace accounts by enabling apps in Gemini. From memory (looking at the settings now, not 100% sure of the magic steps required), but go to admin.google.com, go to 'generative ai', 'gemini app', and 'apps settings', then turn on 'other Google apps'. This lets you play music (and other things) in Android Auto.
Ah, that could be it. I saw that "Other Google apps" and didn't think that would mean Spotify, but I guess its Android Auto or Google Assistant stuff. I'll give that a try, thanks for the tip.
Also have a workspace as a personal email and ended up getting a personal gmail just to try out the subscriptions before I gave up.
I have multiple anthropic and OpenAI max plans. For Gemini I just use my Cursor $200 a month plan (which also gives me the ability to try grok, conductor, etc)
Apparently you cannot turn off using your data as training data with gemini. This is in line with Google's general privacy policies and it's seeming need to create a stasi file on every human.
You can turn it off but it's an all or nothing switch, if you turn it off everything will basically become a temporary chat and all past chats are deleted
It's absolute insanity. They have all the resources to have been able to lead from the front with this new technology. They have the products and users to integrate this technology into people's already existing lives. But they keep fumbling.
They're not even benchmarking against other models now, just against themselves - which tells you everything you need to know.
There is absolutely no loyalty when it comes to coding. Nothing could be more common than people threating to jump ship whenever another frontier or open source model comes.
Google is clearly able to keep growing their free and consumer and small business use cases. Unlike corporate coding, we actually have evidence that solo and small businesses can actually see productivity gains.
Anthropic and OpenAI need to stay dancing like mad, because it's their source revenue which underpins their investments.
Why does Google need to shove something at the top at the same desperate cadence? Other than "recursive self improvement leads to AGI" it seems perfectly fine if they push out something dramatically better every year and half.
> Google somehow managed to snatch defeat from the jaws of success
This is still very early days. Who is "on top" has flipped back and forth many times already. The next frontier model release (from whomever) will change things again.
I don't think it's that early tbh, agentic coding has ~90% adoption in the US.
Claude Code has largely won individual developer mindshare and has been on top ever since it came out. The benchmarks change, but almost nobody opts to use anything other than Claude IME when I ask them. Enterprise is more competitive since they care about costs and other things, but developers leaning towards Claude puts a thumb on the scales there.
The product doesn't have much lock in, so it is possible to dislodge Claude, and Anthropic could (and some may argue is likely to) just shoot themselves in the foot again and again and again, but Google has never been particularly good at enterprise sales, and they have never actually been at the frontier of intelligence.
I think Google's incentives have mostly about building models for their products, which makes them focus more on the cheap end, and while they need that, it feels like the Innovator's Dilemma is biting them here.
I own a lot of Google stock from working there in the past and have been quite happy about their trajectory up until the last 6 months, but I am getting pretty antsy about their AI story these days.
> Claude Code has largely won individual developer mindshare and has been on top ever since it came out.
Claude Code's success is not due to the agent but because the model is considered the best for programming and is very heavily subsidized, compared to pay as you go API prices. Consumers and Enterprise are not really locked in and will go where it makes the most sense.
I think they have almost no loyalty by actual developers.
There is no reason to be loyal....There is no moat.
Basically you may choose to drink brand A water bottle, brand B water bottle or tap water. Oh and you might choose the glass water bottle if you use API/Fable.
There’s no reason to be loyal, but I guess it’s a bit like any tool, once you get used to how one works why would you change to another? There is some stickiness with an LLM + harness.
This isn't a perfect survey, but I am a Codex user and most of my employees are pretty stuck in their workflows and were not really interested in trying even when I was saying good things (even pre-Opus 4.8).
When we work trial people, 100% of people ask for Claude rather than Codex or anything else.
When I talk to people at non-AI tech events everyone basically says they use Claude and have not tried an alternative.
Developers writ large are actually not that interested in trying multiple tools, they like customizing their chosen tool and tweaking it forever.
I think developers are as susceptible to brand marketing as everyone else. It's why almost everyone has a Macbook.
Exactly developers can switch to another coding cli and the learning curve is close to zero. Mindshare without switching costs is just a popsicle in the sun.
Every model has its strengths and weaknesses, being loyal is suboptimal unless you mean being loyal to all of them, which is why cursor would have been well positioned before it got acquired. Now you have to jump through hoops to call Gemini from Claude from codex. Yuck.
Claude code was one of the first agentic code tool and when openai release models similar in performance they didn't do as well in their tools (now codex)
Claude Code (or any other model/infra/harness co-design) is not subsidized in any normal use of that word. It's trough filling (I've written this up in lurid detail so I'm only going to do it again if anyone cares).
Early days or not, Google fucking deleted my IDE and wiped my settings. It took them days to roll out a fix, by which point I had migrated off Antigravity.
I guess? Maybe I'm alone here but I don't feel like Fable is particularly more useful than Sonnet most of the time. I feel like the LLMs are good enough for the majority of uses and the hyper expensive premium ones are way into diminishing returns. At this point with Kimi being as good as it is, if they jack up the price any more I'll just go open source.
I don't think that's true if the reason a company has left the vendor for given model by making it hard to buy. Enterprise IT is enough of a pain in the butt that people will forego the new shiny to avoid the old painful unless it's genuinely better. As you say though, the best frontier model flips regularly, so companies won't go through the hassle of deploying a model if it's proved horrible to do in the past. They'll just skip that model because their current one is fine.
Model quality is only one aspect, the bigger problem is making it work in a fully integrated enterprise platform, and Google has always been lacking when it came to the latter.
At this rate, if Google has a flagship model, you're better off plugging it into a competitor's tooling than hope Google figures out how to use it.
Aren’t the subscriptions extremely subsidized and burning cash for Anthropic and OpenAI? A reasonable explanation is they’re simply abstaining from the war of attrition, especially given cheaper comparable models are breaking the illusion that the “frontier of intelligence” has any kind of per token margin.
I think the subscriptions pay them back in spades because the same dev who maxes out their subscription on their personal account transfers that exact behaviour over to their enterprise work and - guess what - it's all billed per token there. This is a large reason why corporates are reeling from the cost right now, I think.
This is hotly debated and completely unclear. Let's say Anthropics Opus models cost the same to serve as GLM 5.2. GLM 5.2 is 4.4$/MTok while Opus is 5.6 times more expensive. Assume that GLM 5.2 is served at essentially zero margin. Then Anthropic has >80% margin on API pricing. So even if an average person with a subscription pays only 20% of the API price of their usage, Anthropic makes money on subscriptions.
And the real numbers could be better for Anthropic. It's feasible Opus models are actually cheaper to serve than GLM 5.2 because Anthropic have optimized the hell out of inference.
Sure, but then why wouldn’t I use GLM 5.2 at cost or K3?
I guess that’s the big question, will people pay a big margin long term to use their end products / models or will AI tokens be commoditized by many competing players. For coding if I had to pay API costs I’d switch in a heartbeat, enterprise maybe more reluctant?
Possibly, but aren't the tech giants positioned to win a war of attrition? Then again, they're more likely to sit on that cash and wait for the opportune time to buy a frontier lab.
they're subsidized if you max them out, i'd imagine most users are paying $20 for maybe $2-5 of tokens.
anthropic probably has more customers that use more of their sub, but for open ai where a lot of their subs are consumers through chatgpt.com, they have a lot of free money to work with there
Google Gemini agy is not allowing you to use your token via your own harness. My own harness is far more efficient than agy. They can take a simple stance - if you exceed your token limit they block you - with the 5 hr limit they are already doing this . so there should be no reason to block you from using your own harness - if you are more efficient - you gain - if you are less you lose .
They are not allowing me to hit their endpoints which agy hits - it's frustrating . i tried to hack it with gemini itself. what i love about gemini is it's so encouraging and ready to help you - even against the agy client : ) .
Even though im so frustrated with this - i still love Gemini for some reason ! Most encouraging model in the world!
> I was a big proponent of Google and Gemini, but they left us reeling with their abrupt product decisions.
Likewise. This seems like a common feel. I have at least spent $4000 and likely a lot more on Gemini API because I really wanted them to win. I gave up.
I am going to ask a very direct question and only because I am curious.
Why do you care? Why would you spend your own money to a multi trillion dollar company so that they win their own "war" against another multi trillion dollar company?
Please don't get me wrong, I know the question can seem a bit negative, I am really just curious.
No, it's a fair question. The answer: I believe(d) in Demis Hassabis's vision of AI
Although, I think saying 'wanted them to win' was not accurate. More like, I stuck with them hoping it will get better, and it did get better in many ways, coding was not one of them.
This was exactly my thought process. Coupled with a less idealistic one: mid 2025 if you looked arena ELO scores google seemed to be dominating and I was sure the trend would continue. But they really dropped the ball on coding and tool use.
That being said, controlling android and apple mobile devices is kind of a big deal. And their video models are still top notch.
i've got to wonder how much of this is intentional, and how much of this is just google being their usual terrible selves at anything consumer-product related.
Google's biggest and most important customer for all this AI stuff is google. Do they actually want other customers, or is having other people use their AI just an annoyance at this point, where we use up compute that they'd rather use internally...
Life gets simpler and better as soon as you stop giving money to Google.
It’s so silly that individual people still use their shit. Corporations, I understand - they always choose the most mediocre stacks and tools by default. But why people choose to bring the mediocrity of Google into their lives is beyond me.
> they left us reeling with their abrupt product decisions
This is what happens when you put a McKinsey consultant in the role of CEO of an organization where product managers run the asylum instead of engineers.
It's funny that they triggered the infamous "Code Red" moment in OpenAI when the 3-3.1 models came out. I switched to using them for a lot of single-shot LLM calls because they were fast and cheap. Their only area that they were lacking in was agentic/tool calling.
Needless to say 3.5 was a disappointment. Curious to see 3.6.
> Google somehow managed to snatch defeat from the jaws of success with their AI products.
HN lives in a bubble.
I have German/Italian/Polish clients virtually all use Gemini and NotebookLM. Talking insurance, banking, consulting, legal.
The real world doesn't look at pointless benchmarks on writing react tailwind crap, they are already google suite users, get the tools, test them and adopt them, end of story.
It's going to be like with angular, never mentioned on the net, widely used in the real world.
I have friends and family who use Gemini, but entirely because their Pixel phones came with a year of it for free. No other reason, and they will most likely never pay for it.
They literally forced me and my company out of Antigravity by phasing out AI Ultra subscription without any proper product follow-up
That sounds awful.
For those of us who don't follow the AI hype cycle, what does that have to do with the topic of this thread: Gemini 3.6 Flash, 3.5 Flash-Lite, and 3.5 Flash Cyber?
It's a bit disheartening to see no comparison to other models here - and I'm not sure this pushes the curve anywhere. 3.6 flash is more expensive than GLM 5.2 - but seemingly worse, although this post is really light (lite?) on details.
It seemed for a time that Google had finally gotten the ball rolling, but I'm doubting that more and more as time passes. We'll see what happens with 3.5 pro I suppose.
How does your comparison work? It places Gemini 3.6 Flash Medium above GPT 5.6 Sol High and Fable 5 Medium, which makes me skeptical because that... would be making headlines that I'm not seeing right now.
I have created various questions/tests and put the models through the same tests.
I record whether the answers are correct, and the generation stats (costs, latencies, tokens used, etc.).
I have no idea why the Gemini models do so well.
I have recently added new tests, whose sole purpose was to find some cases on which Gemini 3 Flash fails (I don't like cherry-picking models or tests, but I also find it strange Gemini Flash models leading in accuracy). I made a more complex coding/tool-usage test, that I expected it to fail, it did fail it once locally in my debug tests, but when I finalized the test and ran the entire testing suite for all models, somehow Gemini 3 Flash still got it right...
Gemini models are REALLY intelligent (and they are actually my favorite model to use via the chat app to ask questions), but they somehow fail in real-word coding tasks where they have to modify files, check results, debug, etc.
My tests harness provides a lot of mock data, and limits the number of actions a model can choose from. I am starting to think that maybe the models are not bad, just that the coding harness are not optimized for those type of models, and Google doesn't really provide their own "Codex".
Oh, and I've also added weights to different categories, so Coding and Tool usage categories influence the score more. This done both to better account for how most people are being used, and also to reduce Gemini's dominance in general/domain specific knowledge.
So yes, Gemini models are at the top, even if I actually (not proud of it) tried to make tests that actually favour other coding-focused models.
Interesting, well it'd be interesting to check out some individual examples where Gemini beat the others.
Also, would be great if you could add GPT 5.6 Sol XHigh and Fable 5 High as well, just to see if at least those beat Gemini which is currently your #1.
I don't like to divulge tests, but one of them is a chess puzzle.
> would be great if you could add GPT 5.6 Sol XHigh and Fable 5 High as well
I would like too, but I avoided them for several reasons:
1) Cost - this is a hobby project, those models would cost tens of dollars for each benchmark run, multiply this by tens or hundreds of models and ...
2) Time - the high models are already taking a really long answer to respond (5-10minutes per question). I run each question with 3 repeats (run the same test three times), so it would take 30 minutes per test. If I change my tests, methodology, or add a new test, it would take a really long time to run the benchmark. Also, I like having results immediately when a new model is released, now I can post within 30 minutes of a model's release the benchmark results.
3) High reasoning usually does WORSE on most tests - if you look at the leaderboard, it's sometimes counter-intuitive, but models with high or max reasoning usually do worse than medium and low. This is because the questions are quite targeted/direct, and the models overthink the question and miss the solution. Or the long thinking context makes them perform poorly. The generation tasks (SVGs/HTML animation) are usually better with longer reasoning, but short code fixes, trivia questions, puzzles, etc. are answered by low/med reasoning with more accuracy in general
Also, Fable is borderline un-testable, it refuses to answer many questions, so it scores poorly anyway.
Gemini scores 21/22 because it answers all tests, and it does them correctly, consistently. The only failed test is I think because it miscounted the lines in a file, when responding on which line the bug was in a code snippet.
It’s really surprising. When Apple announced the multi-billion dollar deal with Google to power Apple Intelligence I thought great things were coming. Instead we are getting more and more bad news: delayed Pro models and AI leadership leaving. I wonder if Apple know something the rest of us don’t know or if they are already regretting their decision.
Besides Apple apparently making Siri AI model agnostic, the choice to go with Google was almost certainly for practical reasons. Google is a low-risk established player that already has a long work history with Apple. Google also isn't in an existential battle to establish themselves, Gemini still amounts to just another project at Google. There is tangible non-zero risk that either OAI or Anthropic will be gone in 5 years, or will be forced to leave Apple high and dry to save themselves. There is almost no risk Google will be in either such position. And worst case scenario, Google has incredibly deep pockets should Apple pursue a "refund."
What Apple wants out of Google is Siri that runs at 8gb ram and isn’t a horrible embarrassment that feels like a primitive markov chain. Given how good Gemma 4 is, Google can squeeze some serious performance in small models. Whether they can make bleeding edge models is irrelevant to Apple.
As someone on the Apple beta.. the model is almost completely irrelevant to the experience. Apple has gone and done Apple things by nerfing the experience so completely that almost any model in the past year would be fine. I still reach for ChatGPT/Claude/Grok constantly instead of the AI toy that Apple calls the new Siri.
Apple isn't counting on their model to be a frontier coding and cowork model. Gemini is perfectly fine for the tasks that new Siri is supposed to be doing.
The one thing I've found google's models to be the best at is proofreading text in non-english languages. Probably because I imagine they have the most training data for it as Google probably has the most complete archive of the internet.
My usage of GLM 5.2 has been defined by slow throughput and flaky providers, in many (definitely not all!) applications a dumber, faster, and more consistent model makes more sense to me.
I've never questioned the word lite before because it's existed my whole life.. So does it make sense? Why does it exist and where does it come from? More than coming from "light".
I am growing tired of these pelicans posts every time a new model is published. Feels to me like low effort personal brand promotion. Just sharing my 2 cents.
+1. First thing I look for in a model announcement thread. I actually came across this one an hour ago and was sad there were no pelicans yet.
It's a decent heuristic because the better models generate better pelicans. That's all. Nobody sane is going to make a bet on a model based on a pelican. But it's cool, it's tradition by now, and it's a semblance of a good first impression for new models.
But how else am I supposed to know when we've reached AGI, until I see an absolutely flawless pelican?
All of the pelicans so far have had really weird flaws / quirks so I am always a little interested to see how well these models perform at this task, since I've seen all the past pelicans and have some anchoring.
Seeing a truly flawless pelican would tell me that the model has true visual reasoning capabilities as well as good taste.
Not only does it give you a super easy-to-grok understanding of the model quality just by looking at the image, but when you compare tokens and costs (both input and output), you really get a good, simple COST x QUALITY evaluation across models.
At this point, it's kind of a hackernews thing. Simon posts them as a single comment in the relevant thread. It's okay for this place to have a little bit of a sense of community, and you can just ignore the comment.
I agree to rednb that at this point it feels like rather obvious brand building, but also, I agree with you that some value is in it.
It does not feel all that authentic though, and it's good to react allergically to lack of authenticity. Bad for a lot of business models, but good for humanity.
Sorry mate, but you sound jealous in all these replies that the Pelican domain isn't your gig. The below is as labored as nitpicks ever get:
> It does not feel all that authentic though, and it's good to react allergically to lack of authenticity. Bad for a lot of business models, but good for humanity.
I thought the Gemini 3.5 Flash Lite response was quite telling myself. I personally like the Pelican SVG test, to me it is still a charming snapshot of model performance anecdata. No one would argue it's rigorous but I don't think it was ever intended to be.
I get people burning out on the pelican SVG test alongside the rest of the AI burnout, but I guess for myself I'm just choosing to keep enjoying it while I still can.
Here is a different opinion. I’m always looking forward to see the pelican whenever a new model is released. It’s plain simple to understand, memorable, subtle enough in terms of details, and my favorite part is that you have been doing them consistently long enough for it to be useful for comparing almost everything with anything.
Yeah. It's something I can do myself in a couple seconds if I want, also on more varied SVG scenes. If this is going to be a benchmark people turn to I'd like to see more effort put into it than just a one-sentence prompt.
Disagree. They're a nice tradition, but besides that, they're a useful way of eyeballing improvements. I realise labs are likely to be training for Pelicans - but if they're all training for them, the differences in the results are as indicative as they were before labs trained for them.
The 3.6 Flash pelican is just about the best I've seen.
Your comment reads very pedantic with a hint of jealousy. The pelican and xbox controllers are great ways to see how well it can follow direction dealing with svg a difficult format for LLMs to use and testing their spatial vision awareness.
I'll split the difference. When it's a blog post there's usually an interesting observation or two, but if it's totally automated? Maybe just do the ones with a post.
A piece of the frame is missing between pedals and back wheel. The frame of the bike passes through the bird. It also puts a cap on the bird's head, and a fish in it's mouth.
The fish and the cap where always added when I asked an llm to improve it's first attempt.
This continues the trend in LLM progress of better=more stuff
Edit: I wonder if this is a function of the reasoning training, where more tokens/ stuff is rewarded.
3.6 Flash would be a great model at 3.0 flash pricing. At this pricing, its thoroughly trounced by about 10 models on cost/performance including Grok 4.5.
3.5 Flash-ite would be a great model at 2.5 flash-lite pricing, as is, its trounced by many models including Deepseek v4 Flash.
As is, they are thoroughly outclassed for most usecases. I will say the one area where i do see Gemini punching above its weight class is in tasks that are effectively "Google this for me" / knowledge stuff. So it does have a role, and I do use it. So while I think Google is still in a strong position overall, they are really stuck as a tier 2 AI player right now with text models. They are tier 1 in bio, images, and video.
Could also be that they are pricing it at levels where they actually make money. Without seeing the behind the scenes compute cost on all of these its hard to really judge.
That being said with any open model we of course do know the total cost (or estimate)
3.5 Flash was always too expensive for a "flash" model. They marketed it as "near frontier" level, but there are several order-of-magnitude cheaper open models that compete with it.
In my tests, 3.6 Flash is NOT more token efficient, so it actually ends up costing more than 3.5 Flash, even with the output price reduction.
EDIT:
It less less verbose in final output though, but it reasons more.
I assume the optimization comes when you have long-running tasks with many tool calls, and by reasoning more, it reduces the number of tool calls needed.
It seems like they're sticking to a static pricing plan that was made when the only relevant competition were the US labs (im not counting deepseek 2025 as serious competition -> glm and then kimi on the other hand, now that's a different story).
Pricing often reflects what the vendors (expects) the customer is willing to pay. It seems that Google is still trying to find their niche in the market.
I wonder if this is a plateau towards the real pricing of AI, if you layer in gemini-2.0-flash at $0.10 / $0.70 then its a 15x price increase to 3.5/3.6 flash. But it hasnt gone up again which is interesting.
I guess they meant to release Gemini 3.5 Pro shortly after 3.5 Flash, but then Mythos/Fable and later GPT-5.6 came out with higher performance than 3.5 Pro, so the managers decided not to release it.
It's kind of ridiculous how good these are getting. 3.5 Flash lite is pretty comparable to Opus 4.8 (at least for the couple tests I did) while simultaneously being 6x faster and 19x cheaper.
Not for me personally. While setting up a custom website, 3.5 Flash introduced tons of bugs that Claude had to fix. The website has about 3,000 lines of code spread across multiple files, and Gemini somehow couldn't do frontend changes without breaking things. Sharing my 2c, but I've stayed on GPT 5.5+ and Claude Sonnet/Opus 4.6+. Anything past that from those two have been bug-free, but Google's latest hasn't been.
Personally, I often paste screenshots into Claude Code of the application it’s working on. And I’ve even had it work autonomously on something and regularly grab its own screenshots.
Or sometimes I will have tables, charts, or even screenshots of text that I would otherwise have to have another step to OCR or type out.
Multimodal saves me time on a regular basis. Not sure it’s a game changer, but just lets me communicate with the model in all sorts of ways that would be harder otherwise.
At least 30% of my queries to an LLM include deciphering something from an image. If I’m coding pretty much a hundred percent of bugs include some sort of image.
It'd be interesting to know how much the Intelligence as a Service angle serves as a value-add in the minds of Google's executives.
You can get decent open-weight models now. That's not difficult. The difficulty is 1) running them and 2) compliance.
My company runs Claude on GCP's Vertex AI solution. We're in the US healthcare IT space, so the models need to be from somewhere that American healthcare agencies and companies have traditionally been okay with sourcing code from - which means the US, Canada, and maybe Europe. The stuff that handles PHI/PII must be in the US. The expense of hosting is more of a PITA than most customers want to go through this early in the technology's lifecycle, and intelligence gains are simply a matter of degree for most business tasks.
In theory, we could find some open-weight model (likely from China) for our development agentic work and host it anywhere you can host AI models. We don't, though, and I think Google, OpenAI/Microsoft, and Anthropic see that as the core of their business.
Is that statement based on token price? More and more it seems that $/token hides as much as it reveals. Token efficiency, tokenizer differences, etc. I'm not saying that you are wrong, I am just saying it is becoming a bit more difficult making statements like this without a bit more research.
Basically, you avoid anything dynamic: model change, tool change, etc it's also important that your system prompt or main prompt doesn't have non-static data like the date/time/place or someone's name (the person you interact with in a chatbot for example). That should be left to tool call or search.
I just put the varying parameters in a trailer prompt and have them change every time. It doesn’t matter because the cache is prefix keyed. You lose caching for the last 20 tokens or so but that’s not a big deal. Moving it to a tool call makes it too slow (needs full roundtrip).
If you’re constructing the prompt you don’t have to jam everything together you can arrange it appropriately.
All of the models, you need to have a consistent input to get the cache hit. So if you are chatting with a document, and change the system prompt, it will be a cache miss, even if the rest of the items are all the same. If you even pass in the document in not the same order as the prompts, it will be a cache miss. Or if you add tool calls or structured outputs, it will be a cache miss. (Since those generally go at the beginning of the prompt call, not at the end.)
Most of the time when reading documents from URLs directly it will never cache. (Need to typically pass in the bytes directly, or use the provider document store index.)
Gemini has a 4096 minimum token size with the 3 version models before even getting a cache hit. OpenAI it is lower (1024), and is automatic, but only happens in increments of 124. Anthropic can also get cache hits at 1024 tokens, but you need to explicit ask for it (and pay extra).
Caching by default typically lives for 5 minutes since the last cache hit across providers. But some of them you can ask for longer. AWS for Anthropic models can be tricky with multiple endpoint routing, so can get cache misses if it happens to route to a different endpoint.
That’s part of why, since Firebase, I’ve tried to never depend on Google products for business, especially not GCP.
Features stay in Beta for ages, whatever that actually means, and released ones get deprecated things fast.
Where some of the competitions treats deprecating entire services as "let’s not put it on your frontpage, put deprecation notices all over the doc, and politely ask new users not to start new project with them".
its not always that simple. dropping in a new model is trivial, but highly specific workflows may rely on specific _invisible_ aspects of a model. when that model gets deprecated, the workflow needs to be rebuilt/re-tuned to work with a different model.
google's inability or unwillingness to provide stable timelines for model deprecation makes it risky to build complex workflows using their models
it does seem to be moving in that direction. There were really specific things (large, complex json outputs) that gemini-2.5 flash was basically the only model that seemed capable of reliably for a long period. gpt-5+ has covered the usecase for us now pretty well but still evals slightly below what 2.5 could do
You would be surprised how much of a difference the model makes for certain niche tasks.
For my use case, `gemini-3.1-flash-lite` is ~20% higher accuracy than the next best model of comparable cost (considering both proprietary and open-weight alternatives)
Well it is a bit surprising that 3.1 flash-lite could be better than deepseek-v4-pro (cheaper output and way cheaper cache so might cost less for quite a few use cases).
They are not anywhere close according to pretty much every benchmark (even v4-flash is considerably ahead and its way cheaper than flash-lite). Maybe tuning prompts/tools/etc. might be useful?
"Intelligence" being what, math? Coding? Unfortunately there's a billion use cases for LLMs whose performance is not at all captured by the popular benchmarks they're all trying to maxx.
There are plenty of 3rd party providers hosting deepseek models, if you don't want to use the 1st party API. 3rd party providers are generally slightly more expensive, but still quite cheap compared to other models of similar vintage and size.
They know that there's big enterprises that will have a strong preference to work with another big enterprise instead of relying on a younger company. At least that's why I think they believe they can do this sort of thing and get away with it.
same here. our production workloads was on Gemini for 2 years. seeing Google unilaterally dropping perfectly fine models and charing you 50x more for worse results is not good.
I'm running price-sensitive data extraction workloads on flash 2.5 and its still the king when it comes to accuracy + cost, all the gemini 3 variants perform a bit worse and cost a lot more. Low-key freaking out, ngl
And somehow, the most annoying is not even the price hike, but it is that is you expect to build a product on any of theirs models, they spend their time being deprecated and you have like to be on the lookup to start from scratch selecting a model and fitting it every year or so...
Impossible to have any stability...
I felt the same way about openai's text-davinci-002 and code-davinci-002 (gpt-3.5). They were amazing completion models and openai basically dumped them with no equal cost or equal performance replacement. Instead all their models are opaque with no ability to work in completion mode where one actually controls the text input to the model.
These days no company even has completion models where one controls the text input fully. Worthless.
same I just switched to OpenAI after using flash 2.5 lite for almost everything at our company. We spent thousands just to build this workflow now Google says screw off
No word about updating Jules, which is still stuck on 3.1 Pro. I get that it's probably niche but I've really appreciated basically being able to give directions to Jules on my phone, then reviewing and merging a GitHub PR fifteen minutes later. It's been great for getting some progress in on a few personal projects during my commute when I can't exactly pull out my laptop.
Shame. I'm on the $20/mo Gemini Pro plan because the 5tb of cloud storage and the youtube premium lite were good enough perks, and my coding complexity needs were light enough for me to overlook Claude or Codex. But Antigravity is working better than Jules and it's basically giving me a taste of what I'm missing and it's harder to justify not trying out the competitors.
I do not, however I am curious about Jules support. I didn't know if this was a dead project or not. Seemed really interesting but then I didn't see much development/announcements/discussions around it. Last update from their changelog was as you said 3.1-pro support in March.
Wow - Google does not even bother to show benchmarks of these models compared to the frontier and Chinese labs - only against previous versions. I'm not surprised. Having worked there for years it was amazing just how inwardly looking the company is.
Really, what's up with Gemini still not supporting connectors/MCPs/plugins/whatever-they're-called-this-month on web? It makes it a non-starter for any kind of serious use.
Their naming scheme is confusing. Branding has never been Google's strong suit and their marketing copy is pretty bottom-of-the-barrel[0]. Anthropic has a pretty clear set of models but Gemini decided to rebrand their Flash as Flash Lite (and presumably the future will see a Flash Lite Mini, a Flash Lite Mini Nano and a Flash Lite Mini Nano 3B) which confuses the pricing to high hell.
This plus the Vertex, AI Studio, Gemini, Antigravity. It's honestly too confusing to use. I need to use Gemini just to decide on which platform and which model to consider.
0: Famous Kurian Tweet: "We're announcing Duet AI for Google Workspace will now be Gemini for Google Workspace. Consumers and organizations of all sizes can access Gemini across the Workspace apps they know and love. We're introducing a new offering called Gemini Business, which lets organizations use generative AI in Workspace at a lower price point than Gemini Enterprise, which replaces Duet AI for Workspace Enterprise."
I have a side business selling custom fingerprint jewelry and I use gemini nano banana to clean up customer submitted fingerprint images. This was a step I used to do by hand at 10 - 15 minutes per image and nano banana is the first model that is able to do the task (it is astonishingly good at it). I can't wait to see what the next nano banana can do, hopefully its released soon.
Their shop is linked to in the bio. They don't seem to have a privacy policy up on the site. When in the checkout form it links to the generic Shopify privacy policy. No hints to the fact that uploaded images are processed by third parties, as far as I can tell. That should be corrected for sure.
Google seems to have anorexia when it comes to model intelligence. They have an internal hard constraint on price per token it seems, and they are trying to squeeze out intelligence with limited compute.
I wonder if there is something with their TPU cycles that makes them want to postpone training a new model. My guess is that they have been on the same base model for 6 months and they may have waited for the next gen TPUs to train Gemini 4, which greatly limits how much intelligence they can increase and forces them to do cost efficiency increases.
Google really needs to get their product strategy together. The discontinued gemini-cli and introduced antigravity-cli which is a downgrade IMO and the sooner they can partner up with AWS and release the gemini models via Bedrock the easier corporate/business which has strict data protection rules can use their models and make them available for internal engineers.
It's a one thing to research and improve the model, but if they ignore the ease of access and multi-availability of their models in different ways they are going to fall behind again.
Google has not changed. Following two facts are like tautologies by now.
1. Their AI efforts are very fundamental research oriented. They are really good at it.
2. Their productization sucks. The end products gets little attention compared to competition. It can be canceled at any time. You should never build anything around Google only APIs, AI or not.
Because AA Coding "Index" consists only of two benchmarks (Terminal-Bench v2.1, SciCode) and generally fails to be meaningfully representative of agentic coding capabilities.
When? It literally says on the page for Gemini 3.6 Flash "Artificial Analysis Coding Index represents the weighted average of coding benchmarks in the Artificial Analysis Intelligence Index (Terminal-Bench v2.1, SciCode)"
DeepSWE and FrontierCode are more realistic if you read up on what they actually measure. But the most realistic is to try it yourself. Benchmarks can only vaguely represent typical usage, and how you judge the result. Giving the same real task you have to a few models will make you understand them better than chasing benchmarks.
Could be a good tradeoff for the flash model though. 3.5 -> 3.6 is a tiny bit cheaper and maybe faster?
artificialanalysis.ai has it going from 165 tps -> 304 tps. openrouter.ai needs more data but it has it going from ~100 tps -> ~150 tps, though at peak 3.5 has reached 156tps.
In other good news "the model has been trained to minimize refusals for beneficial uses.".
Otherwise, this news feels like a tiny incremental improvement on Gemini Flash series to make it more efficient with token usage, subagent and cost. Nothing big.
Regarding their benchmark scores on CyberGym, I wonder why they didn't compare their 3.5 Flash Cyber model with Fable 5. I mean they included Mythos and GPT-Cyber, so why not Fable 5 too?
They also mentioned Gemini 3.5 Pro is in testing and its about to become available very soon. Another thing maybe worth discussing is the announcement of pre-training Gemini 4. Sadly, not much technical details to discuss on. Many comments in here seem to mostly be about how Google is behind the others, but honestly, is it really worth the investment to be #1 in Artifical Analysis every week?
I don't know if it's a rendering error since it looks like your site renders the SVG instead of hosting a static image of it, but the 3.6 macbook looks like an abstract art piece lol, both ff and chrome desktop
3.5 Flash-Lite seems available in US region, as was 3.5 Flash; but 3.6 Flash looks Global only so far when pinging. If Google employees are watching, will this issue go away?
Bottom line it looks about on equal footing with GLM 5.2 in terms of both overall intelligence and cost per task, while being significantly faster (in fact it is the fastest model on artificial analysis as of rn [0])
It is 17% more token-efficient than 3.5 and performs significantly better in coding and tool usage benchmarks.
It is also cheaper than 3.5:
> This enhanced efficiency is also combined with a lower price than 3.5 Flash. At $1.50/1M input tokens and $7.50/1M output tokens, 3.6 Flash reduces the overall cost per agentic task, making agents more cost-effective to build and run.
I have just tried to switch to 3.6 instead of 3.5 in antigravity and it seems to constantly spit "critical instruction: STOP CALLING TOOLS NOW. YOU MUST WAIT FOR WAKEUP. ". I think I will switch back to 3.5
Kind of excited about this. 3.5 Flash on Antigravity has surprised me recently on a hobby project. When given opportunity to plan, it can deliver on tasks that would take me a while on my own and generates responses at blazing speeds - compared to what I'm used to at work with Opus 4.8 (granted I don't use Opus 4.8 on my hobby projects so just anecdotal). While with Gemini CLI I would just watch it run in circles and run out of 5h allowance before anything useful is produced (or even approached).
Spent half an hour just now benchmarking it against my current 3.5 Flash pipeline excited only see it regressed slightly (0.1% - 0.2% at most, for feature extraction work)
Seems like this is mostly a cost play by Google, hoping this doesn't bring 3.5 Flash capabilities to an end of life, and that 3.6 catches up or gets better.
Proof-of-life release while they figure out how to have a competitive frontier model release. My hunch is they pushed too far in the "omni" model direction, that they made something so ungainly, it wasn't as good for normal tasks.
I often use Gemini free web chat because it's generally quite good at web search-related questions (apparently it has direct token-level access to the Google Search index) but I noticed in the last two weeks output quality of 3.5 Flash seriously degraded. Maybe they were switching over systems.
Google's Knowledge Graph is a massive advantage no other competitor has. I don't think they've fully utilized its full potential but I don't know if any other company could've built something like Scholar Labs
Counter-point, cost per task is almost the same ($0.47 vs $0.50) between GLM 5.2 and Gemini 3.6 Flash. [0] Not to mention the subscription plans likely produce 10x value compared to GLM 5.2 API, unsure the rate limits on a equal subscription vs subscription, but Google subscriptions offer tons of other value, including 1 year of Free Gemini for Education accounts.
Always happy to see new Gemini releases as IMO Antigravity Pro 16.67/mo plan (Annual) is still the best plan available and have been pretty happy with Antigravity IDE.
If it wasn't for Gemini/Antigravity I'd have to go with a Max Claude plan, as it stands now I can get by with just a Claude Pro plan to get Opus when I need it, whilst using Antigravity as my day-to-day workhorse.
Unfortunately Gemini Flash became too expensive to use as a general purpose model (i.e. for AI features in Apps), luckily there are plenty of cheaper Chinese models to fill that gap now.
Feels like they released this to ride the wave of press of GPT-5.6, Kimi K3, and Qwen 3.8. Doesn't feel like Google has much substance with this post except a bump in version and tweaked their pricing.
Don't know why are they even pursuing Gemini. Just download the Kimi, call it Kimini and serve it on your GPU. Maybe then train next architecture based on this!
Gemini 2.5 Flash-Lite has been my go to for cheap document processing at scale (especially with 50% off batch mode), but they are really boiling the frog with pricing increases with each version:
gemini-2.5-flash-lite: $0.10 input / $0.40 output
gemini-3.1-flash-lite: $0.25 input / $1.50 output
gemini-3.5-flash-lite: $0.30 input / $2.50 output (a 6.25x increase over 2.5!)
Now watch them deprecate Gemini 2.5 Flash-Lite in the coming months...
3.6 Flash scores exactly the same as 3.5 Flash on the Artificial Analysis index. Better on some tasks, worse on others. Mostly within what I'd consider the noise window. Looks pretty much indistinguishable from 3.5 Flash, at least on these benchmarks: https://artificialanalysis.ai/models/gemini-3-6-flash
I asked Gemma 4 E2B, and if you use it as a reasoning model, it will give a better answer (that it doesn't know; it was also using the date information from the prompt.)
Because they don't have a lot of parameters to store general Wikipedia knowledge. They're small. Use big models that have high parameter capacity to store general information. Or build a harness around the small model that searches a knowledge base/internet.
Use the right tool for the job. It's like asking why a screwdriver isn't good at sawing wood, or calling C a terrible language because it's hard to make CRUD apps with it.
IMO Gemini has the best free tier models/app for everyday use. Muse-Spark is perhaps just slightly better, but has none of the connectivity to my GApps (for things like “create a recipe in my Google Docs from this image”).
Plus they are probably running these things on every Google search so saving tokens is a huge win for them.
Free plan, the default tier without requiring a subscription. If you use through the Gemini App or gemini.google without paying anything, the model used is 3.6 Flash.
For comparison of Free Tiers:
- Gemini serves 3.6-flash (rank 12)
- ChatGPT serves 5.5-Instant (rank 23)
- Claude serves Sonnet 5 (rank 27)
- Meta AI serves muse-spark-1.1 (rank 5)
While Meta AI serves the better ranked model, it doesn't end up working that well for other things. For example, if I ask "help me buy a new raincoat", it ends up suggesting a Cambodian website, whereas Google is well integrated with Google shopping. It doesn't have the same integration with GApps outside of Gmail/Calendar. A few other email connectors are available.
Claude has one of the best interfaces with connectors, skills, and plugins galore, but the model and limits are restrictive on the free tier.
Gemini, as far as I know, I've never hit a rate limit on Flash.
I believe Gemini is going to gain market share through the free tier funnel while serving models as cost-effectively as possible. People are going to use Gemini because they use GApps and Google.
ChatGPT and Anthropic are going to be competing for the API/Business users, but for everyone else they are going have to become Google before Google becomes them.
For anyone wanting a faster overview: I ran the Gemini 3.6 Flash and 3.5 series release notes through NotebookLM and generated a short video summary. Link: https://www.youtube.com/watch?v=SUFBhvQ2tY4
LLM reception is truly extreme, even worse than AAA game releases.
Ever frontier lab lived it at least once : missing the frontier by a few months triggers extremly negative reactions, then you take back the lead for 2 weeks, and the hype cycle repeats.
I spin a mental roulette on whether the reception on a new release will be "OMG best model by far, no one will be able to catch up for months!" or "OMG this is already outdated, RIP company X, they might as well just give up now, there's no coming back from this."
It's quite a fun game. I click into the comments and see if the roulette wheel was right.
So 3.6 Flash is a somewhat of an admission that Google miscalculated by charging 3-5x for 3.5 Flash what it did for 3.0 Flash (3x input and output costs plus large token inefficiency changes) despite only modest improvements?
3.5 Flash Lite is only a hair cheaper than 3.0 Flash, but I think 3.0 Flash is a massively more capable model?
Especially when Google owns 15% of anthropic and serves them compute. Double especially when your boss (Hassibis) is also an early investor in Anthropic. Hell his NW might be more Anthropic than Google.
Yep, agreed. They still are not releasing anything frontier-class (Gemini Pro) at this point. Feels to me that they keep getting scooped by others (e.g. Kimi 3) and then are retrenching.
I'm a big fan of the Flash-Lite models. They're exceedingly fast and deliver great outputs for high volume use cases where you need to process requests at scale. Can't wait to try the newer version.
It does seem like their releases are getting closer together. I get the feeling they realized they were trying to roll out to their entire ecosystem and now they’re focusing more just directly on the AI model itself. I think give it a little time and they’ll start to be one of the competitors too.
That's wildly ambitious pricing by Google. You can maybe get away with spicy pricing at the SOTA edge but at the lower tiers everything is a lot more price sensitive.
You need to compare cost per task buddy boy. Cost per token doesn't tell you much when you don't know how many tokens a model will use to accomplish a task
They are working on coming up with a better code name. You know, something like ”Fable” or ”Sol”, gotta have one these days. Personally I think they should go with “Mafia”. How cool would that sound? 3.6 Mafia.
tl;dr: 3.6 flash is a bit smarter than 3.5 flash, but also a bit more expensive.
My results [0] put Gemini 3.6 Flash at the top.
3.6 Flash high has same $1.5 input price as 3.5 Flash, but output is cheaper from $9.0 to $7.5.
Google said 3.6 Flash is more token efficient, but in my tests it's actually LESS token efficient[1] than 3.5 Flash, so despite the output price reduction, it still costs more.
Models are expensive and low performance. On top of that they make you jump through hoops to even use these models without being throttled even for the weaker models. The only reason we are using them is credits. As soon as credits run out we are switching immediately.
Why do you care that much? Just say it. You're letting an imaginary score determine if you should express your suggestion for improvement. That's a bit wild.
the real product is the naming confusion we made along the way. Gemini 3.6 Flash, 3.5 Flash-Lite, 3.5 Flash Cyber — at this point even the model cards need a model to explain them
quite a good model, the speed/price/quality ration is a new golden intersection for me, not sure if its as good as grok 4.5 but quite fast/capable model.
It feels like AI is going to be the end of Google. The post-Schmidt company culture cannot produce consistent, consumer-friendly products that any sane person would want to use consistently.
I have no skin in this game and this comment will be gray in a few minutes BUT a friendly reminder that these types of threads are astroturfed heavily by competitor labs and any info should be taken with a massive grain of salt.
Well, you can use the google models from Pi. Go to aistudio.google.com and set up an API key. There is a free quota, it's relatively large for the Flash Lite models.
...and last time I looked the limits were more generous for Gemma 4 there, but they have been tightened a bit. That's how it goes, always changing.
I remember back when Gemini looked like it was the best model that this comment section was full of confident predictions that Google had "won" and that no one would ever catch up with them again. The most embarassing part is that I kinda believed them.
I think it’s safe to say Google seems a bit out of the top AI competition now. The “cyber” stuff also starts to become laughable with open models providing the full power without the crap Anthropic, Google, OpenAI are trying to frontload on you(I.e you are not allowed to develop/review a login system, pay a special cyber operation team to do it for you). They really deserve to become irrelevant in the future of AI.
Google watches over the last few months a flat out assault on the Pareto curve from American and Chinese companies. Release after release pushing the boundaries of frontier intelligence and price/performance.
And the response from arguably the biggest AI research labs in the world by headcount is Flash 3.6.
What do you do when you are given essentially unlimited resources and still find yourself falling behind?
Plugged 3.5 Flash Lite into an existing agent harness that was previously using 3.1 Flash Lite and this shit just does not work. It's not following instructions and is not producing the correct tool calls.
> For autonomous subagents with tool calls, code execution, or multi-step reasoning: set thinking_level to "medium" or "high" to prevent premature tool termination.
Gemini 3.5 flash is already a pretty good model. But, unfortunately, the primary way you can interact with it for coding is through Antigravity - which is actively developer hostile.
It doesn't matter how good the model is if you're (mostly) forced to use it in Antigravity - which turns any model into crap.
I use 3.5 flash 10x more than any other model, despite have access to all of them. If I'm going to play a slot machine, I'd rather get the pain over with quickly.
We're almost five years into the whole GenAI thing and we're still relying on these guys to spoonfeed us incremental updates.
It's time for them to start focusing on open-weight models and efficiency. Otherwise there's just a layer of marketing hype and "will it do this?" that has to be cut through for evaluation of each and every release cycle.
Models are getting easier and easier to create. The money, if there's any here, is in the harness the user interfaces with, and the data centers running them.
Google seems to be falling way behind the pack. antigravity cli is pure trash. gpt 3.5 pro is now behind and isn't released yet. GPT 6 and Fable 6 releasing next month. What the hell is going on over there ?
Google was late to coding agents and as-per-usual fucked it up with their crazy project management culture.
Usually Google gets away with it due to inertia, however this time they are paying a heavy price because they missed out on the training data that Anthropic and OpenAI have gathered with claude and codex.
> we have taken an intentional approach to deploying 3.5 Flash Cyber. The model will be exclusively available to governments and trusted partners
Screw your government! US and Israeli governments should get the least access, but of course we all know they'll be the (only) ones to get full unfiltered access.
I keep saying this and people dont believe me, but I have b2b saas systems with actual agents running around the clock, and the performance/stability of the flash model is higher than most other models.
Meaning, its predictable with tool calls, wont spin off a million tools/do weird behavior, its reasonable. Even sonnet in a real world decision making scenario is not reliable, or will reason so long its incredibly expensive.
The benchmarks arent catching all the value, and most people have never actually ran an ai agent in a real context that matters
Who's most people? What are you talking about? Most people here use agents every day and I wouldn't trust flash or pro to touch any important project of mine because they're both terrible compared to the competition, waste of time every time I give them a chance
With all the naysayers on Gemini models I'm curious how many people actually use Gemini regularly?
For me, Gemini models are the most usable. Claude Opus and Mistral always try to turn queries into one-shot enormous commits, which just burns tokens, time and annoys me for something which is still wrong more often than not.
Gemini seems far better at listening to instructions and giving me what I actually want, on top of using far fewer tokens and wasting my time. Fable is the only model that's come close to Gemini Pro for me.
And as this is about Flash, it's exciting, I find Flash can usually get the right answer pretty quickly and without too much nonsense.
I use all of the major providers daily and I tend to go to Gemini for "fast lookups" where a good enough answer is probably OK. I use ChatGPT and Claude for anything where it matters and generally when I invoke all three -- Gemini is the most surface level with responses, and also sycophantic.
It gets worse from there. Gemini is terrible at agentic coding, primarily because Agy is terrible. I noticed Google updated Agy with this release, so perhaps that's finally going in a good direction. I'll have to test it. Thus far, my experience in countless experiments has been Gemini models being 2x faster yet with less depth in their solutions and a lot more going off track.
I very rarely have to stop Codex or Claude Code sessions because they're doing something random and unexpected (or not asked for). I genuinely think Gemini models are brilliant but virtually useless in agentic scenarios in my experience of the last few years (2.5, 3, 3.1, 3.5).
I should also note that Gemini web UI annoying resets to its lowest intelligence which feels scummy and Google is not transparent about what "extended thinking" really is. Past posts have pointed to "extended" being medium. Every other provider gives you the raw value (medium/high/etc).
So honestly, I feel Google would rather I don't use their models. They just want to get a little bit of mindshare and stay in the conversation. I had the Ultra plan and cancelled it once it was apparent they were not improving the agentic experience nor trying to compete.
not a google fanboy by any stretch... though i've been thrilled with the flash line of models... i exclusively use it on high, and have found it to be a great fit for increasing productivity 10-fold while maintaining quality... sure it can't just go off and one-shot a bunch of work, but at the complexity level i tend to work at, neither can the frontier in a robust way that i can be confident in... sure i have to be in the loop more, but that helps keep me grounded and course-correct earlier before wasting tokens... and when you sufficiently spec out a coding/software problem, and i mean really document all of the critical nuance, it will successfully satisfy the constraints... the quality is rarely acceptable on first-pass, but it forces me to stay connected to the architecture more than i would be if using a frontier model... i've found this to be a happy middle-ground of productivity and awareness...
tested the models on aistudio. despite that the knowledge cut off is march 2026 it still knows nothing about 2025!
you can check by asking "list notable world events in 2025, only list unplanned" on aistudio. or you can ask for Charlie Kirk, it also does not know. I tried it multiple time to ensure that I didn't not get routed to older models!
> but google has search
irrelevant, without deeper knowledge about cutting edge technologies or latest libraries, all of it suggestions are crap. even you ask it to search it will still use outdated keyword thus only getting outdated information.
Google desperately needs to make some leadership changes within their Gemini team now that they've been surpassed by 3-5 open weight models and risk loosing frontier status all together in the near future.
Open weight models aren't likely to be open weight in the long-term. China has started considering export controlling and limiting access to model weights [0].
They don't need to be open weight in the long term; once there's an open-weight Fable-level model with 1M context it'll be pretty much good enough for all coding tasks, no need for new models.
At this point, I think google should consider becoming a hyper scaler for anthropic and open ai, and I predict that that is exactly what they do. The model is no longer the most valuable part of the stack.
I wonder how big the Pro model is that Google is using behind the scenes to train these smaller ones.
Going on baseless speculation, the lack of accompanying pro models with these flash releases either means: 1) the model is too big to be economical, 2) google doesn't have the compute to serve the big model, 3) their big model has too many alignment issues to serve to the public.
edit: looks like benchmarks are up on https://artificialanalysis.ai/models/gemini-3-6-flash. It's solidly middle-of-pack. However, if you want to be most fair to flash, look at the intelligence vs time per task and intelligence vs outputspeed benchmarks. This is a very fast model.
edit 2: I use antigravity from time to time and in my experience, 3.5 flash is an underrated model, so long as you know what it's good for. It's very good at frontend (much better than gpt 5.5) and it's fast, so it's a great tool for iteration. I expect 3.6 to be no different.
It's also very possible that they know their big model underperforms chatgpt 5.6 and fable by too much, so they are focusing on what they can get wins in like speed instead.
That's the only explanation that makes sense. If it was frontier but cost or compute were limiting factors, they'd release it at an obscene price for the bragging rights. Google doesn't care that much about alignment, and I don't think it's likely to be significantly different than 3.5 anyway. The only reason it would need to be soft-canceled is if it's terrible, and has to end up in a ditch like Llama 4 to avoid shareholder panic.
3.5 pro was clearly a miss. It should have been in prod mid may, not MIA in late July. The brain drain at deep mind is a clear indicator that the people who know the most think that they can’t stay at the frontier.
Antigravity NEEDED to be game-changing. Without the stream of data that Claude, Codex, and Cursor enjoy there is little chance of getting an effective reinforcement learning loop. For the first time in its history, GOOG is at a meaningful data disadvantage, and apparently a cultural one as well.
“Google doesn't care that much about alignment”
I don’t think this is necessarily true, did we all forget how much Google cared about alignment that their AI wasn’t able to render a white polar bear?
That and/or the business case isn’t as clear when serving enormous models? You’re constantly stuck in a red queen’s race where your profitability window is increasingly measured in weeks because the Chinese are right behind you.
For small models (which are probably distilled from their big ones) you can serve them economically all the time and not hemorrhage money.
> For small models (which are probably distilled from their big ones) you can serve them economically all the time and not hemorrhage money.
For smaller models, you're competing with DeepSeek V4 Flash. (Which I think is a 284B A13B?) Subjectively, this feels about as smart as Sonnet 4.5, give or take. And it costs $0.09/$0.18 on Open Router, compared to $1/$5 for the latest Claude Haiku. See https://openrouter.ai/deepseek/deepseek-v4-flash#providers The developer antirez of Redis fame uses this as a local coding model.
DeepSeek did some extremely clever research on hybrid attention to get the prices that low, reducing per-user context cache sizes dramatically.
So, no, when it comes to low-price models, the US models probably can't sustain their current margins there, either.
Yes agreed - I wrote this up a while back https://martinalderson.com/posts/whats-going-on-with-gemini/
My view then was they are optimising the models for inference ability on their own hardware AND use cases, which is often speed and time to first token.
They've somehow seemed to end up with terrible compute shortages, which again is surprising given how good Google is at infra deployments AND have their own hardware. From rumors out there they are turning down enterprise deals for Gemini because they don't have the compute.
The problem is they're falling further and further behind on frontier class on coding especially, and since I wrote that article it's got even worse with open weights models undercutting them on price AND intelligence.
They include an LLM response with every single Google search, whether it is warranted or not. That scale is, my guess, many orders of magnitude higher than what OpenAI and Anthropic serve. And for Google none of these are paid interactions since their LLMs do not (YET) insert ads into the responses.
So my guess is that Google will continue having compute shortages until the Gemini enshittification starts.
I don't think so. According to some very basic research there are around 8bn searches a day, or 250bn a month.
Let's assume Google serves AI overviews on every SERP (they don't) and don't cache them (they do, afiak).
And let's assume that each AI overview is 2000 tokens (blended input/output), that's 500T tokens a month.
It's rumoured that anthropic is serving somewhere close to 10Q tokens a month.
Now it may be that AI overviews uses vastly more tokens than that per search, but I doubt it based on speed to render the overview.
My very rough napkin math on this is that maybe AI overviews is consuming 100T tokens/month max (after adjusting for caching and SERPs that don't have them), which would be 1% of Anthropic token volume.
There was some recent reporting that a July release of the Pro model got pushed back for exactly that reason. Its performance was not good compared to the OpenAI/Anthropic big models. They are having a lot of problems with posttrain.
This is the feeling i get too. Cant produce quality, but can produce something that is super fast...so take the wins where they are.
We don't have enough fast models, so I see this as a positive. I just test drove Gemini Flash Lite and it's crazy fast.
For a coding LLM specifically, when is fast a good tradeoff for quality?
> ...when is fast a good tradeoff for quality?
When it is cheaper, and the "lower quality" model is adequate for the task at hand.
Plenty of problems have a low(er) skill/intelligence floor, anyone who uses the dual-mode agent paradigm (plan, then act) figures out the second phase can be completed by a less capable model. Even when disregarding costs - speed is important here because the agent can rapidly iterate without human supervision, based on compiler errors, lint and test failures
A coding agent driven by a large LLM can delegate smaller tasks to a faster model. For example searching through the codebase for references, examples, or established patterns. They are treated as tools and don't pollute the main agent's context.
I wouldnt say it is, but there are circumstances when speed is helpful. I wouldn't argue that coding is one of them.
I personally doubt that.
It would be a shame if they cannot beat Kimi K3 or Qwen3.8 Max, both of which are claimed to be Fable-like. If that is true, it will be [or would be] the first time a major American lab falls behind a Chinese competitor.
Google can't compete with China, neither can Meta. Only two labs in the US can keep chucking billions at the frontier race. Everyone else has a real business to run.
China can keep up because it's cheaper to run a frontier lab there. They also have more researchers and a stronger cultural inclination for this sort of thing. And I guess the business case in China doesn't have to work as well as it does in the US.
> China can keep up because it's cheaper to run a frontier lab there
Not sure if this is what you meant, but their training runs are significantly cheaper. This was one of the big shockers from the Deepseek R1 paper. US foreign policy has helped to ensure that the Chinese are compute constrained, so they literally cannot buy the most expensive and powerful training rigs.
This has led to a steady drumbeat of innovations which are not revolutionary on their own but stack together to make things much more efficient.
> focusing on what they can get wins in like speed instead
Speed as a differentiator has always been Google's thing. They (used to?) show the microseconds it took to query & rank web-scale search results. Chrome, notoriously, focused on speed at the expense of resource use. The very many efforts to efficiently speed up Android & its runtime since its inception, and so on...
> their big model underperforms chatgpt 5.6
Possible but TFA claims:
That sonds like they can't compete with 3.5 or 3.6 so they must increase the model size and are training v4.
Didn't they already acknowledge this?
Paywalled article, but the headline is basically all you need: https://www.bloomberg.com/news/articles/2026-07-16/google-ge...
I choose fourth option.
4) googles big model just performs worse than K3 and GLM so they choose not to embarass themself.
Like I love Gemini and use it a lot to one-shot whole MR with huge contexts, but its just much worse when its come to tool use and agentic coding.
I wonder if the broad use of AI overviews on Google search results is having an impact. Maybe the numbers make it more profitable to use their compute on several billion searches a day rather than selling API access.
I think it's a safe bet that Google seems more interested in making a model that improves Google rather than making a model that improves workers.
Fast, light weight, ok intelligence. Perfect for serving 20B+ prompts per day mostly surrounding banal human things.
OAI and Anthropic's cloud spend can cover the revenue gap, as Google is already capturing a large chunk of those guy's revenue.
Not to mention internal use cases, such as prediction-related tasks like serving ads.
The AI mode on Google search is pretty impressive. Helped me figure out what a bunch of stuff I was seeing out the window was while traveling.
Microsoft also seems to be working in this space. They recently released this:
https://huggingface.co/microsoft/bitnet-embedding-0.6b
It’s a small multilingual embedding model designed for things like search, RAG, and semantic similarity. It supports a fairly large context window and is designed to run efficiently on a CPU in a GPU starved world.
The interesting part is that it builds on BitNet, using ternary weights of -1, 0, and 1 instead of the usual floating-point weights. That should make indexing and searching large amounts of text much cheaper without giving up too much accuracy.
AI overview is just a summarization of the top 2-3 results. Of course at Google scale that will still need a ton of compute, but the requirement for generating an overview is many orders of magnitude lower than asking the same question in Gemini.
based off what?
vibes (coding)
Friend works for Google vendor who generates data for training. His team alone is 200 people (in US).
He says there are many similar vendors and teams with thousands of people in India and other countries.
It seems like there are some credible rumors that Google is actually winning in terms of actually building models that work and don't lose money- between how they're able to price them, the TPU advantage and their capex advantage (being able to raise debt + just having a lot of cash - well I said not lose money... more like not go bankrupt).
From the outside they look like they're behind in terms of frontier models, but I think they might be the best positioned to not go out of business when the bubble pops.
Also look at the fact that they've been able to deploy AI-assisted search at google scale. It must be another order of magnitude larger (at least) than the model deployments for OpenAI and Anthropic.
Of course unless you're inside Google it's impossible to know for sure.
In terms of open models, Gemma 4 beats the pants off everything else to the point that paying for APIs becomes hard to justify. Qwen has the meme-share for coding, but it feels much less well rounded. I have no doubt that Google have both the infrastructure and the expertise to curb stomp everyone else, should they resolve in earnest to do so.
Lest we forget, "Attention is All You Need" came from Google.
> "Attention is All You Need" came from Google
It also came directly from the university of Toronto, and the university of Toronto seeded all American frontier labs (including Grok (why do you think they could start so fast))
Interesting, glad to hear. We have gemma4 at work, and I was considering localhosting qwen, but gemma4 is so far behind the Opus and Fable I have at home that I've decided to hold off for another model release.
"We have a company provided Toyota at work but it is so far behind the Ferrari I rent at home that I've decided to hold off for another model release."
Are you suggesting Gemma beats GLM 5.2?
At 20x the parameter count I should hope GLM beats Gemma! But is it 20x better? Expertise is demonstrated, not by making big models, but by making small ones. Bigger isn't better if you can't run it at all.
How long until Gemma 5 hits?
It's rumored that Gemini 3.5 flash has a >50% margin, and I'd imagine 3.6 flash is even higher.
I do not think OpenAI or Anthropic are actively chasing margins - though, Anthropic is supposed to be profitable on some form of non-GAAP accounting...
I suspect Google isn't really interested in seeing how far it can get dragged into a race of selling dollars for $0.25, and is more interested to see if it can stay in the race selling $0.50 for a dollar - when everyone else is losing or barely breaking even.
google has to make money. flash is awesome. you can run it free on their infra and the performance and latency is excellent for what you wait and pay for right now, with great perf per watt. every person in the world going to google.com runs it. every query. its far larger than free gpt, localhost qween and what not.
it's their pro that isn't awesome at all. in fact, their pro kinda suck now that everyone else woke up.
It kind of doesn't make sense though, because typically a large org like Google can afford to crush competitors on pricing. They could probably even go toe to toe with chinese model pricing for years without feeling it.
Maybe they don't want to price war with the other labs so they can comfortably maintain healthy margins on selling them compute?
Maybe they are hoping that when the bottom drops out they will just be able to buy Anthropic or OpenAI for a few tens of billion.
That "TPU advantage" might be slowing Google down (though likely not as much as their internal bureaucracy).
Porting CUDA-based research, debugging, and overall experimentation speed is likely slower.
The GPU is still king for training.
But maybe the TPU advantage is in inference? That's what I assume because the number of compute cycles are going to be all in inference vs training. So they could train on GPUs if they want.
lmao, you know all Anthropic models are trained on TPU right?
thats funny because my company sells them nvidia gpu for training. but im happy for the billions, they prolly use them for counterstrike!
They basically don't exist in the currently most profitable LLM market (coding).
Yes, subs like codex are heavily subsidized. But API billing has massive margins and that's what enterprises pay.
Does it have "massive" margins? Afaik no one has said publicly what margins there are on an API call?
"As of October [2025], OpenAI's compute margins reached 70%, up from 52% at the end of 2024 and double the rate in January 2024, [The Information] said, citing a person familiar with the figures."
https://www.bloomberg.com/news/articles/2025-12-21/openai-se...
As for Anthropic, the rumors I remember seeing for their API margins were more like 85-90%, but I don't have a reference at hand for those. But once you know the API is wildly profitable and the subscriptions are roughly break-even and not even a big slice of their income, all of the investment makes a lot more sense.
It says the original report was in the Information, which I can't see, but I'm skeptical that they includes the training cost? And how much that changes the figure?
Logan Kilpatrick said on an interview not too long ago that flash 3 and 3.5 are the same pre-train. all gains on top of 3 flash are post-training
Maybe, but they said they have “started” the Gemini 4 pretrain. So not having done any significant pretrain in a year or so seems odd to me.
Pre-trains take a huge chunk of your compute offline, incurring both an raw expense (24/7 max power for all training clusters) and an opportunity cost (could have sold excess compute during that time). They also don't come with any great guarantees, as lots of techniques look good on small scale and crumble or plateau once scaled.
More likely they don't manage to advance benchmarks on the SOTA level anymore. In other words: They can't beat 5.6 nor Fable
I think it's 2. I frequently get told there's no capacity for Pro and the query is answered by Flash with extended thinking. And tbh it's hard to tell the difference between the two, especially if you're not coding with it.
It's hard to tell the difference because they nerfed Pro to oblivion, it used to be much, much better model (even for non-coding/chat)
Maybe it's like Meta not releasing the big version of Llama 4 a year or two ago
2.5 flash was absurdly capable on a cost basis
I wonder if they waited for the new TPU generation to train a larger base model.
"3.5 pro is testing with partners! will hopefully land soon."
https://x.com/OfficialLoganK/status/2079596415509303596
> the lack of accompanying pro models with these flash releases either means:
Rumors say 4) it didn't perform well, especially in coding so has been delayed
Or perhaps 4) it's outcompeted severely by other models & releasing it would only tarnish their name
My hunch is Google is trying to integrate a fast and relatively cheap AI across search and every other surface of their product suite. And for that objective, a model that can move faster while being accurate and cheap enough is more important to them than producing a frontier class heavyweight model.
This. Give me cheap tokens that produce accurate information and the deal is done
Google somehow managed to snatch defeat from the jaws of success with their AI products.
They literally forced me and my company out of Antigravity by phasing out AI Ultra subscription without any proper product follow-up. Antigravity IDE cannot even have poweruser subscriptions now from Google Workspace an Gemini Enterprise Agent Platform cannot be attached to Antigravity IDE.
Gemini Enterprise Agent Platform has an incredibly abysmal setup process, and if I want to limit spending per-user I have to create projects per user. The fact that you cannot activate Anthropic models on it if the billing still has free credits is almost a joke.
I was a big proponent of Google and Gemini, but they left us reeling with their abrupt product decisions. Forced us to buy $200 subscriptions directly from Anthropic/OpenAI.
Also a big proponent of Google and Gemini, but their stubbornness in artificially splitting their consumer and enterprise products is extremely annoying. It's pretty weird that I have access to more powerful tools when using my personal Google account compared to my corporate Google Workspace account.
It's literally the meme of the MS org chart pointing guns at each other.
The GCP team wants their slice, the other team wants some otjer slice, and so on. Everyone wants some crap for their promotion package.
It's no wonder Meta has shit the bed even worse.
It's also why Google still releases actually decent, useful models despite the product being such a hilarious mess. A lot of the time Gemini models have actually been better as production LLMs as part of LLM-based production applications than OpenAI and Anthropic models when it comes to the complete cost:quality:latency:adherence picture. And they still are. We have products in prod that use Gemini because they're better than any other model at the specific task. But we wouldn't dare use it for anything coding related, or even just as productivity tool to rely on, because as a consumer product it's a joke.
> The GCP team wants their slice, the other team wants some otjer slice, and so on. Everyone wants some crap for their promotion package.
I got a Google One plan for Gemini, but it came bundled with YT Premium lite, and that somehow made it impossible to renew YT Premium for 30 days. I suspect different teams stealing customers from each other.
Google also gave away 1 year Gemini plans with Pixel phones that either did not work at all for existing Google One users or messed up subscriptions by downgrading your account to worse plan, or making your existing paid time shorter if you been on cheaper plan or recently changee countries. Etc.
Like when you try to give Google money they try to squeeze you as much as possible.
At the same time you can get 5 time more limits for free just by registering 10 free Google accounts.
Google subscriptions are one big mess.
Exactly my experience. I'm building an AI document-extraction platform, so I had to benchmark a bunch of models — on the cost:quality:latency:adherence picture, flash wins hands down for structured extraction. (Caveat: I've only tested the three US labs and Mistral.). So like u said, totally viable in prod for a relatively static tool. Didn't build the tool suite with gemini, but if you use service mode it currently mainly runs on flash.
Haven't done any serious coding work with the flash models though — but I'm seeing more and more HN comments from people who seem to have picked it up for that in the last couple of months.
As someone who's been using Workspace as a personal email account for over a decade this has been such a struggle forever. Just lots of odd limitations to feature sets all over the place.
When they swapped Google Assistant for Gemini as the default voice provider in Android Auto it was so annoying. My wife's non-work space account can get Gemini to do the normal things like play music and what not, but my Workspace one can't do much of anything at all. I can talk about nearly any random topic with it, but getting it to change the playlist, nah, can't help you there.
It's no surprise to me to see them fumble actually supporting a lot of the consumer features of Gemini into Workspace.
I'm in the same situation. But I was shocked discovering it goes both ways: many new Gemini functionalities are only accessible using a consumer account instead of a Workspace account. Also, Gemini is now the only major AI assistant with no support for MCP connectors. Instead of adding this to the core product, like ChatGPT and Claude did, somebody at Google decided that it was smarter to add this fundamental feature to a new product instead: for enterprises this is Gemini Enterprise (which is a product completely different from the Gemini app); for consumers this the new Gemini Spark agent (meaning that you can use MCP within Spark but not within a "non-agentic" chat)... It's clear to me this a symptom of Google shipping their org chart, which is a disaster from a product perspective.
I currently have a free trial AI Pro subscription that will run out next month.
If it weren't for the $10 GCP credit, I'd straight away cancel it. I don't see enough value in Gemini to justify the $20 subscription.
Microsoft’s Copilot products are a very similar situation where the enterprise and consumer (and GitHub) features only make sense if you think about the org chart. Both companies need stronger top down product thinking.
Maybe because they want to train on your data? Workspace AIs are not trained on your corporate data.
i think Google would see more success if they kept the CEO and everyone at the bottom (ie. doesn't manage anyone), and fired everyone else.
Build a whole new management tree - the current people all do a terrible job.
This drove me bonkers. You can enable play music (etc) in Android Auto for workspace accounts by enabling apps in Gemini. From memory (looking at the settings now, not 100% sure of the magic steps required), but go to admin.google.com, go to 'generative ai', 'gemini app', and 'apps settings', then turn on 'other Google apps'. This lets you play music (and other things) in Android Auto.
Ah, that could be it. I saw that "Other Google apps" and didn't think that would mean Spotify, but I guess its Android Auto or Google Assistant stuff. I'll give that a try, thanks for the tip.
The entire Google Workspace division is basically Microsoft. I assume they are making a lot of money in order to support their continued disfunction.
Also have a workspace as a personal email and ended up getting a personal gmail just to try out the subscriptions before I gave up.
I have multiple anthropic and OpenAI max plans. For Gemini I just use my Cursor $200 a month plan (which also gives me the ability to try grok, conductor, etc)
Workspace users have different terms of service. This is why new features always launch to consumer first.
Apparently you cannot turn off using your data as training data with gemini. This is in line with Google's general privacy policies and it's seeming need to create a stasi file on every human.
You can turn it off but it's an all or nothing switch, if you turn it off everything will basically become a temporary chat and all past chats are deleted
It's absolute insanity. They have all the resources to have been able to lead from the front with this new technology. They have the products and users to integrate this technology into people's already existing lives. But they keep fumbling.
They're not even benchmarking against other models now, just against themselves - which tells you everything you need to know.
What does "leading from the front" get them?
There is absolutely no loyalty when it comes to coding. Nothing could be more common than people threating to jump ship whenever another frontier or open source model comes.
Google is clearly able to keep growing their free and consumer and small business use cases. Unlike corporate coding, we actually have evidence that solo and small businesses can actually see productivity gains.
Anthropic and OpenAI need to stay dancing like mad, because it's their source revenue which underpins their investments.
Why does Google need to shove something at the top at the same desperate cadence? Other than "recursive self improvement leads to AGI" it seems perfectly fine if they push out something dramatically better every year and half.
Unfortunately them giving up coding means they have less traces to train on.
They simply don't have the leadership to lead. And it starts from the top.
Google promotes people based on shipping features, not making good software. So this breakage means someone is shipping features.
You should be happy for them.
> Google somehow managed to snatch defeat from the jaws of success
This is still very early days. Who is "on top" has flipped back and forth many times already. The next frontier model release (from whomever) will change things again.
I don't think it's that early tbh, agentic coding has ~90% adoption in the US.
Claude Code has largely won individual developer mindshare and has been on top ever since it came out. The benchmarks change, but almost nobody opts to use anything other than Claude IME when I ask them. Enterprise is more competitive since they care about costs and other things, but developers leaning towards Claude puts a thumb on the scales there.
The product doesn't have much lock in, so it is possible to dislodge Claude, and Anthropic could (and some may argue is likely to) just shoot themselves in the foot again and again and again, but Google has never been particularly good at enterprise sales, and they have never actually been at the frontier of intelligence.
I think Google's incentives have mostly about building models for their products, which makes them focus more on the cheap end, and while they need that, it feels like the Innovator's Dilemma is biting them here.
I own a lot of Google stock from working there in the past and have been quite happy about their trajectory up until the last 6 months, but I am getting pretty antsy about their AI story these days.
> Claude Code has largely won individual developer mindshare and has been on top ever since it came out.
Claude Code's success is not due to the agent but because the model is considered the best for programming and is very heavily subsidized, compared to pay as you go API prices. Consumers and Enterprise are not really locked in and will go where it makes the most sense.
I think they have almost no loyalty by actual developers.
There is no reason to be loyal....There is no moat.
Basically you may choose to drink brand A water bottle, brand B water bottle or tap water. Oh and you might choose the glass water bottle if you use API/Fable.
There’s no reason to be loyal, but I guess it’s a bit like any tool, once you get used to how one works why would you change to another? There is some stickiness with an LLM + harness.
This isn't a perfect survey, but I am a Codex user and most of my employees are pretty stuck in their workflows and were not really interested in trying even when I was saying good things (even pre-Opus 4.8).
When we work trial people, 100% of people ask for Claude rather than Codex or anything else.
When I talk to people at non-AI tech events everyone basically says they use Claude and have not tried an alternative.
Developers writ large are actually not that interested in trying multiple tools, they like customizing their chosen tool and tweaking it forever.
I think developers are as susceptible to brand marketing as everyone else. It's why almost everyone has a Macbook.
Exactly developers can switch to another coding cli and the learning curve is close to zero. Mindshare without switching costs is just a popsicle in the sun.
Every model has its strengths and weaknesses, being loyal is suboptimal unless you mean being loyal to all of them, which is why cursor would have been well positioned before it got acquired. Now you have to jump through hoops to call Gemini from Claude from codex. Yuck.
Claude code was one of the first agentic code tool and when openai release models similar in performance they didn't do as well in their tools (now codex)
Claude Code (or any other model/infra/harness co-design) is not subsidized in any normal use of that word. It's trough filling (I've written this up in lurid detail so I'm only going to do it again if anyone cares).
It's not true, it's just a play for margin.
> I don't think it's that early tbh, agentic coding has ~90% adoption in the US.
Where does that 90% figure come from?
I pulled it out of my ass based on anecdotes of talking to engineers and customers, but actual surveys back this up as well with numbers from 84-91%: https://www.digitalapplied.com/blog/ai-coding-adoption-stati...
Early days or not, Google fucking deleted my IDE and wiped my settings. It took them days to roll out a fix, by which point I had migrated off Antigravity.
I guess? Maybe I'm alone here but I don't feel like Fable is particularly more useful than Sonnet most of the time. I feel like the LLMs are good enough for the majority of uses and the hyper expensive premium ones are way into diminishing returns. At this point with Kimi being as good as it is, if they jack up the price any more I'll just go open source.
I don't think that's true if the reason a company has left the vendor for given model by making it hard to buy. Enterprise IT is enough of a pain in the butt that people will forego the new shiny to avoid the old painful unless it's genuinely better. As you say though, the best frontier model flips regularly, so companies won't go through the hassle of deploying a model if it's proved horrible to do in the past. They'll just skip that model because their current one is fine.
Model quality is only one aspect, the bigger problem is making it work in a fully integrated enterprise platform, and Google has always been lacking when it came to the latter.
At this rate, if Google has a flagship model, you're better off plugging it into a competitor's tooling than hope Google figures out how to use it.
Aren’t the subscriptions extremely subsidized and burning cash for Anthropic and OpenAI? A reasonable explanation is they’re simply abstaining from the war of attrition, especially given cheaper comparable models are breaking the illusion that the “frontier of intelligence” has any kind of per token margin.
I think the subscriptions pay them back in spades because the same dev who maxes out their subscription on their personal account transfers that exact behaviour over to their enterprise work and - guess what - it's all billed per token there. This is a large reason why corporates are reeling from the cost right now, I think.
This is hotly debated and completely unclear. Let's say Anthropics Opus models cost the same to serve as GLM 5.2. GLM 5.2 is 4.4$/MTok while Opus is 5.6 times more expensive. Assume that GLM 5.2 is served at essentially zero margin. Then Anthropic has >80% margin on API pricing. So even if an average person with a subscription pays only 20% of the API price of their usage, Anthropic makes money on subscriptions.
And the real numbers could be better for Anthropic. It's feasible Opus models are actually cheaper to serve than GLM 5.2 because Anthropic have optimized the hell out of inference.
Sure, but then why wouldn’t I use GLM 5.2 at cost or K3?
I guess that’s the big question, will people pay a big margin long term to use their end products / models or will AI tokens be commoditized by many competing players. For coding if I had to pay API costs I’d switch in a heartbeat, enterprise maybe more reluctant?
Makes sense, subsidizing tokens doesn’t seem like a great strategy for a public company.
And Google alway has a target on its back for antitrust (regardless of claim validity)
Possibly, but aren't the tech giants positioned to win a war of attrition? Then again, they're more likely to sit on that cash and wait for the opportune time to buy a frontier lab.
they're subsidized if you max them out, i'd imagine most users are paying $20 for maybe $2-5 of tokens.
anthropic probably has more customers that use more of their sub, but for open ai where a lot of their subs are consumers through chatgpt.com, they have a lot of free money to work with there
The subsidizing thing is repeated over and over without proof. Personally, I doubt they're actually subsidized.
Google Gemini agy is not allowing you to use your token via your own harness. My own harness is far more efficient than agy. They can take a simple stance - if you exceed your token limit they block you - with the 5 hr limit they are already doing this . so there should be no reason to block you from using your own harness - if you are more efficient - you gain - if you are less you lose .
They are not allowing me to hit their endpoints which agy hits - it's frustrating . i tried to hack it with gemini itself. what i love about gemini is it's so encouraging and ready to help you - even against the agy client : ) .
Even though im so frustrated with this - i still love Gemini for some reason ! Most encouraging model in the world!
> I was a big proponent of Google and Gemini, but they left us reeling with their abrupt product decisions.
Likewise. This seems like a common feel. I have at least spent $4000 and likely a lot more on Gemini API because I really wanted them to win. I gave up.
I am going to ask a very direct question and only because I am curious.
Why do you care? Why would you spend your own money to a multi trillion dollar company so that they win their own "war" against another multi trillion dollar company?
Please don't get me wrong, I know the question can seem a bit negative, I am really just curious.
No, it's a fair question. The answer: I believe(d) in Demis Hassabis's vision of AI
Although, I think saying 'wanted them to win' was not accurate. More like, I stuck with them hoping it will get better, and it did get better in many ways, coding was not one of them.
This was exactly my thought process. Coupled with a less idealistic one: mid 2025 if you looked arena ELO scores google seemed to be dominating and I was sure the trend would continue. But they really dropped the ball on coding and tool use.
That being said, controlling android and apple mobile devices is kind of a big deal. And their video models are still top notch.
i've got to wonder how much of this is intentional, and how much of this is just google being their usual terrible selves at anything consumer-product related.
Google's biggest and most important customer for all this AI stuff is google. Do they actually want other customers, or is having other people use their AI just an annoyance at this point, where we use up compute that they'd rather use internally...
I don't know where you've been for the last decade but all google ever does is retire products with no recourse.
I trust Google to:
* Abruptly ban me and all users from a Google Workspace for no reason whatsoever.
* Abruptly shut down a service.
I'll never give Google my direct money.
Life gets simpler and better as soon as you stop giving money to Google.
It’s so silly that individual people still use their shit. Corporations, I understand - they always choose the most mediocre stacks and tools by default. But why people choose to bring the mediocrity of Google into their lives is beyond me.
They'll probably be back later. It's very possible they are "saving up" for a much bigger run.
More likely an acquisition.
+1 same way vscode killed its usage base and gave it to its fork - cursor
> they left us reeling with their abrupt product decisions
This is what happens when you put a McKinsey consultant in the role of CEO of an organization where product managers run the asylum instead of engineers.
Snatching defeat from the jaws of victory is the specialty of product managers.
It's funny that they triggered the infamous "Code Red" moment in OpenAI when the 3-3.1 models came out. I switched to using them for a lot of single-shot LLM calls because they were fast and cheap. Their only area that they were lacking in was agentic/tool calling.
Needless to say 3.5 was a disappointment. Curious to see 3.6.
> Google somehow managed to snatch defeat from the jaws of success with their AI products.
HN lives in a bubble.
I have German/Italian/Polish clients virtually all use Gemini and NotebookLM. Talking insurance, banking, consulting, legal.
The real world doesn't look at pointless benchmarks on writing react tailwind crap, they are already google suite users, get the tools, test them and adopt them, end of story.
It's going to be like with angular, never mentioned on the net, widely used in the real world.
I have friends and family who use Gemini, but entirely because their Pixel phones came with a year of it for free. No other reason, and they will most likely never pay for it.
They literally forced me and my company out of Antigravity by phasing out AI Ultra subscription without any proper product follow-up
That sounds awful.
For those of us who don't follow the AI hype cycle, what does that have to do with the topic of this thread: Gemini 3.6 Flash, 3.5 Flash-Lite, and 3.5 Flash Cyber?
That's kinda like asking why someone might complain about Chrome's memory usage in a thread about a new version of V8.
From the posted link: "3.6 Flash: Our workhorse model that delivers better coding, knowledge work, and multimodal performance."
What is Google's recently released AI coding product?
"Your new models are great, but uh... How exactly am I supposed to use them?"
That is, given a lot of users' contexts, the way they can and the way they want to use these models are increasingly disjoint.
You don't see what Google phasing out an AI product subscription has to do with a Google AI product release?
It's a bit disheartening to see no comparison to other models here - and I'm not sure this pushes the curve anywhere. 3.6 flash is more expensive than GLM 5.2 - but seemingly worse, although this post is really light (lite?) on details.
It seemed for a time that Google had finally gotten the ball rolling, but I'm doubting that more and more as time passes. We'll see what happens with 3.5 pro I suppose.
Here, my comparison of 3.6 Flash vs Sol vs Luna vs Terra: https://aibenchy.com/compare/google-gemini-3-6-flash-medium/...
How does your comparison work? It places Gemini 3.6 Flash Medium above GPT 5.6 Sol High and Fable 5 Medium, which makes me skeptical because that... would be making headlines that I'm not seeing right now.
I have created various questions/tests and put the models through the same tests.
I record whether the answers are correct, and the generation stats (costs, latencies, tokens used, etc.).
I have no idea why the Gemini models do so well.
I have recently added new tests, whose sole purpose was to find some cases on which Gemini 3 Flash fails (I don't like cherry-picking models or tests, but I also find it strange Gemini Flash models leading in accuracy). I made a more complex coding/tool-usage test, that I expected it to fail, it did fail it once locally in my debug tests, but when I finalized the test and ran the entire testing suite for all models, somehow Gemini 3 Flash still got it right...
Gemini models are REALLY intelligent (and they are actually my favorite model to use via the chat app to ask questions), but they somehow fail in real-word coding tasks where they have to modify files, check results, debug, etc.
My tests harness provides a lot of mock data, and limits the number of actions a model can choose from. I am starting to think that maybe the models are not bad, just that the coding harness are not optimized for those type of models, and Google doesn't really provide their own "Codex".
Oh, and I've also added weights to different categories, so Coding and Tool usage categories influence the score more. This done both to better account for how most people are being used, and also to reduce Gemini's dominance in general/domain specific knowledge.
So yes, Gemini models are at the top, even if I actually (not proud of it) tried to make tests that actually favour other coding-focused models.
Interesting, well it'd be interesting to check out some individual examples where Gemini beat the others.
Also, would be great if you could add GPT 5.6 Sol XHigh and Fable 5 High as well, just to see if at least those beat Gemini which is currently your #1.
I don't like to divulge tests, but one of them is a chess puzzle.
> would be great if you could add GPT 5.6 Sol XHigh and Fable 5 High as well
I would like too, but I avoided them for several reasons:
1) Cost - this is a hobby project, those models would cost tens of dollars for each benchmark run, multiply this by tens or hundreds of models and ...
2) Time - the high models are already taking a really long answer to respond (5-10minutes per question). I run each question with 3 repeats (run the same test three times), so it would take 30 minutes per test. If I change my tests, methodology, or add a new test, it would take a really long time to run the benchmark. Also, I like having results immediately when a new model is released, now I can post within 30 minutes of a model's release the benchmark results.
3) High reasoning usually does WORSE on most tests - if you look at the leaderboard, it's sometimes counter-intuitive, but models with high or max reasoning usually do worse than medium and low. This is because the questions are quite targeted/direct, and the models overthink the question and miss the solution. Or the long thinking context makes them perform poorly. The generation tasks (SVGs/HTML animation) are usually better with longer reasoning, but short code fixes, trivia questions, puzzles, etc. are answered by low/med reasoning with more accuracy in general
Also, Fable is borderline un-testable, it refuses to answer many questions, so it scores poorly anyway.
Gemini scores 21/22 because it answers all tests, and it does them correctly, consistently. The only failed test is I think because it miscounted the lines in a file, when responding on which line the bug was in a code snippet.
It’s really surprising. When Apple announced the multi-billion dollar deal with Google to power Apple Intelligence I thought great things were coming. Instead we are getting more and more bad news: delayed Pro models and AI leadership leaving. I wonder if Apple know something the rest of us don’t know or if they are already regretting their decision.
Besides Apple apparently making Siri AI model agnostic, the choice to go with Google was almost certainly for practical reasons. Google is a low-risk established player that already has a long work history with Apple. Google also isn't in an existential battle to establish themselves, Gemini still amounts to just another project at Google. There is tangible non-zero risk that either OAI or Anthropic will be gone in 5 years, or will be forced to leave Apple high and dry to save themselves. There is almost no risk Google will be in either such position. And worst case scenario, Google has incredibly deep pockets should Apple pursue a "refund."
What Apple wants out of Google is Siri that runs at 8gb ram and isn’t a horrible embarrassment that feels like a primitive markov chain. Given how good Gemma 4 is, Google can squeeze some serious performance in small models. Whether they can make bleeding edge models is irrelevant to Apple.
"Siri, please solve the Jacobian conjecture, and also set an alarm for 8am tomorrow"
As someone on the Apple beta.. the model is almost completely irrelevant to the experience. Apple has gone and done Apple things by nerfing the experience so completely that almost any model in the past year would be fine. I still reach for ChatGPT/Claude/Grok constantly instead of the AI toy that Apple calls the new Siri.
Apple isn't counting on their model to be a frontier coding and cowork model. Gemini is perfectly fine for the tasks that new Siri is supposed to be doing.
GLM was twice as verbose running the Artificial Analysis benchmark. So it ends up being more expensive
>verbose
GLM defaults to max effort btw
https://docs.together.ai/docs/glm-5.2-quickstart#reasoning-e...
Not really. Gemini 3.6 Flash actually cost $0.01 more per task, compared to GLM 5.2.
https://artificialanalysis.ai/models/gemini-3-6-flash
All the benchmarks I see put it around the capabilities of Opus 4.8 Medium or Sonnet 5 High.
As far as I can tell it's slightly better than GLM 5.2.
according to AA it's not better than GLM 5.2 and that's surprising to me
I personally take AA with a giant handful of salt.
The one thing I've found google's models to be the best at is proofreading text in non-english languages. Probably because I imagine they have the most training data for it as Google probably has the most complete archive of the internet.
My usage of GLM 5.2 has been defined by slow throughput and flaky providers, in many (definitely not all!) applications a dumber, faster, and more consistent model makes more sense to me.
> really light (lite?) on
Light. Lite is product marketing seepage.
Yeah that’s the joke :p
I've never questioned the word lite before because it's existed my whole life.. So does it make sense? Why does it exist and where does it come from? More than coming from "light".
Sorry, it went right over my head!
> It's a bit disheartening to see no comparison to other models here
Disheartening, but not surprising: the comparison would not be very flattering for Google.
Pelicans for 3.6 Flash and 3.5 Flash-Lite (Cyber isn't available to me through the API yet.)
https://tools.simonwillison.net/markdown-svg-renderer#url=ht...
I am growing tired of these pelicans posts every time a new model is published. Feels to me like low effort personal brand promotion. Just sharing my 2 cents.
You and a few other people, but enough people still appreciate the bit that I'm going to keep doing it.
They're easy enough to skip - click the little "-" icon and you'll collapse the entire sub-thread.
+1. First thing I look for in a model announcement thread. I actually came across this one an hour ago and was sad there were no pelicans yet.
It's a decent heuristic because the better models generate better pelicans. That's all. Nobody sane is going to make a bet on a model based on a pelican. But it's cool, it's tradition by now, and it's a semblance of a good first impression for new models.
I love and appreciate you doing and sharing them, with stats and details.
thank you.
I love the pelican escapades.
But how else am I supposed to know when we've reached AGI, until I see an absolutely flawless pelican?
All of the pelicans so far have had really weird flaws / quirks so I am always a little interested to see how well these models perform at this task, since I've seen all the past pelicans and have some anchoring.
Seeing a truly flawless pelican would tell me that the model has true visual reasoning capabilities as well as good taste.
I feel the same way. It was fun at first but has gotten tiresome. Does anyone actually use these models to generate SVGs?
Yes? And even for simple interactions, they use SVG by default.
Yeah I don’t get it. It tells me which model can draw an svg of a pelican riding a bicycle. It does a great job at that and the presentation is good.
But why is this an indication of literally anything else?
It is simply a benchmark. It is well known that benchmarks are not meant to apply to every possible task you might perform.
Yes
I think you're underweighting the Pelican test.
Not only does it give you a super easy-to-grok understanding of the model quality just by looking at the image, but when you compare tokens and costs (both input and output), you really get a good, simple COST x QUALITY evaluation across models.
Simon explains it well: https://simonwillison.net/2026/Jul/16/kimi-k3/#what-can-we-l...
Simon, you should put up a summary table page that you update after every release.
At this point, it's kind of a hackernews thing. Simon posts them as a single comment in the relevant thread. It's okay for this place to have a little bit of a sense of community, and you can just ignore the comment.
Its a nice benchmark. Like hearing the ice cream truck on a summer day.
It's both.
I agree to rednb that at this point it feels like rather obvious brand building, but also, I agree with you that some value is in it.
It does not feel all that authentic though, and it's good to react allergically to lack of authenticity. Bad for a lot of business models, but good for humanity.
Sorry mate, but you sound jealous in all these replies that the Pelican domain isn't your gig. The below is as labored as nitpicks ever get:
> It does not feel all that authentic though, and it's good to react allergically to lack of authenticity. Bad for a lot of business models, but good for humanity.
I hope SimonW keeps them coming.
My ancestors are smiling at me, Imperials. Can you say the same?
More like living next to an ice cream truck car park
Every parent groans haha
At this point it does not show anything as models are fine tuned on all kinds of benchmarks.
I thought the Gemini 3.5 Flash Lite response was quite telling myself. I personally like the Pelican SVG test, to me it is still a charming snapshot of model performance anecdata. No one would argue it's rigorous but I don't think it was ever intended to be.
I get people burning out on the pelican SVG test alongside the rest of the AI burnout, but I guess for myself I'm just choosing to keep enjoying it while I still can.
Here is a different opinion. I’m always looking forward to see the pelican whenever a new model is released. It’s plain simple to understand, memorable, subtle enough in terms of details, and my favorite part is that you have been doing them consistently long enough for it to be useful for comparing almost everything with anything.
Yeah. It's something I can do myself in a couple seconds if I want, also on more varied SVG scenes. If this is going to be a benchmark people turn to I'd like to see more effort put into it than just a one-sentence prompt.
Disagree. They're a nice tradition, but besides that, they're a useful way of eyeballing improvements. I realise labs are likely to be training for Pelicans - but if they're all training for them, the differences in the results are as indicative as they were before labs trained for them.
The 3.6 Flash pelican is just about the best I've seen.
My 2 cents: you don’t have to look at the pelican if you don’t want to.
All the models do this well. It's a test that tell us nothing at this point.
Hard disagree. I love a little bit of whimsy (which I feel the world is lacking more and more everyday) from Simon everytime a new model is announced.
Perhaps freshen it up and extend the test by feeding the model the rendered output so it can iterate once. Assuming a multi-modal model.
I'm happy for Simon to post what he wants, when he wants. He's earned it.
I love them. Keep 'em coming.
Vibe code an extension that autocollapses any post mentioning pelicans and by simonw?
I've been close to writing one that will automatically upvote ALL downvoted posts. I'd call it something like Anti-echochamber.HN
Do something instead of complain
Your comment reads very pedantic with a hint of jealousy. The pelican and xbox controllers are great ways to see how well it can follow direction dealing with svg a difficult format for LLMs to use and testing their spatial vision awareness.
It's just how he is. Prior to LLMs he was cramming a datasette link into every thread. Downvote and move on.
I find Simon's work informative and entertaining; the last thing he can be accused of is low effort. The Pelicans are just a bit of fun icing on top.
Sending a one sentence prompt to an LLM and posting it to hackernews constantly isn’t low effort? Today I learnt something new
I'll split the difference. When it's a blog post there's usually an interesting observation or two, but if it's totally automated? Maybe just do the ones with a post.
I like seeing the pelicans, it's a tradition.
Yeah and it's surely in the training data by now. Long past time to stop.
You say that, and yet 3.5 Flash-Lite produced an SVG without a pelican.
A new model arrives. The pelican, with uncanny commercial instinct, is never far behind.
Sponsored blogs and paid newsletters are after all, notoriously poor at subsisting on silence :)
Linking directly to the rendered markdown as opposed to a post on my blog is a poor way to promote my blog.
A piece of the frame is missing between pedals and back wheel. The frame of the bike passes through the bird. It also puts a cap on the bird's head, and a fish in it's mouth.
The fish and the cap where always added when I asked an llm to improve it's first attempt.
This continues the trend in LLM progress of better=more stuff
Edit: I wonder if this is a function of the reasoning training, where more tokens/ stuff is rewarded.
I generated a very stylish Pelican using the webapp. Hard to put a judgement on it relative to yours https://share.gemini.google/XSfmve2mEGDV
3.1 Flash Lite has a better pelican that 3.6?
You should be banned for your constant spamming of this. Its ridiculous. Every single AI post! Constant personal promotion.
Flash-lite did the John Cena Pelican
Pricing per million input/output tokens:
2.5 Flash: $0.3 / $2.5
3.0 Flash: $0.5 / $3
3.5 Flash: $1.5 / $9
3.6 Flash: $1.5 / $7.5
---
2.5 Flash-Lite: $0.1 / $0.4
3.1 Flash-Lite: $0.25 / $1.5
3.5 Flash-Lite: $0.3 / $2.5
3.6 Flash would be a great model at 3.0 flash pricing. At this pricing, its thoroughly trounced by about 10 models on cost/performance including Grok 4.5. 3.5 Flash-ite would be a great model at 2.5 flash-lite pricing, as is, its trounced by many models including Deepseek v4 Flash.
As is, they are thoroughly outclassed for most usecases. I will say the one area where i do see Gemini punching above its weight class is in tasks that are effectively "Google this for me" / knowledge stuff. So it does have a role, and I do use it. So while I think Google is still in a strong position overall, they are really stuck as a tier 2 AI player right now with text models. They are tier 1 in bio, images, and video.
Could also be that they are pricing it at levels where they actually make money. Without seeing the behind the scenes compute cost on all of these its hard to really judge.
That being said with any open model we of course do know the total cost (or estimate)
3.5 Flash was always too expensive for a "flash" model. They marketed it as "near frontier" level, but there are several order-of-magnitude cheaper open models that compete with it.
In my tests, 3.6 Flash is NOT more token efficient, so it actually ends up costing more than 3.5 Flash, even with the output price reduction.
EDIT: It less less verbose in final output though, but it reasons more.
I assume the optimization comes when you have long-running tasks with many tool calls, and by reasoning more, it reduces the number of tool calls needed.
Am I off, or does Google have the pricing that varies the most between model generation releases?
It seems that they're trying to push up-market, or at least they were.
Given the extremely competitive releases of GLM 5.2 and DeepSeek V4 (both pro and flash), I don't think there'll be appetite for it.
It seems like they're sticking to a static pricing plan that was made when the only relevant competition were the US labs (im not counting deepseek 2025 as serious competition -> glm and then kimi on the other hand, now that's a different story).
Pricing often reflects what the vendors (expects) the customer is willing to pay. It seems that Google is still trying to find their niche in the market.
I wonder if this is a plateau towards the real pricing of AI, if you layer in gemini-2.0-flash at $0.10 / $0.70 then its a 15x price increase to 3.5/3.6 flash. But it hasnt gone up again which is interesting.
2.5 flash was the only reason we were paying four digits a month to Google....
i guess we'll use 3.0 flash but thats going to get replaced too right ?
these flash lite models aren't very reliable or consistent
A couple tidbits:
> Beyond today’s releases, Gemini 3.5 Pro is currently testing with partners and we plan to make it broadly available as soon as it’s ready.
> We have started our most ambitious pre-training run yet, for Gemini 4, and are excited by the progress.
While them fixing token bloat on 3.5 Flash is good work. That paragraph was the real highlight.
Hopefully 3.5 Pro is soon, and that Gemini 4 can be here end of year and finally have an updated knowledge cutoff.
I guess they meant to release Gemini 3.5 Pro shortly after 3.5 Flash, but then Mythos/Fable and later GPT-5.6 came out with higher performance than 3.5 Pro, so the managers decided not to release it.
It's kind of ridiculous how good these are getting. 3.5 Flash lite is pretty comparable to Opus 4.8 (at least for the couple tests I did) while simultaneously being 6x faster and 19x cheaper.
https://fy2zp1ri90.evvl.io/
Not for me personally. While setting up a custom website, 3.5 Flash introduced tons of bugs that Claude had to fix. The website has about 3,000 lines of code spread across multiple files, and Gemini somehow couldn't do frontend changes without breaking things. Sharing my 2c, but I've stayed on GPT 5.5+ and Claude Sonnet/Opus 4.6+. Anything past that from those two have been bug-free, but Google's latest hasn't been.
It is both less intelligent and more expensive than GLM-5.2, while being closed weight.
But they make up for it by shipping it late.
It's also got vision and audio. So, the better comparison is any of the other large Chinese open models that are better and cheaper than Gemini Flash.
Google's models are always very well spoken and much more pleasant to talk to. And I've used the open weight models a lot, too.
It's also about 15x faster.
It's roughly equal on price and intelligence as GLM 5.2 while being ~8x faster.
It’s multimodal though.
> It’s multimodal though.
Sure.
And how does that make your day better? I know it does not improve my work in any way shape or form.
I'll take a better coding model that's not multi-modal any time.
If I need an LLM to do images or sound, I'd rather use a dedicated one instead of a jack-of-all-trades-master-of-none model.
Personally, I often paste screenshots into Claude Code of the application it’s working on. And I’ve even had it work autonomously on something and regularly grab its own screenshots.
Or sometimes I will have tables, charts, or even screenshots of text that I would otherwise have to have another step to OCR or type out.
Multimodal saves me time on a regular basis. Not sure it’s a game changer, but just lets me communicate with the model in all sorts of ways that would be harder otherwise.
At least 30% of my queries to an LLM include deciphering something from an image. If I’m coding pretty much a hundred percent of bugs include some sort of image.
It'd be interesting to know how much the Intelligence as a Service angle serves as a value-add in the minds of Google's executives.
You can get decent open-weight models now. That's not difficult. The difficulty is 1) running them and 2) compliance.
My company runs Claude on GCP's Vertex AI solution. We're in the US healthcare IT space, so the models need to be from somewhere that American healthcare agencies and companies have traditionally been okay with sourcing code from - which means the US, Canada, and maybe Europe. The stuff that handles PHI/PII must be in the US. The expense of hosting is more of a PITA than most customers want to go through this early in the technology's lifecycle, and intelligence gains are simply a matter of degree for most business tasks.
In theory, we could find some open-weight model (likely from China) for our development agentic work and host it anywhere you can host AI models. We don't, though, and I think Google, OpenAI/Microsoft, and Anthropic see that as the core of their business.
Is that statement based on token price? More and more it seems that $/token hides as much as it reveals. Token efficiency, tokenizer differences, etc. I'm not saying that you are wrong, I am just saying it is becoming a bit more difficult making statements like this without a bit more research.
It's based on the Artificial Analysis "Intelligence Index vs. Cost per Intelligence Index Task" here:
https://artificialanalysis.ai/#intelligence-comparison-tabs
Differences in token "density" are accounted for by pricing per task
It's scary relying on Google's models.
I have a very price sensitive workload that used to run on flash 2.5 lite - it's deprecated now.
The replacement 3.1 flash lite is a lot more expensive, but now also has a sunset date.
3.5 flash lite is even more expensive.
So the price is rising and you have no choice but to keep paying more and more.
I moved directly from 2.5 flash lite to deepseek v4 flash, its already cheaper and if your prompt caching is good you can save so much more money.
could you explain how to optimize prompt caching or point to a doc about it?
Anything Sam Rose is worth reading: https://ngrok.com/blog/prompt-caching
but the implementation will be up to your provider and harness, for deepseek, they expose some numbers: https://api-docs.deepseek.com/guides/kv_cache/ and Anthropic has a list of actions invalidating your cache: https://platform.claude.com/docs/en/build-with-claude/prompt...
Basically, you avoid anything dynamic: model change, tool change, etc it's also important that your system prompt or main prompt doesn't have non-static data like the date/time/place or someone's name (the person you interact with in a chatbot for example). That should be left to tool call or search.
Sam Rose here. Thank you <3
The man himself, thank you for the articles =)
You are extremely welcome.
samwho? samrose.
I just put the varying parameters in a trailer prompt and have them change every time. It doesn’t matter because the cache is prefix keyed. You lose caching for the last 20 tokens or so but that’s not a big deal. Moving it to a tool call makes it too slow (needs full roundtrip).
If you’re constructing the prompt you don’t have to jam everything together you can arrange it appropriately.
Yes indeed! Mostly don't put changing data in the beginning or prepend.
Not an open source, but I discuss it in my book with examples for OpenAI/Anthropic/Gemini, https://crimede-coder.com/blogposts/2026/LLMsForMortals.
All of the models, you need to have a consistent input to get the cache hit. So if you are chatting with a document, and change the system prompt, it will be a cache miss, even if the rest of the items are all the same. If you even pass in the document in not the same order as the prompts, it will be a cache miss. Or if you add tool calls or structured outputs, it will be a cache miss. (Since those generally go at the beginning of the prompt call, not at the end.)
Most of the time when reading documents from URLs directly it will never cache. (Need to typically pass in the bytes directly, or use the provider document store index.)
Gemini has a 4096 minimum token size with the 3 version models before even getting a cache hit. OpenAI it is lower (1024), and is automatic, but only happens in increments of 124. Anthropic can also get cache hits at 1024 tokens, but you need to explicit ask for it (and pay extra).
Caching by default typically lives for 5 minutes since the last cache hit across providers. But some of them you can ask for longer. AWS for Anthropic models can be tricky with multiple endpoint routing, so can get cache misses if it happens to route to a different endpoint.
That’s part of why, since Firebase, I’ve tried to never depend on Google products for business, especially not GCP.
Features stay in Beta for ages, whatever that actually means, and released ones get deprecated things fast.
Where some of the competitions treats deprecating entire services as "let’s not put it on your frontpage, put deprecation notices all over the doc, and politely ask new users not to start new project with them".
Just switch the model, its not that much effort tbh. And u can also get a cheaper model than 2.5 lite for the same intelligence
its not always that simple. dropping in a new model is trivial, but highly specific workflows may rely on specific _invisible_ aspects of a model. when that model gets deprecated, the workflow needs to be rebuilt/re-tuned to work with a different model.
google's inability or unwillingness to provide stable timelines for model deprecation makes it risky to build complex workflows using their models
Load-bearing (whoops) quirks were noticeable months back, but haven't most flagship models become predictable and reliable?
it does seem to be moving in that direction. There were really specific things (large, complex json outputs) that gemini-2.5 flash was basically the only model that seemed capable of reliably for a long period. gpt-5+ has covered the usecase for us now pretty well but still evals slightly below what 2.5 could do
100% agreed in the same boat right now. Feeling really screwed over by Google rn
You would be surprised how much of a difference the model makes for certain niche tasks.
For my use case, `gemini-3.1-flash-lite` is ~20% higher accuracy than the next best model of comparable cost (considering both proprietary and open-weight alternatives)
Well it is a bit surprising that 3.1 flash-lite could be better than deepseek-v4-pro (cheaper output and way cheaper cache so might cost less for quite a few use cases).
They are not anywhere close according to pretty much every benchmark (even v4-flash is considerably ahead and its way cheaper than flash-lite). Maybe tuning prompts/tools/etc. might be useful?
Gemini flash lite family of models currently has the best ratio for price/speed/intelligence for understanding images, no real alternative AFAIK
"Intelligence" being what, math? Coding? Unfortunately there's a billion use cases for LLMs whose performance is not at all captured by the popular benchmarks they're all trying to maxx.
if you are relying on a model for a business process, it should be simple enough to benchmark on that process
> So the price is rising and you have no choice but to keep paying more and more.
I presume you can't use deepseek?
There are plenty of 3rd party providers hosting deepseek models, if you don't want to use the 1st party API. 3rd party providers are generally slightly more expensive, but still quite cheap compared to other models of similar vintage and size.
sadly it's not multimodal
They know that there's big enterprises that will have a strong preference to work with another big enterprise instead of relying on a younger company. At least that's why I think they believe they can do this sort of thing and get away with it.
same here. our production workloads was on Gemini for 2 years. seeing Google unilaterally dropping perfectly fine models and charing you 50x more for worse results is not good.
we are switching to Deepseek.
Opencode Go is just the same. Each month I will I can do less. Dont ask me why?
I'm running price-sensitive data extraction workloads on flash 2.5 and its still the king when it comes to accuracy + cost, all the gemini 3 variants perform a bit worse and cost a lot more. Low-key freaking out, ngl
And somehow, the most annoying is not even the price hike, but it is that is you expect to build a product on any of theirs models, they spend their time being deprecated and you have like to be on the lookup to start from scratch selecting a model and fitting it every year or so... Impossible to have any stability...
I felt the same way about openai's text-davinci-002 and code-davinci-002 (gpt-3.5). They were amazing completion models and openai basically dumped them with no equal cost or equal performance replacement. Instead all their models are opaque with no ability to work in completion mode where one actually controls the text input to the model.
These days no company even has completion models where one controls the text input fully. Worthless.
All models are increasing in price. Everything up to now has been subsidized by investors, private and public.
same I just switched to OpenAI after using flash 2.5 lite for almost everything at our company. We spent thousands just to build this workflow now Google says screw off
>So the price is rising and you have no choice but to keep paying more and more.
You can also just write code like you did a year or two ago.
No word about updating Jules, which is still stuck on 3.1 Pro. I get that it's probably niche but I've really appreciated basically being able to give directions to Jules on my phone, then reviewing and merging a GitHub PR fifteen minutes later. It's been great for getting some progress in on a few personal projects during my commute when I can't exactly pull out my laptop.
Anyone have any good alternatives?
Both Claude and Codex can code in the cloud, it works quite well!
I tested Jules and while the idea is good in theory, I found the model's intelligence to be very lackluster.
Shame. I'm on the $20/mo Gemini Pro plan because the 5tb of cloud storage and the youtube premium lite were good enough perks, and my coding complexity needs were light enough for me to overlook Claude or Codex. But Antigravity is working better than Jules and it's basically giving me a taste of what I'm missing and it's harder to justify not trying out the competitors.
I do not, however I am curious about Jules support. I didn't know if this was a dead project or not. Seemed really interesting but then I didn't see much development/announcements/discussions around it. Last update from their changelog was as you said 3.1-pro support in March.
I have no affiliations with the team or product, but Superconductor reminded me of Jules when I tried it a couple of months ago.
It might be overkill features-wise, but there's a free tier and it likely won't be left for dead anytime soon.
If you own a Raspberry Pi or similar: Hermes + Tailscale + iSH over tmux.
Claude Code
Wow - Google does not even bother to show benchmarks of these models compared to the frontier and Chinese labs - only against previous versions. I'm not surprised. Having worked there for years it was amazing just how inwardly looking the company is.
Really, what's up with Gemini still not supporting connectors/MCPs/plugins/whatever-they're-called-this-month on web? It makes it a non-starter for any kind of serious use.
The mention of an "ambitious" gemini 4 pre-train signals to me that 3.5 pro is probably a lost cause.
That being said, it seems that Gemini is still the best image analysis model, so hopefully 3.6 flash builds on this even more.
Their naming scheme is confusing. Branding has never been Google's strong suit and their marketing copy is pretty bottom-of-the-barrel[0]. Anthropic has a pretty clear set of models but Gemini decided to rebrand their Flash as Flash Lite (and presumably the future will see a Flash Lite Mini, a Flash Lite Mini Nano and a Flash Lite Mini Nano 3B) which confuses the pricing to high hell.
This plus the Vertex, AI Studio, Gemini, Antigravity. It's honestly too confusing to use. I need to use Gemini just to decide on which platform and which model to consider.
0: Famous Kurian Tweet: "We're announcing Duet AI for Google Workspace will now be Gemini for Google Workspace. Consumers and organizations of all sizes can access Gemini across the Workspace apps they know and love. We're introducing a new offering called Gemini Business, which lets organizations use generative AI in Workspace at a lower price point than Gemini Enterprise, which replaces Duet AI for Workspace Enterprise."
I have a side business selling custom fingerprint jewelry and I use gemini nano banana to clean up customer submitted fingerprint images. This was a step I used to do by hand at 10 - 15 minutes per image and nano banana is the first model that is able to do the task (it is astonishingly good at it). I can't wait to see what the next nano banana can do, hopefully its released soon.
Are your customers clearly informed that you're sending their immutable fingerprints to an AI service?
Their shop is linked to in the bio. They don't seem to have a privacy policy up on the site. When in the checkout form it links to the generic Shopify privacy policy. No hints to the fact that uploaded images are processed by third parties, as far as I can tell. That should be corrected for sure.
Yes this is extremely unresponsible if so. Fingerprints are legally protected biometric data in most juristictions.
It is not mentioned in their FAQ. [0].
[0] https://lulimjewelry.com/pages/faq
So it's GLM-5.2 performance for almost twice the price.
That said, the speed looks really good. I think it's competitive with Fireworks's GLM 5.2 Fast, although Fireworks is still cheaper.
Google seems to have anorexia when it comes to model intelligence. They have an internal hard constraint on price per token it seems, and they are trying to squeeze out intelligence with limited compute.
I wonder if there is something with their TPU cycles that makes them want to postpone training a new model. My guess is that they have been on the same base model for 6 months and they may have waited for the next gen TPUs to train Gemini 4, which greatly limits how much intelligence they can increase and forces them to do cost efficiency increases.
Could it be that they have to serve their models to billions of users?
> Could it be that they have to serve their models to billions of users?
And how is that different from their competitors exactly?
I'd guess they did model-hardware codesign but the design ended up limiting the scaling capability of the model (i.e. they overoptimized too soon).
Google Cloud is probably Google Deepminds biggest competitor. Big company kinda bullshit.
How so?
Google cloud sells compute out from under Deepmind to other labs. So they basically are in competition with Google cloud for compute.
Unfortunately says more about how competitive 3.5 pro would be today at the frontier if they forgo it for 3.6 flash.
3.5-lite is the real showpiece here; agentic models of this size are a huge value-add for 90% of knowledge work agent tasks
Google really needs to get their product strategy together. The discontinued gemini-cli and introduced antigravity-cli which is a downgrade IMO and the sooner they can partner up with AWS and release the gemini models via Bedrock the easier corporate/business which has strict data protection rules can use their models and make them available for internal engineers.
It's a one thing to research and improve the model, but if they ignore the ease of access and multi-availability of their models in different ways they are going to fall behind again.
Google has not changed. Following two facts are like tautologies by now.
1. Their AI efforts are very fundamental research oriented. They are really good at it.
2. Their productization sucks. The end products gets little attention compared to competition. It can be canceled at any time. You should never build anything around Google only APIs, AI or not.
Wow, Laguna S 2.1 (released today) just destroys Flash-Lite underly and completely. What a weak and embarrasing release from Google.
I'm more excited for 3.5 pro. Gemini has fallen behind in some areas, but is still one of the best multimodal models.
Has anybody found any models better at image or audio analysis?
Came here to ask basically this. We use 3.1 Pro internally and it's great.
Why would 3.6 flash perform a little worse than 3.5 flash on Artificial Analysis Coding Index...
https://artificialanalysis.ai/models/gemini-3-6-flash?intell...
Because AA Coding "Index" consists only of two benchmarks (Terminal-Bench v2.1, SciCode) and generally fails to be meaningfully representative of agentic coding capabilities.
AA coding index has been updated to use DeepSWE, Terminal-Bench v2, and SWE-Atlas-QnA.
When? It literally says on the page for Gemini 3.6 Flash "Artificial Analysis Coding Index represents the weighted average of coding benchmarks in the Artificial Analysis Intelligence Index (Terminal-Bench v2.1, SciCode)"
Whats a better option for AA Coding Index?
DeepSWE and FrontierCode are more realistic if you read up on what they actually measure. But the most realistic is to try it yourself. Benchmarks can only vaguely represent typical usage, and how you judge the result. Giving the same real task you have to a few models will make you understand them better than chasing benchmarks.
it would be nice if these benchmark reports actually specified which tasks they passed and which ones they didn't.
Could be a good tradeoff for the flash model though. 3.5 -> 3.6 is a tiny bit cheaper and maybe faster?
artificialanalysis.ai has it going from 165 tps -> 304 tps. openrouter.ai needs more data but it has it going from ~100 tps -> ~150 tps, though at peak 3.5 has reached 156tps.
Tons of guardrails, lazy model, super confusing plans, expensive 3.5/3.6 flash and lite and 3.5 pro MiA?
Rough patch for google ai
In other good news "the model has been trained to minimize refusals for beneficial uses.".
Otherwise, this news feels like a tiny incremental improvement on Gemini Flash series to make it more efficient with token usage, subagent and cost. Nothing big.
Regarding their benchmark scores on CyberGym, I wonder why they didn't compare their 3.5 Flash Cyber model with Fable 5. I mean they included Mythos and GPT-Cyber, so why not Fable 5 too?
They also mentioned Gemini 3.5 Pro is in testing and its about to become available very soon. Another thing maybe worth discussing is the announcement of pre-training Gemini 4. Sadly, not much technical details to discuss on. Many comments in here seem to mostly be about how Google is behind the others, but honestly, is it really worth the investment to be #1 in Artifical Analysis every week?
Pelican svg and a near-perfect 3D MacBook at max effort for $0.16, about a fifth of Fable's price.
Fable 5 still wins on detail with no visible errors, but it's close. And this isn't a memorized pelican;
https://playcode.io/blog/macbook-svg-benchmark#gemini-3-6-fl...
I don't know if it's a rendering error since it looks like your site renders the SVG instead of hosting a static image of it, but the 3.6 macbook looks like an abstract art piece lol, both ff and chrome desktop
From my experience, this thing is crazy fast.
Spawn 10 on the same problem and have them debate to reach a consensus, you’ll get Fable-like results but 100x faster.
Gemini 3.5 flash-lite is more expensive than Gemini 3.1 flash-lite. Every upgrade is getting more expensive.
3.5 Flash-Lite seems available in US region, as was 3.5 Flash; but 3.6 Flash looks Global only so far when pinging. If Google employees are watching, will this issue go away?
Bottom line it looks about on equal footing with GLM 5.2 in terms of both overall intelligence and cost per task, while being significantly faster (in fact it is the fastest model on artificial analysis as of rn [0])
[0] https://artificialanalysis.ai/models/gemini-3-6-flash
It is 17% more token-efficient than 3.5 and performs significantly better in coding and tool usage benchmarks.
It is also cheaper than 3.5:
> This enhanced efficiency is also combined with a lower price than 3.5 Flash. At $1.50/1M input tokens and $7.50/1M output tokens, 3.6 Flash reduces the overall cost per agentic task, making agents more cost-effective to build and run.
I have just tried to switch to 3.6 instead of 3.5 in antigravity and it seems to constantly spit "critical instruction: STOP CALLING TOOLS NOW. YOU MUST WAIT FOR WAKEUP. ". I think I will switch back to 3.5
I struggle to see any value in this when DeepSeek is still a thing.
It's smarter than DeepSeek v4 Pro (preview) on several benchmarks, like HLE.
multimodal + latency
Kind of excited about this. 3.5 Flash on Antigravity has surprised me recently on a hobby project. When given opportunity to plan, it can deliver on tasks that would take me a while on my own and generates responses at blazing speeds - compared to what I'm used to at work with Opus 4.8 (granted I don't use Opus 4.8 on my hobby projects so just anecdotal). While with Gemini CLI I would just watch it run in circles and run out of 5h allowance before anything useful is produced (or even approached).
Spent half an hour just now benchmarking it against my current 3.5 Flash pipeline excited only see it regressed slightly (0.1% - 0.2% at most, for feature extraction work)
Seems like this is mostly a cost play by Google, hoping this doesn't bring 3.5 Flash capabilities to an end of life, and that 3.6 catches up or gets better.
Proof-of-life release while they figure out how to have a competitive frontier model release. My hunch is they pushed too far in the "omni" model direction, that they made something so ungainly, it wasn't as good for normal tasks.
I often use Gemini free web chat because it's generally quite good at web search-related questions (apparently it has direct token-level access to the Google Search index) but I noticed in the last two weeks output quality of 3.5 Flash seriously degraded. Maybe they were switching over systems.
Google's Knowledge Graph is a massive advantage no other competitor has. I don't think they've fully utilized its full potential but I don't know if any other company could've built something like Scholar Labs
Knowledge Graph + automated transcriptions of almost every YouTube video = giant untapped moat of data
It's not exactly untapped, my AI company has scraped YouTube transcripts for 3 years now for RAG
Here's the issue:
GLM 5.2 is better, also cheaper, and almost as fast.
So essentially, a big L for Google. Combine this with them not being able to produce a frontier model this generation... hmm implications
Counter-point, cost per task is almost the same ($0.47 vs $0.50) between GLM 5.2 and Gemini 3.6 Flash. [0] Not to mention the subscription plans likely produce 10x value compared to GLM 5.2 API, unsure the rate limits on a equal subscription vs subscription, but Google subscriptions offer tons of other value, including 1 year of Free Gemini for Education accounts.
[1] https://artificialanalysis.ai/models/gemini-3-6-flash
3.6 is roughly 50% faster, which isn't totally insignificant for being marginally more expensive.[1]
[1]artificialanalysis.ai
Sure, but there's no sota alternative from Google. That's it, and its beaten by GLM 5.2 on every measure except somewhat speed.
I find that quite staggering. GLM is open weights
Speed is definitely a marketable quality. All these things are a trade-off and solely measuring against SOTA I don't feel is always helpful.
Always happy to see new Gemini releases as IMO Antigravity Pro 16.67/mo plan (Annual) is still the best plan available and have been pretty happy with Antigravity IDE.
If it wasn't for Gemini/Antigravity I'd have to go with a Max Claude plan, as it stands now I can get by with just a Claude Pro plan to get Opus when I need it, whilst using Antigravity as my day-to-day workhorse.
Unfortunately Gemini Flash became too expensive to use as a general purpose model (i.e. for AI features in Apps), luckily there are plenty of cheaper Chinese models to fill that gap now.
What's the current outlook on Antigravity IDE vs. 2.0? How long will they begrudgingly keep it going before kicking everyone to 2.0/3.0?
(I actually use a mix of both for some offline projects, nothing serious.)
Why do you think it's the best plan available?
(Rate limits * capability of the model) / cost
Feels like they released this to ride the wave of press of GPT-5.6, Kimi K3, and Qwen 3.8. Doesn't feel like Google has much substance with this post except a bump in version and tweaked their pricing.
Don't know why are they even pursuing Gemini. Just download the Kimi, call it Kimini and serve it on your GPU. Maybe then train next architecture based on this!
Gemini 2.5 Flash-Lite has been my go to for cheap document processing at scale (especially with 50% off batch mode), but they are really boiling the frog with pricing increases with each version:
gemini-2.5-flash-lite: $0.10 input / $0.40 output
gemini-3.1-flash-lite: $0.25 input / $1.50 output
gemini-3.5-flash-lite: $0.30 input / $2.50 output (a 6.25x increase over 2.5!)
Now watch them deprecate Gemini 2.5 Flash-Lite in the coming months...
How has your experience been with Gemma 4?
gemma4 is the same price as 2.5-flash-lite, and performs better.
3.6 Flash scores exactly the same as 3.5 Flash on the Artificial Analysis index. Better on some tasks, worse on others. Mostly within what I'd consider the noise window. Looks pretty much indistinguishable from 3.5 Flash, at least on these benchmarks: https://artificialanalysis.ai/models/gemini-3-6-flash
Looks like 3.6 Flash is the first model with their newest pretraining run (cutoff date is 2026/03), long after 2.5 series.
I deeply wish Google would focus on models like Gemma. Small, powerful, open-weight models you can run on phones or regular computer hardware.
Gemma 4 was released in April. It's a good series of multimodal models.
gemma 4 thinks joe biden is president
I asked Gemma 4 E2B, and if you use it as a reasoning model, it will give a better answer (that it doesn't know; it was also using the date information from the prompt.)
Small open source models shouldn't be used for world knowledge, that's not their purpose.
Why not? Seems like a cop out.
Being able to ask questions to small open models seems.... obviously useful?
Because they don't have a lot of parameters to store general Wikipedia knowledge. They're small. Use big models that have high parameter capacity to store general information. Or build a harness around the small model that searches a knowledge base/internet.
Use the right tool for the job. It's like asking why a screwdriver isn't good at sawing wood, or calling C a terrible language because it's hard to make CRUD apps with it.
i mean eventually then will, losing means open source, vertically integrated hardware means you can opensource and win on cost
IMO Gemini has the best free tier models/app for everyday use. Muse-Spark is perhaps just slightly better, but has none of the connectivity to my GApps (for things like “create a recipe in my Google Docs from this image”).
Plus they are probably running these things on every Google search so saving tokens is a huge win for them.
Free? Did I misread the pricing details?
Free plan, the default tier without requiring a subscription. If you use through the Gemini App or gemini.google without paying anything, the model used is 3.6 Flash.
Rankings for text are here https://arena.ai/leaderboard/text
For comparison of Free Tiers: - Gemini serves 3.6-flash (rank 12) - ChatGPT serves 5.5-Instant (rank 23) - Claude serves Sonnet 5 (rank 27) - Meta AI serves muse-spark-1.1 (rank 5)
While Meta AI serves the better ranked model, it doesn't end up working that well for other things. For example, if I ask "help me buy a new raincoat", it ends up suggesting a Cambodian website, whereas Google is well integrated with Google shopping. It doesn't have the same integration with GApps outside of Gmail/Calendar. A few other email connectors are available.
Claude has one of the best interfaces with connectors, skills, and plugins galore, but the model and limits are restrictive on the free tier.
Gemini, as far as I know, I've never hit a rate limit on Flash.
I believe Gemini is going to gain market share through the free tier funnel while serving models as cost-effectively as possible. People are going to use Gemini because they use GApps and Google.
ChatGPT and Anthropic are going to be competing for the API/Business users, but for everyone else they are going have to become Google before Google becomes them.
For anyone wanting a faster overview: I ran the Gemini 3.6 Flash and 3.5 series release notes through NotebookLM and generated a short video summary. Link: https://www.youtube.com/watch?v=SUFBhvQ2tY4
LLM reception is truly extreme, even worse than AAA game releases.
Ever frontier lab lived it at least once : missing the frontier by a few months triggers extremly negative reactions, then you take back the lead for 2 weeks, and the hype cycle repeats.
I spin a mental roulette on whether the reception on a new release will be "OMG best model by far, no one will be able to catch up for months!" or "OMG this is already outdated, RIP company X, they might as well just give up now, there's no coming back from this."
It's quite a fun game. I click into the comments and see if the roulette wheel was right.
So 3.6 Flash is a somewhat of an admission that Google miscalculated by charging 3-5x for 3.5 Flash what it did for 3.0 Flash (3x input and output costs plus large token inefficiency changes) despite only modest improvements?
3.5 Flash Lite is only a hair cheaper than 3.0 Flash, but I think 3.0 Flash is a massively more capable model?
Pretty underwhelming, as expected honestly. I don't want to know what morale is like at DeepMind right now.
Especially when Google owns 15% of anthropic and serves them compute. Double especially when your boss (Hassibis) is also an early investor in Anthropic. Hell his NW might be more Anthropic than Google.
Yep, agreed. They still are not releasing anything frontier-class (Gemini Pro) at this point. Feels to me that they keep getting scooped by others (e.g. Kimi 3) and then are retrenching.
I'm a big fan of the Flash-Lite models. They're exceedingly fast and deliver great outputs for high volume use cases where you need to process requests at scale. Can't wait to try the newer version.
It does seem like their releases are getting closer together. I get the feeling they realized they were trying to roll out to their entire ecosystem and now they’re focusing more just directly on the AI model itself. I think give it a little time and they’ll start to be one of the competitors too.
Flash Lite: 0.3/m and 2.5/m
Deepseek Pro: 0.435/m 0.87/m
That's wildly ambitious pricing by Google. You can maybe get away with spicy pricing at the SOTA edge but at the lower tiers everything is a lot more price sensitive.
You need to compare cost per task buddy boy. Cost per token doesn't tell you much when you don't know how many tokens a model will use to accomplish a task
>boy
Seriously?
"..and in some benchmarks like DeepSWE by Datacurve, we observe up to 65%, all at a lower cost per output token."
"3.6 Flash delivers higher precision with fewer unwanted code edits and reduced execution loops, as seen in DeepSWE (49% vs. 37%)"
So which one is it? 65% or 49%?
First sentence is about token efficiency.
You're right, should've gotten some LLM to summarize it instead of skimming.
Or you could have read it more closely before posting a comment saying it didn't make sense. You know, the old school way.
If I'm going to read it wrong might as well have an LLM to blame.
I was expecting 3.6 Pro. It's been so long since the last Pro model...
They are working on coming up with a better code name. You know, something like ”Fable” or ”Sol”, gotta have one these days. Personally I think they should go with “Mafia”. How cool would that sound? 3.6 Mafia.
3.6 Gangsta Pro
Are they comparing 3.6 Flash to 5.6 Luna and losing? That's ruff.
Why wouldn't they? Luna isn't a Flash model. OpenAI hasn't released a flash-equivalent model since gpt-oss-120b.
Did you ask me a question and then answered it yourself in the very next sentence?
Anyway, given that both Gemini and OpenAI have 3 sizes of models, one would think Google compares their medium size to OpenAIs.
I read this as a soft let down to not expect too much from 3.5 Pro.
> We have started our most ambitious pre-training run yet, for Gemini 4, and are excited by the progress.
The benchmarks are not particularly impressive. I suppose they needed to release something since the long pause. But not clear why would I use it now.
"We made 3.6/4 Pro, but it sucks, so this is the distilled model" vibes.
That’s the fear.
I have liked using their consumer products but they don't make it easy, that's for sure.
Glad to see the price is going down but it's still too high for a "fast" model
A lot of disappointment here in the comments, but models like these aren't meant to compete with the likes of Fable or GPT 5.6.
I use 3.1 Flash Lite regularly to classify listings on eCommerce websites. It's great for this task - fast, cheap and accurate.
In fact, it was the single best model we tried in terms of the speed vs accuracy vs price tradeoffs - including the Chinese models.
Of course, 3.5 Flash was more accurate but the 5x cost increase couldn't be justified.
3.5 Flash Lite sounds like it could be a strict upgrade for our use case, without a significant increase in costs or drop in speed.
It's not GPT-6 but it's not trying to be. It's a completely different tool and great at what it does.
Nice to see that it's cheaper than 3.5
So about the same “intelligence” as Muse Spark 1.1 but 2x faster and about 2x as expensive.
Google is walking backwards, with such a pile of cash in pocket, i feel they are doomed.
tl;dr: 3.6 flash is a bit smarter than 3.5 flash, but also a bit more expensive.
My results [0] put Gemini 3.6 Flash at the top.
3.6 Flash high has same $1.5 input price as 3.5 Flash, but output is cheaper from $9.0 to $7.5.
Google said 3.6 Flash is more token efficient, but in my tests it's actually LESS token efficient[1] than 3.5 Flash, so despite the output price reduction, it still costs more.
[0]: https://aibenchy.com/compare/google-gemini-3-6-flash-medium/...
[1]: https://aibenchy.com/compare/google-gemini-3-6-flash-high/go...
Models are expensive and low performance. On top of that they make you jump through hoops to even use these models without being throttled even for the weaker models. The only reason we are using them is credits. As soon as credits run out we are switching immediately.
Haven't been excited for a Gemini release since December. Wild to see.
I'm going to get downvoted/flagged but I feel like we need a new type of "Show HN/Tell HN" etc for "New AI Model Available".
Front page is tedious these days.
If you refresh the Home page, once a day. The top post will likely be 'New AI Model Available '
> I'm going to get downvoted/flagged but [...]
Why do you care that much? Just say it. You're letting an imaginary score determine if you should express your suggestion for improvement. That's a bit wild.
No I'm not, I expressed it...
I prefaced it with that to simply say I knew it'd be an unpopular opinion.
the real product is the naming confusion we made along the way. Gemini 3.6 Flash, 3.5 Flash-Lite, 3.5 Flash Cyber — at this point even the model cards need a model to explain them
Not good enough for high-end, not cheap enough to be for low-end. Next!
quite a good model, the speed/price/quality ration is a new golden intersection for me, not sure if its as good as grok 4.5 but quite fast/capable model.
It feels like AI is going to be the end of Google. The post-Schmidt company culture cannot produce consistent, consumer-friendly products that any sane person would want to use consistently.
I have no skin in this game and this comment will be gray in a few minutes BUT a friendly reminder that these types of threads are astroturfed heavily by competitor labs and any info should be taken with a massive grain of salt.
"The model will be exclusively available to governments and trusted partners via CodeMender soon as part of a limited-access pilot program"
we are stealing plutocracy from the jaws of emancipation.
i don't want to live in a world where abundance is guarded and shared among politicians and cronies, whilst the rest are left to rot.
never tried gemini for coding, but this news seems to be compelling, i would definitely give it a try
dupe? https://news.ycombinator.com/item?id=48993130
2 red flags
1- no comparison with gemini 3.1 pro
2- no comparison with any other model
The model card has comparisons with both 3.1 Pro and other models:
https://storage.googleapis.com/deepmind-media/Model-Cards/Ge...
Specific to task these can be huge plus point.
no actual cyber model release, useless
If only I could use Pi.
Well, you can use the google models from Pi. Go to aistudio.google.com and set up an API key. There is a free quota, it's relatively large for the Flash Lite models.
...and last time I looked the limits were more generous for Gemma 4 there, but they have been tightened a bit. That's how it goes, always changing.
I remember back when Gemini looked like it was the best model that this comment section was full of confident predictions that Google had "won" and that no one would ever catch up with them again. The most embarassing part is that I kinda believed them.
I think it’s safe to say Google seems a bit out of the top AI competition now. The “cyber” stuff also starts to become laughable with open models providing the full power without the crap Anthropic, Google, OpenAI are trying to frontload on you(I.e you are not allowed to develop/review a login system, pay a special cyber operation team to do it for you). They really deserve to become irrelevant in the future of AI.
if only DeepSeeek supported vision, would never use Gemini.
Other discussion from a few minutes earlier: https://news.ycombinator.com/item?id=48993130
The silence is deafening.
Google watches over the last few months a flat out assault on the Pareto curve from American and Chinese companies. Release after release pushing the boundaries of frontier intelligence and price/performance.
And the response from arguably the biggest AI research labs in the world by headcount is Flash 3.6.
What do you do when you are given essentially unlimited resources and still find yourself falling behind?
Start your own openrouter.
I wonder whether this is more the fault of Hassabis or Pichai. They are clearly both less capable than Altman or Amodei.
Plugged 3.5 Flash Lite into an existing agent harness that was previously using 3.1 Flash Lite and this shit just does not work. It's not following instructions and is not producing the correct tool calls.
Maybe this is relevant? Just in case
> For autonomous subagents with tool calls, code execution, or multi-step reasoning: set thinking_level to "medium" or "high" to prevent premature tool termination.
I just happened to see that in the docs: https://ai.google.dev/gemini-api/docs/latest-model
Thanks, I'll give that a try.
Gemini 3.5 flash is already a pretty good model. But, unfortunately, the primary way you can interact with it for coding is through Antigravity - which is actively developer hostile.
It doesn't matter how good the model is if you're (mostly) forced to use it in Antigravity - which turns any model into crap.
Wake me up when Antigravity doesn't suck.
Why exactly are they announcing these completely milquetoast models ?
I'd be low-keying the release if anything, given how lame they are compared to their competition.
What am I missing?
I use 3.5 flash 10x more than any other model, despite have access to all of them. If I'm going to play a slot machine, I'd rather get the pain over with quickly.
We're almost five years into the whole GenAI thing and we're still relying on these guys to spoonfeed us incremental updates.
It's time for them to start focusing on open-weight models and efficiency. Otherwise there's just a layer of marketing hype and "will it do this?" that has to be cut through for evaluation of each and every release cycle.
Models are getting easier and easier to create. The money, if there's any here, is in the harness the user interfaces with, and the data centers running them.
Google seems to be falling way behind the pack. antigravity cli is pure trash. gpt 3.5 pro is now behind and isn't released yet. GPT 6 and Fable 6 releasing next month. What the hell is going on over there ?
> What the hell is going on over there
Google was late to coding agents and as-per-usual fucked it up with their crazy project management culture.
Usually Google gets away with it due to inertia, however this time they are paying a heavy price because they missed out on the training data that Anthropic and OpenAI have gathered with claude and codex.
Just switched to AI Plus from Pro, seems like I won't be missing much.
> we have taken an intentional approach to deploying 3.5 Flash Cyber. The model will be exclusively available to governments and trusted partners
Screw your government! US and Israeli governments should get the least access, but of course we all know they'll be the (only) ones to get full unfiltered access.
3.5 Pro must really suck.
They are comparing against their own previous models instead of competitors. Not a great sign.
At this point just put the Pareto in the bag bruh
whatever, dude. give gemma5
is it just me or is this one-upping each other every few days getting ridiculous secreting a whiff of desperation?
Just you. This is typical market competition in a fast moving field.
Some more discussion:
Gemini 3.6 Flash https://news.ycombinator.com/item?id=48993130
I keep saying this and people dont believe me, but I have b2b saas systems with actual agents running around the clock, and the performance/stability of the flash model is higher than most other models.
Meaning, its predictable with tool calls, wont spin off a million tools/do weird behavior, its reasonable. Even sonnet in a real world decision making scenario is not reliable, or will reason so long its incredibly expensive.
The benchmarks arent catching all the value, and most people have never actually ran an ai agent in a real context that matters
Who's most people? What are you talking about? Most people here use agents every day and I wouldn't trust flash or pro to touch any important project of mine because they're both terrible compared to the competition, waste of time every time I give them a chance
I mean like an ai agent doing some sort of HR work, not a coding agent. Very few businesses are trusting an autonomous agent.
With all the naysayers on Gemini models I'm curious how many people actually use Gemini regularly?
For me, Gemini models are the most usable. Claude Opus and Mistral always try to turn queries into one-shot enormous commits, which just burns tokens, time and annoys me for something which is still wrong more often than not.
Gemini seems far better at listening to instructions and giving me what I actually want, on top of using far fewer tokens and wasting my time. Fable is the only model that's come close to Gemini Pro for me.
And as this is about Flash, it's exciting, I find Flash can usually get the right answer pretty quickly and without too much nonsense.
I use all of the major providers daily and I tend to go to Gemini for "fast lookups" where a good enough answer is probably OK. I use ChatGPT and Claude for anything where it matters and generally when I invoke all three -- Gemini is the most surface level with responses, and also sycophantic.
It gets worse from there. Gemini is terrible at agentic coding, primarily because Agy is terrible. I noticed Google updated Agy with this release, so perhaps that's finally going in a good direction. I'll have to test it. Thus far, my experience in countless experiments has been Gemini models being 2x faster yet with less depth in their solutions and a lot more going off track.
I very rarely have to stop Codex or Claude Code sessions because they're doing something random and unexpected (or not asked for). I genuinely think Gemini models are brilliant but virtually useless in agentic scenarios in my experience of the last few years (2.5, 3, 3.1, 3.5).
I should also note that Gemini web UI annoying resets to its lowest intelligence which feels scummy and Google is not transparent about what "extended thinking" really is. Past posts have pointed to "extended" being medium. Every other provider gives you the raw value (medium/high/etc).
So honestly, I feel Google would rather I don't use their models. They just want to get a little bit of mindshare and stay in the conversation. I had the Ultra plan and cancelled it once it was apparent they were not improving the agentic experience nor trying to compete.
not a google fanboy by any stretch... though i've been thrilled with the flash line of models... i exclusively use it on high, and have found it to be a great fit for increasing productivity 10-fold while maintaining quality... sure it can't just go off and one-shot a bunch of work, but at the complexity level i tend to work at, neither can the frontier in a robust way that i can be confident in... sure i have to be in the loop more, but that helps keep me grounded and course-correct earlier before wasting tokens... and when you sufficiently spec out a coding/software problem, and i mean really document all of the critical nuance, it will successfully satisfy the constraints... the quality is rarely acceptable on first-pass, but it forces me to stay connected to the architecture more than i would be if using a frontier model... i've found this to be a happy middle-ground of productivity and awareness...
tested the models on aistudio. despite that the knowledge cut off is march 2026 it still knows nothing about 2025!
you can check by asking "list notable world events in 2025, only list unplanned" on aistudio. or you can ask for Charlie Kirk, it also does not know. I tried it multiple time to ensure that I didn't not get routed to older models!
> but google has search
irrelevant, without deeper knowledge about cutting edge technologies or latest libraries, all of it suggestions are crap. even you ask it to search it will still use outdated keyword thus only getting outdated information.
in other word, what a disaster!
Google desperately needs to make some leadership changes within their Gemini team now that they've been surpassed by 3-5 open weight models and risk loosing frontier status all together in the near future.
Open weight models aren't likely to be open weight in the long-term. China has started considering export controlling and limiting access to model weights [0].
[0] - https://www.ft.com/content/6049a031-9e9b-464c-97bb-414da04d5...
That contradicts this:
https://www.wsj.com/tech/ai/chinas-xi-touts-open-source-ai-a...
So who even knows.
They don't need to be open weight in the long term; once there's an open-weight Fable-level model with 1M context it'll be pretty much good enough for all coding tasks, no need for new models.
At this point, I think google should consider becoming a hyper scaler for anthropic and open ai, and I predict that that is exactly what they do. The model is no longer the most valuable part of the stack.