Some conjecture on my part, but I imagine that the bugs being found are not the kind you can just press a button on the LLM to fix anyway, but are instead the sort of bugs which require the reviewer to deeply consider design tradeoffs, of which I believe there are simply more in a DB or programming language codebase, for example. Stuff I am finding in review is often like "this thing isn't like the other thing, which has this behavior, which is wrong if the aim is X, but sort of correct if the aim is Y, and totally perfect is the goal is Z and the use case is J, but not I. Unless it's Q, in which case you probably need to reconsider some fundamental decision making from 12 months ago and have some conversations with people about how users desire it to work in the first place, and then set about understanding the tradeoffs inherent to that design."
ETA: This is extremely generous to the LLMs, btw. If I got an LLM to pop something this concise out I would fully commit to finding a different profession tomorrow, but for now I am for better or worse stuck doing prompting some non-zero number of times in order to get all of this assembled context, and then need to use my human powers to decide whether or not more humans indeed need to discuss it.
I like the way you put this. I agree this is where the frontier of software development is right now. You can have LLM chug and bug fix all day long but in the end most real issues are in this layer of intent. I don't think this problem is particular to DB/programming language codebase either - it happens a lot in product design it just doesn't surface in code review. "Everything is working perfectly" yet the resulting workflow or UI is not truly making the user's life easier, or has grown grotesque with additions/modifications.
I will say, if the problem is purely code you occasionally see "sparks of AGI" where the model will step back and say "hold on, there is a massive simplification we can make here...". This started happening with Fable-class models. But rarely if ever will the model push for a huge simplification at the product level.
In my experience it's the opposite - the model's design decisions always try to add complexity, even when it's not necessary.
So you get permission flags and multilayered data structured with complex interfaces, and there is never any sense of "Wait a minute - we don't need this."
It's much like the way AI art looks meaninglessly busy with too many random lines or other elements, and AI writing is always exactly, precisely, specifically, unnecessarily adverbial.
>> In my experience it's the opposite - the model's design decisions always try to add complexity, even when it's not necessary.
This happens if the model does not have good context regarding what is being built, why and for whom.
For one of my side projects, my CLAUDE.md has a line that says "the users of this product will be non-technical people who find technology scary." And you know what? Claude always steers me away from complexity when working on that project.
This is why code is going to be automated, and the software engineer is a link between organizational intent and the code. This is what I’m working on.
Finding the holes in your specs under automation: time spent is returned 10x
Writing code: Human 15 lines per minute. LLM: 600+ lines per minute
Functional Validation: same time per iteration. Fewer bugs with LLM when properly specified => fewer iterations. Increased awareness of LLM escaping specification boundaries feeds back into increasing quality of spec writing => fewer bugs.
The difference is that instead of spending half your time understanding the business requirements and half your time writing code, you'll spend half your time understanding the business requirements, 5% of your time typing these requirements into a chatbot, and the rest of your time doing... ????
Some of the rest of the time can be spent making it look like your ratios are the same but in reality you just take the saved time and do whatever you want with it. The decision making and reviews aren’t happening any faster anyway so just take it easy and coast a bit
I guarantee there will not actually be saved time - the bullshit will increase to more than compensate, as everyone actually fixing things will be drowned out by bullshitters just pretending to fix things and creating more actual problems.
You nailed it! I often got bug reports from AI that would touch far more code to fix than what I am willing to touch, and the likelihood of that large piece of new code (which I honestly would not be able to review thoroughly) will likely contain more bugs than the one I am fixing.
So in the end I often ignore these bug reports after evaluating their actual impact on the product.
And imagine you had an LLM ingest the entire corpus of comments and discussion threads for all of Postgres' history, and asked it to make a judgment call on the tradeoffs - or even on how to present the tradeoffs for discussion.
How much of its judgment would be driven by the data vs. its own internal knowledge? How much might be driven by old vs. new discussions that might not represent new directions of the project? How much might be driven by over-matching against how bugs in less mission-critical code paths were handled?
There's no way to quantify the answers to these questions. Which is not to say this applies to every problem out there! But human judgment is still a bottleneck for design questions on widely, disparately-used complex systems.
ETA: This is extremely generous to the LLMs, btw. If I got an LLM to pop something this concise out I would fully commit to finding a different profession tomorrow
Many of us have already experienced that uncomfortable revelation, where a leading-edge LLM actually comments correctly on our questionable judgement. You will, too, soon enough.
The reply with "many of us..." is so dismissive that it's hard to believe you understand what I'm saying. You act as if I haven't been using these models on the daily for real work since the beginning. Note the word "concise". Note the rest of what I wrote after it, namely the part about how there is often a bunch of accretion of prompts that leads to any output resembling the one I posted.
You seem more concerned with shining your AI badge than understanding what I wrote.
QA is becoming the blocker for projects in an AI era, and the real source of value. AIs can lay down a good base of testing but I see even frontier models still need help with testing because they are basically as bad as humans in terms of testing only what comes to mind, if not even a touch worse. Though they are really good at blasting through problems once they are found.
Ironically, despite the predictions that AI will murder SaaS, I think the result is they end up even more entrenched, because Job Bob's Bait Shack And Tax Filing Services may be happy to instantiate some front office system for themselves via AI prompt, but they aren't going to want to QA that post-AI any more than they want to now. Once we all settle down, SaaS will manage more code than before, and do more QA, and Joe Bob's etc. will be even less interested in trying to replicate the work with AI because it will represent an even larger QA load than it does today, relative to the size of their business.
Or, to put it in the context of this article, even if you can prompt a database into being today, which you can, you can't prompt something into existence that has been tested and QA'd like Postgres, and you would be well advised to ponder that gap between your prompt and billions of computer hours of field testing very carefully before you decide where to store your company's most important data.
> Ironically, despite the predictions that AI will murder SaaS, I think the result is they end up even more entrenched, because Job Bob's Bait Shack And Tax Filing Services
OK, so this is slightly different since it's more about how runtime behavior than design-time behavior, but... I'm in a SaaS workplace, and I've been trying to seed the following idea:
1. The very cheapest/fastest end of the market will be eaten up by Job Bob's Bait Shack stuff. Cheap wrappers around a prompt that goes "Pretend you're a computer that obeys the following rules" jank. A race to the bottom we can't win.
2. Most of our (desirable) customers will (eventually) realize they can't tolerate systems which casually or unexpectedly violate known business rules. That leads to angry consumers, junk data, broken workflows, and lawsuits.
3. When our product can capture and enforce those rules with confidence, that's both an important differentiator and also a kind of moat/stickiness.
4. Therefore there is a strategic business justification for Engineering to fence-off where we use LLMs in production. When we use them for design, it's to create better deterministic systems than before.
The last point is one I make often. AI cannot be trusted to be deterministic, but they can produce deterministic tools that are verifiable as such. Those tools I can trust as much as any other complex tool from the pre-AI era.
I agree that it makes little sense for all but the largest customers to write their own software. Isn't there more of a threat from another software cloning the SaaS and being happy with smaller margin?
I don’t think I’ve worked at a place that wasn’t trying to automate away QA. Perhaps Delivery Hero pre-acquisition, since we had 2,5 QAs for 4 devs in my team.
So… are we gonna use the extra productivity (and money) from coding agents to hire QA people? Oh boy, is that highly unlikely.
But I like your take. I wish it were so, perhaps we wouldn’t have had Tahoe.
But, will that problem remain or is it simply that the models haven't been RLVRd on QA tasks yet? QA feels like a very RL suitable problem. You can take a codebase that has a known set of bugs in it, and proceed to train the model until it identifies those bugs. You can even generate such programs automatically by having adversarial training going on (one model deliberately writes a buggy program, another searches for the bugs). This works even with GUI bugs, drifts from the specs and so on.
I don't see any theoretical reason that this weakness must remain for longer than six months or so. I mean, both Claude and GPT are very good at doing code reviews and that's a form of QA.
The problem will remain due to information theoretic concerns. It is not possible for an AI to read minds. You can say to an AI "build me an accounting system for a small tax firm" and we can, for the sake of argument, even stipulate that it will produce a functional system based strictly off that prompt. However, no matter how smart the AI, the result can not be any more detailed than that. Did you want it for MacOS? It may have built it for windows. Did you want it integrated into an already-existing database server? It can't have done that, and since you didn't specify the database you want it could have guessed wrong, and certainly can't guess about what tables you may already have you want to integrate into. It probably didn't guess your jurisdiction correctly (unless it was fed that in some other contextual prompt). It can't guess your most common work flows and optimize them. It can't pick out specialized features you want just for your special use case.
None of that is because AIs aren't good. It's just plain information.
An AI could sit there and interrogate the user for this information. In that alone, it would already have exceeded what a lot of people are going to want from a system like this. People, normal people, want AIs to magic stuff into existence, not spend hours interrogating them for a design requirements elicitation. That's a barrier a lot of people will not cross, and the AI companies, who fully realize that, aren't building that way.
Normal people do not want to even do the work it takes to fully specify a problem, even if an AI walks them right through it, let alone the verification that the work is correct, and actually what they intended rather than what they asked for.
And of course, in practice, AIs are not emitting bug-free code, miss entire requirements at a shot, and have a variety of other characteristic failure modes in this space that I expect are fundamental to the LLM architecture and aren't ever going to leave. If they continue to be able to make bigger systems they will simply continue to make correspondingly big errors and omissions in the process. No comment as to what the next generation AI architecture will do, though.
Not to mention the blast radius of any small mistake in payment processing, tax filing, or any other financial or regulatory issue, is just so massive compared to making some ai slop bar special flyer. Joe's bait shack is still just going to pay for square.
There will be a ton of smaller Joe’s bait shacks, all competing against each other in a race to the bottom - and with relatively limited blast radius per shop.
This is one angle of the productivity story. You don't have any sort of plausible deniability covering all these bugs, test cases etc. when they are comprehensively discoverable by tooling. It's like having typing bugs on a strongly typed language with static analysis, you just can't make a lot of excuses.
So that's one measure of extra time that will be spent on every single feature, where in the past things would've been rolled out in a more barebones state. It's better overall, but is it going to be perceived as being more productive?
Not in my experience. I've pointed agents like Claude at existing codebases I work on, and it does a pretty solid job at identifying bugs. It's not garbage or slop at all.
There's a big difference between randos getting agents to hallucinate bug reports on GitHub, vs. when your own internal workflow of LLM-assisted code review is identifying real, legitimate problems in your new features.
This totally matches my experience. It's actually kind of insane how well LLM's like Claude are able to find edge cases, race conditions, overflows, states you didn't think could be reached. It really makes you think about code quality in a whole new way. Becoming aware of so many potential bugs when you write the code, vs. finding out from users over the coming months/years -- it forces you to really consider the costs of quality all at once in a way that simply wasn't possible before.
There’s plenty of things you can very deterministically verify — if the LLM writes a test case which causes an ASan violation, you probably have a bug; if the LLM writes a test case which run the various corruption checking tools and they find corruption then you either have corruption or you have a bug in the corruption checking tool; etc.
There’s plenty of ways in which you can guide LLMs, and sandbox them, to get a much higher hit rate.
And that’s before the increase of validity of bugs compared with a year ago — we’re past the peak of slopageddon, and maybe even at a point where a higher percentage of security bugs reported are valid than was the case pre-LLMs.
Opus continues to tell me that my API is missing a `/ready` route that a smoke test can check against. I assume it’s part of its memories, but it’s now 3 or 4 weeks that it won’t let go of that whenever tasked with test design.
I remember an LWN article on this question a few weeks ago. The debate was going on and they weren’t sure if it was that there were more bugs or just that because of AI they suddenly knew about more bugs than usual.
Either way, unfortunate but a very understandable decision.
I'm inclined to think the latter - all sorts of software, particularly long-standing open source Unix software written in C, is riddled with bugs that sometimes don't get discovered until years later. Most ordinary Unix system software like Postgres is written in such a way that it's not possible to be confident that the code is free of subtle bugs, and we should assume that it is not free of subtle bugs.
Seeing how Fable handled Linux kernel bugs, where out of 82 bugs reported only 6-8 were actually bugs (not already fixed or fixed in newest version or straight up made up) I'm skeptical till humans actually sift through this pile.
Not surprising. I've been running AI reviews through old codebases and finding all sorts of interesting things. AI is significantly raising the quality bar and there is a lot of backfilling to do.
I have a unique perspective on this as I've been working on rewriting and rearchitecting Postgres to solve a lot of the long standing issues Postgres has[0][1].
If I had to root cause why we're finding so many bugs now, I would attribute it to the limited Postgres test suite. AI is merely uncovering bugs that already existed. For the most part, Postgres relies on individual regression tests to test each feature. While regression tests are great for making sure features don't break (yes, SELECT 'Hello, World' LIKE '%Hello%' still returns true), they actually aren't that great hitting the edge cases (passing a string that's over 2MB to a specific function will trigger a buffer overflow and crash your DB). The main way Postgres has prevented bugs in the past has been by minimizing risky changes and having lots of human review. As we're learning now, there were lots of gaps that were not caught by the existing process.
While the Postgres team is skeptical of AI, I've had lots of success in using AI to scale more advanced forms of testing such as Deterministic Simulation Testing, differential coverage guided fuzzing, and formal verification. To put it in perspective, in the process of doing testing for pgrust, I've uncovered ~50 different bugs in core Postgres. The bugs I'm finding aren't through AI review, but are instead through using AI to do more advanced testing than regression testing. All of the techniques I'm using existed existed pre-AI, but aren't used in meaningful capacity as part of the core Postgres development.
Right now I'm going all in on formal verification. I wrote a compiler to translate C and Rust functions to Lean and am then having an LLM write a proof that the two functions are identical. So far I've been able to verify 10.5% of the codebases behave identically (10.5% = 67k/640k LOC of Postgres). I'm finding details as minute as a parsing function where Postgres trimmed spaces using isspace() and pgrust only trimmed ascii whitespaces. These aren't cases you would find with traditional regression testing or human review. The thoroughness of the types of bugs I'm finding is giving me confidence that that this is catching more bugs than any sort of manual testing could. I don't currently see any reason that I won't be able to scale up the formal verification to cover 100% of the codebase. The only issue is it takes a long time to do.
A lot of this is downstream of C not having a proper widely accepted unit test framework. There's no JUnit equivalent for C, in terms of sheer hegemonic adoption, nor any of the techniques that were developed to make code testable (DI, mocking libs, etc).
> What does "rebuild" mean? Translation from C to Rust?
We translated from C to Rust, rearchitected several components and are working on rearchitecting more. For instance, we replaced the process-per-connection model with a thread-per-connection model and added a columnar format + new query executor that's hundreds of times faster than Postgres for OLAP workloads.
Good! We need to delay all software releases until security audits of the code by AI/fuzzers/etc. Otherwise people ship buggy software, the hackers find the exploits, and we're compromised as soon as we upgrade. This is why we absolutely must have un-nerfed security models. And if you want to stop having your personal information leaked by compromised companies, we need to make this process mandatory.
I feel the pain. We have been delaying release because sooo many bug reports have come in. AI is finding a lot of obscure, and interesting, and annoying bugs.
Doesn't that just imply the existing version of Postgres also probably has more bugs than humans can fix? Just release it. Software is never perfect and always a work in progress. I'm guessing the bulk of these bugs are probably stupid edge cases that just get development bogged down trying to plug so many holes at once.
So why aren't they sounding the alarm on the obviously deficient postgres 18 that is currently obviously wreaking havoc on the earth, having not gone through any AI code review? Pull the plug already, right?
Indeed, but I for one very much appreciate Postgres’ stability and quality.
As a lead contributor explains in the article, if they were to ship some of these broken features, they wouldn’t be able to fix them until v20, rather than patching them on-the-go. Sounds like a Very Bad Idea given many installations stay behind for some years.
I tried that this summer. I encountered this doom loop where the review agent would find bugs, the coding agent would fix them in some overengineered way that introduced new bugs, the review agent found them, the coding agent made more while fixing them, repeat until insanity.
I've been building a side project end-to-end with agents, and it's really not true for complex software.
At a micro scale their code is right, but tons of invented complexity no sane person would introduce. I've found the two most common triggers are the models polluting their own context windows (poorly worded instructions in comments/issues/AGENTS.md) and a poor "taste" for breaking changes (they'll constantly try to make their additions non-breaking, and if prompted they can make breaking changes, it has to be very carefully scoped or they start assigning great reward to ripping out things)
Same experience. It's especially bad with Claude and anything with non-code text, where it will happily write up endless Claude-ese comments that break any rules in the repo and are constantly incorrect in small ways, catch them in the review, rewrite specific lines in ways that are still wrong despite still having the review in its context, and repeat twenty times in a row.
That is how you end up with a test suite that now takes 10x as long, unless you occasionally prompt the agent to consolidate and group tests.
There are still a lot of issues with coding that require intervention. Just recently, I had Astra detect a minor bug in an HTTP client that resulted in it deciding that the best approach would be to spend 1800 LOC on implementing its own from scratch. Some other time, Opus 5.5 was trying to tell me that test runs on a small to mid-sized application would take another three weeks until they would be fully finished.
Y'know, I had a teammate who had been claiming this up until it became clear that the horrific spike in production defects that we were struggling with was mostly attributable to them.
I'm not going to say that's always what happens. But I will say that you can't find problems you aren't watching for.
> But I will say that you can't find problems you aren't watching for
I can't, but Claude almost regularly comes across "BTW things". I catch them because I read the lead agent's transcript, and have worker agents always add a retro section to their report, so I'm almost always seeing attention-worthy things pop up.
But think of Claude as being like a particularly keen outsourced developer: great at the tactical stuff, but sometimes lacks common sense in weird and surprising ways, so you’ve still got to keep an eye on the work product.
That's not my experience. Agent usually follows the pattern correctly by creating a relevant test and then making it pass. Where strange tests are more likely to fall in is when agent writes the test after the testee.
Yes, it adds tests and makes them pass. But it adds tests for things that no human would ever add. I regularly wind up removing tests where it is guarding dumb crap like "facts" in markdown files, or "drift" across files. Today I came across a file where it was walking ASTs across multiple languages to ensure that there was no drift. I get why it'd do that, but simple changes become absurd.
Sounds like better guidance needed. You have to always keep in mind that the agent doesn't understand the repo from the "business" perspective, only from the functional. You have to give it the why, and ensure a shared understanding. Whenever I plan epics with an agent I always check on acceptance criteria before the tickets hit the tracker, so I know it gets the business aspect. Tends to lead to far better code.
Yes, I understand that. And yet I keep opening up repos written by other people that have insane tests. And sometimes I even come across people who managed to notice it but thought it was a good thing because "moar tests == moar good"
Just because *you* can avoid it doesn't mean it's not an issue with the tooling.
Nah that's not a tooling issue (although - more - mitigation tools can always be built). That's a people issue. And it still may not be their fault per se, like an accountant might have read in a random programming blog that tests are good, so when they decide to "vibe code an award-winning accounting tool", they're going to stick a "ensure there are lots of tests" into that prompt.
I do. My experience has been that most people don't. And then the ridiculous tests become blockers for the agent as they believe the tests are enforcing "truth", which may not be correct post-evolution.
Anecdata, but most of the time when something doesn't smell right to me looking at genAI code, the root cause is ridiculous tests that made their way into the repo. And then the agents built up around it.
The coding loop and the review loop are orthogonal and probably should remain so. Not least because it's probably worth the money for more intelligent models for code reviews where it's a more focused task while coding loops are all over the place.
Weirdly enough I've found the opposite, using too smart a review model gets you overfit instead of benefit. The coding side can be more complex in a way that benefits from fable/astra, but the review side sometimes not so much.
This is the happy path for LLM assisted coding. Obviously consumes less tokens being more constrained and focused so it's not what gets the most marketing but having the second loop of code reviews seems worth investment even if the coding loop is entirely manual or you're in full LLM autopilot mode
I have seen people discussing the fact that AI can literally find an infinite number of bugs. There are always more bugs to be found, whether they would have ever affected any actual real world case or been noticed or not.
What makes you think that the number of bugs is finite?
There are alternatives to that possibility. A perfect program without bugs might be like a perfect definition of the concept of "chair". Whatever definition you come up with there will likely be edges cases that makes the definition awkward or incorrect, and any definition that takes into account all those cases will not be a particularly good or useful definition.
> whether they would have ever affected any actual real world case or been noticed or not.
That's what I was thinking as well. Wouldn't the majority of the bugs not also have existed in Postgres 18? The version that everyone is using atm...
There is a similar thing with AI detecting vulnerabilities. It's not that the vulnerability isn't real, but just that in most cases for these vulnerabilities to exist there also needs to be a very specific set of circumstances as well making the likelihood of the vulnerability to exist in the wild quite low.
Training data is filled with shortcuts, conveniences, bugs, hacks, hacks that fix other hacks, hot fixes, patches, mappings that map the hack to hack to a better solution and all types of other surprises.
That's the cost of trying to scale so quickly. You dont filter out the crap.
A nice side effect is that the best way to fix it is more AI.
From the article: "He was also at pains to point out that the reverts were not down to 'vibe coding' in development, as the technique was not available when most of the new features were written."
I wonder when people will finally feel comfortable not putting vibe coding in scare quotes. Maybe when it accounts for 50% of all new software, instead of only 10%.
Why read the article when you can read the headline and make up your own?
I think these type of new techniques will ultimately create more jobs for humans. If we think the answer is just “AI” everyone will create their own Postgres with a lot more bugs and vulnerabilities.
"Explaining the number of reverts — features whose rollouts have now been postponed to next year's release, PostgreSQL 20 — Peter Eisentraut, veep at EDB, a PostgreSQL support and consultancy, said the reason could be at least in part attributed to the LLM code reviews used by developers."
They have been doing Postgres for a long time, and I believe they have been significant contributors from before AI coding.
Their marketing has to mention AI. Everyone has to in a business like that. I would not take it as evidence they are slop coders or hyping AI just on that basis.
Some conjecture on my part, but I imagine that the bugs being found are not the kind you can just press a button on the LLM to fix anyway, but are instead the sort of bugs which require the reviewer to deeply consider design tradeoffs, of which I believe there are simply more in a DB or programming language codebase, for example. Stuff I am finding in review is often like "this thing isn't like the other thing, which has this behavior, which is wrong if the aim is X, but sort of correct if the aim is Y, and totally perfect is the goal is Z and the use case is J, but not I. Unless it's Q, in which case you probably need to reconsider some fundamental decision making from 12 months ago and have some conversations with people about how users desire it to work in the first place, and then set about understanding the tradeoffs inherent to that design."
ETA: This is extremely generous to the LLMs, btw. If I got an LLM to pop something this concise out I would fully commit to finding a different profession tomorrow, but for now I am for better or worse stuck doing prompting some non-zero number of times in order to get all of this assembled context, and then need to use my human powers to decide whether or not more humans indeed need to discuss it.
I like the way you put this. I agree this is where the frontier of software development is right now. You can have LLM chug and bug fix all day long but in the end most real issues are in this layer of intent. I don't think this problem is particular to DB/programming language codebase either - it happens a lot in product design it just doesn't surface in code review. "Everything is working perfectly" yet the resulting workflow or UI is not truly making the user's life easier, or has grown grotesque with additions/modifications.
I will say, if the problem is purely code you occasionally see "sparks of AGI" where the model will step back and say "hold on, there is a massive simplification we can make here...". This started happening with Fable-class models. But rarely if ever will the model push for a huge simplification at the product level.
In my experience it's the opposite - the model's design decisions always try to add complexity, even when it's not necessary.
So you get permission flags and multilayered data structured with complex interfaces, and there is never any sense of "Wait a minute - we don't need this."
It's much like the way AI art looks meaninglessly busy with too many random lines or other elements, and AI writing is always exactly, precisely, specifically, unnecessarily adverbial.
>> In my experience it's the opposite - the model's design decisions always try to add complexity, even when it's not necessary.
This happens if the model does not have good context regarding what is being built, why and for whom.
For one of my side projects, my CLAUDE.md has a line that says "the users of this product will be non-technical people who find technology scary." And you know what? Claude always steers me away from complexity when working on that project.
[flagged]
Yes > real issues are in this layer of intent
This is why code is going to be automated, and the software engineer is a link between organizational intent and the code. This is what I’m working on.
How does this differ from what I'm doing today? Want to say any more on the topic? This is a bit vague.
Design, writing specs: Same amount of time.
Finding the holes in your specs under automation: time spent is returned 10x
Writing code: Human 15 lines per minute. LLM: 600+ lines per minute
Functional Validation: same time per iteration. Fewer bugs with LLM when properly specified => fewer iterations. Increased awareness of LLM escaping specification boundaries feeds back into increasing quality of spec writing => fewer bugs.
The difference is that instead of spending half your time understanding the business requirements and half your time writing code, you'll spend half your time understanding the business requirements, 5% of your time typing these requirements into a chatbot, and the rest of your time doing... ????
Some of the rest of the time can be spent making it look like your ratios are the same but in reality you just take the saved time and do whatever you want with it. The decision making and reviews aren’t happening any faster anyway so just take it easy and coast a bit
I guarantee there will not actually be saved time - the bullshit will increase to more than compensate, as everyone actually fixing things will be drowned out by bullshitters just pretending to fix things and creating more actual problems.
You nailed it! I often got bug reports from AI that would touch far more code to fix than what I am willing to touch, and the likelihood of that large piece of new code (which I honestly would not be able to review thoroughly) will likely contain more bugs than the one I am fixing.
So in the end I often ignore these bug reports after evaluating their actual impact on the product.
And imagine you had an LLM ingest the entire corpus of comments and discussion threads for all of Postgres' history, and asked it to make a judgment call on the tradeoffs - or even on how to present the tradeoffs for discussion.
How much of its judgment would be driven by the data vs. its own internal knowledge? How much might be driven by old vs. new discussions that might not represent new directions of the project? How much might be driven by over-matching against how bugs in less mission-critical code paths were handled?
There's no way to quantify the answers to these questions. Which is not to say this applies to every problem out there! But human judgment is still a bottleneck for design questions on widely, disparately-used complex systems.
ETA: This is extremely generous to the LLMs, btw. If I got an LLM to pop something this concise out I would fully commit to finding a different profession tomorrow
Many of us have already experienced that uncomfortable revelation, where a leading-edge LLM actually comments correctly on our questionable judgement. You will, too, soon enough.
I don’t think you followed the point being made.
If I got an LLM to pop something this concise out I would fully commit to finding a different profession tomorrow
Help me out. What'd I miss?
In my case, rather than finding a different profession, I chose to walk through the open door to the next level of the same profession.
The reply with "many of us..." is so dismissive that it's hard to believe you understand what I'm saying. You act as if I haven't been using these models on the daily for real work since the beginning. Note the word "concise". Note the rest of what I wrote after it, namely the part about how there is often a bunch of accretion of prompts that leads to any output resembling the one I posted.
You seem more concerned with shining your AI badge than understanding what I wrote.
I see, thanks for the clarification.
QA is becoming the blocker for projects in an AI era, and the real source of value. AIs can lay down a good base of testing but I see even frontier models still need help with testing because they are basically as bad as humans in terms of testing only what comes to mind, if not even a touch worse. Though they are really good at blasting through problems once they are found.
Ironically, despite the predictions that AI will murder SaaS, I think the result is they end up even more entrenched, because Job Bob's Bait Shack And Tax Filing Services may be happy to instantiate some front office system for themselves via AI prompt, but they aren't going to want to QA that post-AI any more than they want to now. Once we all settle down, SaaS will manage more code than before, and do more QA, and Joe Bob's etc. will be even less interested in trying to replicate the work with AI because it will represent an even larger QA load than it does today, relative to the size of their business.
Or, to put it in the context of this article, even if you can prompt a database into being today, which you can, you can't prompt something into existence that has been tested and QA'd like Postgres, and you would be well advised to ponder that gap between your prompt and billions of computer hours of field testing very carefully before you decide where to store your company's most important data.
> Ironically, despite the predictions that AI will murder SaaS, I think the result is they end up even more entrenched, because Job Bob's Bait Shack And Tax Filing Services
OK, so this is slightly different since it's more about how runtime behavior than design-time behavior, but... I'm in a SaaS workplace, and I've been trying to seed the following idea:
1. The very cheapest/fastest end of the market will be eaten up by Job Bob's Bait Shack stuff. Cheap wrappers around a prompt that goes "Pretend you're a computer that obeys the following rules" jank. A race to the bottom we can't win.
2. Most of our (desirable) customers will (eventually) realize they can't tolerate systems which casually or unexpectedly violate known business rules. That leads to angry consumers, junk data, broken workflows, and lawsuits.
3. When our product can capture and enforce those rules with confidence, that's both an important differentiator and also a kind of moat/stickiness.
4. Therefore there is a strategic business justification for Engineering to fence-off where we use LLMs in production. When we use them for design, it's to create better deterministic systems than before.
The last point is one I make often. AI cannot be trusted to be deterministic, but they can produce deterministic tools that are verifiable as such. Those tools I can trust as much as any other complex tool from the pre-AI era.
> pre-AI
I'd say pre-LLM, just because I don't want to surrender the term "AI" to one particular subtype without a fight.
Some of those complex tools would be "Good Old-Fashioned AI", which generally means stuff doing symbolic logic rather than mystery-weights.
I agree that it makes little sense for all but the largest customers to write their own software. Isn't there more of a threat from another software cloning the SaaS and being happy with smaller margin?
I don’t think I’ve worked at a place that wasn’t trying to automate away QA. Perhaps Delivery Hero pre-acquisition, since we had 2,5 QAs for 4 devs in my team.
So… are we gonna use the extra productivity (and money) from coding agents to hire QA people? Oh boy, is that highly unlikely.
But I like your take. I wish it were so, perhaps we wouldn’t have had Tahoe.
But, will that problem remain or is it simply that the models haven't been RLVRd on QA tasks yet? QA feels like a very RL suitable problem. You can take a codebase that has a known set of bugs in it, and proceed to train the model until it identifies those bugs. You can even generate such programs automatically by having adversarial training going on (one model deliberately writes a buggy program, another searches for the bugs). This works even with GUI bugs, drifts from the specs and so on.
I don't see any theoretical reason that this weakness must remain for longer than six months or so. I mean, both Claude and GPT are very good at doing code reviews and that's a form of QA.
The problem will remain due to information theoretic concerns. It is not possible for an AI to read minds. You can say to an AI "build me an accounting system for a small tax firm" and we can, for the sake of argument, even stipulate that it will produce a functional system based strictly off that prompt. However, no matter how smart the AI, the result can not be any more detailed than that. Did you want it for MacOS? It may have built it for windows. Did you want it integrated into an already-existing database server? It can't have done that, and since you didn't specify the database you want it could have guessed wrong, and certainly can't guess about what tables you may already have you want to integrate into. It probably didn't guess your jurisdiction correctly (unless it was fed that in some other contextual prompt). It can't guess your most common work flows and optimize them. It can't pick out specialized features you want just for your special use case.
None of that is because AIs aren't good. It's just plain information.
An AI could sit there and interrogate the user for this information. In that alone, it would already have exceeded what a lot of people are going to want from a system like this. People, normal people, want AIs to magic stuff into existence, not spend hours interrogating them for a design requirements elicitation. That's a barrier a lot of people will not cross, and the AI companies, who fully realize that, aren't building that way.
Normal people do not want to even do the work it takes to fully specify a problem, even if an AI walks them right through it, let alone the verification that the work is correct, and actually what they intended rather than what they asked for.
And of course, in practice, AIs are not emitting bug-free code, miss entire requirements at a shot, and have a variety of other characteristic failure modes in this space that I expect are fundamental to the LLM architecture and aren't ever going to leave. If they continue to be able to make bigger systems they will simply continue to make correspondingly big errors and omissions in the process. No comment as to what the next generation AI architecture will do, though.
Not to mention the blast radius of any small mistake in payment processing, tax filing, or any other financial or regulatory issue, is just so massive compared to making some ai slop bar special flyer. Joe's bait shack is still just going to pay for square.
There will be a ton of smaller Joe’s bait shacks, all competing against each other in a race to the bottom - and with relatively limited blast radius per shop.
This is one angle of the productivity story. You don't have any sort of plausible deniability covering all these bugs, test cases etc. when they are comprehensively discoverable by tooling. It's like having typing bugs on a strongly typed language with static analysis, you just can't make a lot of excuses.
So that's one measure of extra time that will be spent on every single feature, where in the past things would've been rolled out in a more barebones state. It's better overall, but is it going to be perceived as being more productive?
If its more "bugs than humans can fix", its also likely "more bugs than humans can triage and verify as legitimate bugs".
Sorry, garbage in garbage out doesnt fly. I feel sad for open source maintainers having to deal with this massive slop-ageddon.
Not in my experience. I've pointed agents like Claude at existing codebases I work on, and it does a pretty solid job at identifying bugs. It's not garbage or slop at all.
There's a big difference between randos getting agents to hallucinate bug reports on GitHub, vs. when your own internal workflow of LLM-assisted code review is identifying real, legitimate problems in your new features.
This totally matches my experience. It's actually kind of insane how well LLM's like Claude are able to find edge cases, race conditions, overflows, states you didn't think could be reached. It really makes you think about code quality in a whole new way. Becoming aware of so many potential bugs when you write the code, vs. finding out from users over the coming months/years -- it forces you to really consider the costs of quality all at once in a way that simply wasn't possible before.
There’s plenty of things you can very deterministically verify — if the LLM writes a test case which causes an ASan violation, you probably have a bug; if the LLM writes a test case which run the various corruption checking tools and they find corruption then you either have corruption or you have a bug in the corruption checking tool; etc.
There’s plenty of ways in which you can guide LLMs, and sandbox them, to get a much higher hit rate.
And that’s before the increase of validity of bugs compared with a year ago — we’re past the peak of slopageddon, and maybe even at a point where a higher percentage of security bugs reported are valid than was the case pre-LLMs.
More bugs than humans can fix, but not more bugs than humans created.
More hypothetical bugs, perhaps?
Opus continues to tell me that my API is missing a `/ready` route that a smoke test can check against. I assume it’s part of its memories, but it’s now 3 or 4 weeks that it won’t let go of that whenever tasked with test design.
[flagged]
"What even 'IS' a bug really?" HN topics from 2036
I remember an LWN article on this question a few weeks ago. The debate was going on and they weren’t sure if it was that there were more bugs or just that because of AI they suddenly knew about more bugs than usual.
Either way, unfortunate but a very understandable decision.
I'm inclined to think the latter - all sorts of software, particularly long-standing open source Unix software written in C, is riddled with bugs that sometimes don't get discovered until years later. Most ordinary Unix system software like Postgres is written in such a way that it's not possible to be confident that the code is free of subtle bugs, and we should assume that it is not free of subtle bugs.
Yeah, I don't understand how this can even be a question when we're finding an unprecedented number of bugs that are decades old.
I think the concern was the bugs in the new features specifically but still I suspect you’re right it’s just that AI finding them “early”.
https://lwn.net/Articles/1092003/
Determinable for a code base, with enough tokens.
Seeing how Fable handled Linux kernel bugs, where out of 82 bugs reported only 6-8 were actually bugs (not already fixed or fixed in newest version or straight up made up) I'm skeptical till humans actually sift through this pile.
Not surprising. I've been running AI reviews through old codebases and finding all sorts of interesting things. AI is significantly raising the quality bar and there is a lot of backfilling to do.
I have a unique perspective on this as I've been working on rewriting and rearchitecting Postgres to solve a lot of the long standing issues Postgres has[0][1].
If I had to root cause why we're finding so many bugs now, I would attribute it to the limited Postgres test suite. AI is merely uncovering bugs that already existed. For the most part, Postgres relies on individual regression tests to test each feature. While regression tests are great for making sure features don't break (yes, SELECT 'Hello, World' LIKE '%Hello%' still returns true), they actually aren't that great hitting the edge cases (passing a string that's over 2MB to a specific function will trigger a buffer overflow and crash your DB). The main way Postgres has prevented bugs in the past has been by minimizing risky changes and having lots of human review. As we're learning now, there were lots of gaps that were not caught by the existing process.
While the Postgres team is skeptical of AI, I've had lots of success in using AI to scale more advanced forms of testing such as Deterministic Simulation Testing, differential coverage guided fuzzing, and formal verification. To put it in perspective, in the process of doing testing for pgrust, I've uncovered ~50 different bugs in core Postgres. The bugs I'm finding aren't through AI review, but are instead through using AI to do more advanced testing than regression testing. All of the techniques I'm using existed existed pre-AI, but aren't used in meaningful capacity as part of the core Postgres development.
Right now I'm going all in on formal verification. I wrote a compiler to translate C and Rust functions to Lean and am then having an LLM write a proof that the two functions are identical. So far I've been able to verify 10.5% of the codebases behave identically (10.5% = 67k/640k LOC of Postgres). I'm finding details as minute as a parsing function where Postgres trimmed spaces using isspace() and pgrust only trimmed ascii whitespaces. These aren't cases you would find with traditional regression testing or human review. The thoroughness of the types of bugs I'm finding is giving me confidence that that this is catching more bugs than any sort of manual testing could. I don't currently see any reason that I won't be able to scale up the formal verification to cover 100% of the codebase. The only issue is it takes a long time to do.
[0] https://github.com/malisper/pgrust
[1] https://malisper.me/the-four-horsemen-behind-thousands-of-po...
A lot of this is downstream of C not having a proper widely accepted unit test framework. There's no JUnit equivalent for C, in terms of sheer hegemonic adoption, nor any of the techniques that were developed to make code testable (DI, mocking libs, etc).
What does "rebuild" mean? Translation from C to Rust?
You look at Postgres C functions and prove they are identical, but the license is AGPL without mentioning Postgres authors. Consider me confused.
> What does "rebuild" mean? Translation from C to Rust?
We translated from C to Rust, rearchitected several components and are working on rearchitecting more. For instance, we replaced the process-per-connection model with a thread-per-connection model and added a columnar format + new query executor that's hundreds of times faster than Postgres for OLAP workloads.
> without mentioning Postgres authors
We have a NOTICE file which references the Postgres license here: https://github.com/malisper/pgrust/blob/main/NOTICE
Wow, super cool. Are you the PG Rust guy? Very impressive what you're doing.
Yep - I've been doing the vast majority of the development of pgrust
Good! We need to delay all software releases until security audits of the code by AI/fuzzers/etc. Otherwise people ship buggy software, the hackers find the exploits, and we're compromised as soon as we upgrade. This is why we absolutely must have un-nerfed security models. And if you want to stop having your personal information leaked by compromised companies, we need to make this process mandatory.
I feel the pain. We have been delaying release because sooo many bug reports have come in. AI is finding a lot of obscure, and interesting, and annoying bugs.
Doesn't that just imply the existing version of Postgres also probably has more bugs than humans can fix? Just release it. Software is never perfect and always a work in progress. I'm guessing the bulk of these bugs are probably stupid edge cases that just get development bogged down trying to plug so many holes at once.
> Software is never perfect and always a work in progress.
Relational databases are one of those things where you really do want to get a whole lot closer to perfect than a lot of other software.
So why aren't they sounding the alarm on the obviously deficient postgres 18 that is currently obviously wreaking havoc on the earth, having not gone through any AI code review? Pull the plug already, right?
Indeed, but I for one very much appreciate Postgres’ stability and quality.
As a lead contributor explains in the article, if they were to ship some of these broken features, they wouldn’t be able to fix them until v20, rather than patching them on-the-go. Sounds like a Very Bad Idea given many installations stay behind for some years.
[flagged]
I guess with LLM as the reviewer, it is going to take a lot longer to be Zero Defect, unless they use LLM as the coder, too.
I tried that this summer. I encountered this doom loop where the review agent would find bugs, the coding agent would fix them in some overengineered way that introduced new bugs, the review agent found them, the coding agent made more while fixing them, repeat until insanity.
Not every time. But often enough.
FWIW gpt-6.1-sol and opus 5.5 are much, much less prone to this, if you include some coaching around complexity, YAGNI, that kind of thing
I've been building a side project end-to-end with agents, and it's really not true for complex software.
At a micro scale their code is right, but tons of invented complexity no sane person would introduce. I've found the two most common triggers are the models polluting their own context windows (poorly worded instructions in comments/issues/AGENTS.md) and a poor "taste" for breaking changes (they'll constantly try to make their additions non-breaking, and if prompted they can make breaking changes, it has to be very carefully scoped or they start assigning great reward to ripping out things)
Same experience. It's especially bad with Claude and anything with non-code text, where it will happily write up endless Claude-ese comments that break any rules in the repo and are constantly incorrect in small ways, catch them in the review, rewrite specific lines in ways that are still wrong despite still having the review in its context, and repeat twenty times in a row.
Yeah, you got to reinforce through prompting (which isn't dead!) at multi touch-points to make narrow bug fixes and not touch unrelated code.
And as others have said some models people prefer for implementation work are less prone to this.. 5.6 Sol, 6.1 Sol, Opus 5.5
Agents understand red/green/refactor. Tell them to follow it, and use a coverage tool.
That is how you end up with a test suite that now takes 10x as long, unless you occasionally prompt the agent to consolidate and group tests.
There are still a lot of issues with coding that require intervention. Just recently, I had Astra detect a minor bug in an HTTP client that resulted in it deciding that the best approach would be to spend 1800 LOC on implementing its own from scratch. Some other time, Opus 5.5 was trying to tell me that test runs on a small to mid-sized application would take another three weeks until they would be fully finished.
The only time I really get tests which take crazy time is when I'm doing mobile dev, because emulator testing on my system is slow.
Y'know, I had a teammate who had been claiming this up until it became clear that the horrific spike in production defects that we were struggling with was mostly attributable to them.
I'm not going to say that's always what happens. But I will say that you can't find problems you aren't watching for.
> But I will say that you can't find problems you aren't watching for
I can't, but Claude almost regularly comes across "BTW things". I catch them because I read the lead agent's transcript, and have worker agents always add a retro section to their report, so I'm almost always seeing attention-worthy things pop up.
And I’m definitely not saying don’t do that.
But think of Claude as being like a particularly keen outsourced developer: great at the tactical stuff, but sometimes lacks common sense in weird and surprising ways, so you’ve still got to keep an eye on the work product.
And then you discover it’s added tests for insane things that shouldn’t be getting tests
That's not my experience. Agent usually follows the pattern correctly by creating a relevant test and then making it pass. Where strange tests are more likely to fall in is when agent writes the test after the testee.
Yes, it adds tests and makes them pass. But it adds tests for things that no human would ever add. I regularly wind up removing tests where it is guarding dumb crap like "facts" in markdown files, or "drift" across files. Today I came across a file where it was walking ASTs across multiple languages to ensure that there was no drift. I get why it'd do that, but simple changes become absurd.
Sounds like better guidance needed. You have to always keep in mind that the agent doesn't understand the repo from the "business" perspective, only from the functional. You have to give it the why, and ensure a shared understanding. Whenever I plan epics with an agent I always check on acceptance criteria before the tickets hit the tracker, so I know it gets the business aspect. Tends to lead to far better code.
Yes, I understand that. And yet I keep opening up repos written by other people that have insane tests. And sometimes I even come across people who managed to notice it but thought it was a good thing because "moar tests == moar good"
Just because *you* can avoid it doesn't mean it's not an issue with the tooling.
Nah that's not a tooling issue (although - more - mitigation tools can always be built). That's a people issue. And it still may not be their fault per se, like an accountant might have read in a random programming blog that tests are good, so when they decide to "vibe code an award-winning accounting tool", they're going to stick a "ensure there are lots of tests" into that prompt.
Then you remove them and move on with your life? We're still software developers at the end of the day, my delete key works better than ever.
Also maybe I'm a curmudgeon but I generally at least mark-one-eyeball all the tests it's generating and tell it when it's being extra dumb.
I do. My experience has been that most people don't. And then the ridiculous tests become blockers for the agent as they believe the tests are enforcing "truth", which may not be correct post-evolution.
Anecdata, but most of the time when something doesn't smell right to me looking at genAI code, the root cause is ridiculous tests that made their way into the repo. And then the agents built up around it.
The coding loop and the review loop are orthogonal and probably should remain so. Not least because it's probably worth the money for more intelligent models for code reviews where it's a more focused task while coding loops are all over the place.
Weirdly enough I've found the opposite, using too smart a review model gets you overfit instead of benefit. The coding side can be more complex in a way that benefits from fable/astra, but the review side sometimes not so much.
This is the happy path for LLM assisted coding. Obviously consumes less tokens being more constrained and focused so it's not what gets the most marketing but having the second loop of code reviews seems worth investment even if the coding loop is entirely manual or you're in full LLM autopilot mode
I find this very inspiring - the forecast of bug free software seems a little more likely when I read this kind of news.
Using AI to find bugs (rather than write them) is a great use case IMO. Even if you have skilled reviewers, more eyes are almost always better.
Just don’t post the unedited AI “bug report” as an issue please. A human should make sure it’s actually a real problem before dumping it on devs.
I have seen people discussing the fact that AI can literally find an infinite number of bugs. There are always more bugs to be found, whether they would have ever affected any actual real world case or been noticed or not.
> There are always more bugs to be found
A common misunderstanding.
The # of bugs lurking in a codebase is finite. At some point, the only way to find more is introduce fresh bugs through new code.
What we're seeing now is a transition phase where pre-existing bugs are found (and hopefully fixed) at an increased rate.
What makes you think that the number of bugs is finite? There are alternatives to that possibility. A perfect program without bugs might be like a perfect definition of the concept of "chair". Whatever definition you come up with there will likely be edges cases that makes the definition awkward or incorrect, and any definition that takes into account all those cases will not be a particularly good or useful definition.
> whether they would have ever affected any actual real world case or been noticed or not.
That's what I was thinking as well. Wouldn't the majority of the bugs not also have existed in Postgres 18? The version that everyone is using atm...
There is a similar thing with AI detecting vulnerabilities. It's not that the vulnerability isn't real, but just that in most cases for these vulnerabilities to exist there also needs to be a very specific set of circumstances as well making the likelihood of the vulnerability to exist in the wild quite low.
That is obviously not true, and seems like a strange thing to say.
[flagged]
[flagged]
[dead]
[flagged]
[flagged]
[dead]
[flagged]
[flagged]
Training data is filled with shortcuts, conveniences, bugs, hacks, hacks that fix other hacks, hot fixes, patches, mappings that map the hack to hack to a better solution and all types of other surprises.
That's the cost of trying to scale so quickly. You dont filter out the crap.
A nice side effect is that the best way to fix it is more AI.
It's a endless money loop.
From the article: "He was also at pains to point out that the reverts were not down to 'vibe coding' in development, as the technique was not available when most of the new features were written."
I wonder when people will finally feel comfortable not putting vibe coding in scare quotes. Maybe when it accounts for 50% of all new software, instead of only 10%.
Journalists are in general slower to do this kind of thing than technologists.
The article mentioned the exact opposite.
Why read the article when you can read the headline and make up your own?
I think these type of new techniques will ultimately create more jobs for humans. If we think the answer is just “AI” everyone will create their own Postgres with a lot more bugs and vulnerabilities.
[flagged]
From the article:
"Explaining the number of reverts — features whose rollouts have now been postponed to next year's release, PostgreSQL 20 — Peter Eisentraut, veep at EDB, a PostgreSQL support and consultancy, said the reason could be at least in part attributed to the LLM code reviews used by developers."
Naturally, EDB is an AI shop (https://www.enterprisedb.com):
"Agents need direct, governed access to data at the source. EDB Postgres AI is built for that."
Of course, nothing ever has worked in PostgreSQL before AI!
They have been doing Postgres for a long time, and I believe they have been significant contributors from before AI coding.
Their marketing has to mention AI. Everyone has to in a business like that. I would not take it as evidence they are slop coders or hyping AI just on that basis.
If by "they" you mean Eisentraut, I know who he is. That is precisely why I'm disappointed.