While I agree with the premise, I think this angle only applies on work one was going to do no matter what. The real power of these tools is that there are so many ideas people would like to try, but never have the time or motivation to pursue.
So the comparison is not only "built with and without LLM" but "would you even build this if you didn't have the LLM?". The gap in productivity in this case is much more wide.
I think the ideas worth pursuing still require substantial time and motivation, even with LLMs. Otherwise you just end up with a half-baked sloppy artifact that nobody will ever use or study, for a net negative benefit to humanity.
Yes, I work in hardware design ans the cost to code up the RTL for an idea and compare it to the existing code in terms of PPA, its very expensive in terms of engineering time. This week I had 5 different architecites coded, pushed through a functional test bench, bit accurate models created and then run through ppa analysis all done via llms. So its a task tbis project could not afford to do without llms, but can save us 10% in ppa
Those sort of ideas are unlikely to yield major benefit, though.
This is basically replicating the plight of the solo open source dev, writ large. Individual programmers have long built the thing they've cared about on their own time (essentially "for free" because, despite kindergarten economics theory, a programmer cannot usually monetize a marginal hour). And it usually goes that the project never gets adopted anywhere. It might acrue more features and total man-hour effort than most of what FAANG does in open source to drown out the solo devs. But the market will decide that "no organizational buy-in" is a signal the project doesn't matter. Other developers will decide, "if he could do it, so could I" and also not adopt.
Same exact thing is happening and will continue to happen with all these generated "but we wouldn't have done it otherwise" projects. It's just very, very unlikely to go anywhere.
That which took very little effort to create will receive very little effort to promote.
This reminds me of themes I recently saw in [Harness Engineering is not Enough: Why Software Factories Fail](https://www.youtube.com/watch?v=Ib5GBkD555M) (Warning: the last 3 slides seem like an advertisement). One thing I liked is how Dex has a little graphic he glossed over showing software development is
- 25% planning & aligning with other teams
- 25% coding
- 25% testing/verifying
- 25% code review/rework
One argument was that agentic coding speeds up that coding part a bunch. So maybe there's 2x speedup in coding. But that's only a small speedup in the totality of everything software engineers do.
AI is great because my job is a lot easier, but it produces some extremely crappy code that would've NEVER passed a code review 1 year ago.
You could argue that it speeds up development by 5X or more, but then it slows down testing/verifying, code review, and in many cases it makes it impossible to review/rework by hand.
I just do what I'm told at work but even though I'm sure I'm in the minority I'm extremely skeptical that LLMs can produce any good-quality code.
Looks to me like we've just lowered the bar--by a lot--and stopped looking at the code that goes out. Every time I look into how Claude implemented something it's completely insane, with no way to refactor it or maintain it in the future.
I'd say that thanks to LLM assistance I'm 10x faster at coding, code review/rework, and testing/verifying. (LLMs can partially automate testing/verifying too, and the code is higher quality now as well so less testing/verifying is necessary).
So that leaves us with:
- 92.5% planning & aligning with other teams
- 2.5% coding
- 2.5% testing/verifying
- 2.5% code review/rework
Obviously, that makes zero sense as a split. If you saw any organization doing that, you'd suggest having fewer teams, more silos, etc. Maybe you have designers produce code, instead of showing the designs to coders and having the coders implement it. Maybe you force all your engineers to dogfood the product that way they can identify issues themselves rather than needing QA teams to do it. And so on. so the last category, "planning & aligning with other teams", falls too.
1) because they’re not doing those others spheres of work
2) because they don’t think about the work
nor how tired they are afterwards
3) they’re proselytizing AI work as the future and
that conflicts with that vision
4) the other shoe hasn’t dropped
5) they really don’t see it
6) some people genuinely hate programming and this
helps them skip that.
The other hard to measure part is that I see/hear a lot of LLM usage going towards dev work that was never prioritized before.
Suddenly devs who were cranking out features with no interest in infrastructure are attacking giant refactors to make the code more understandable to the LLM.
Other devs are using LLMs to build themselves quality of life SDLC tools completely separate from the core code base.
The quality of my bash scripts is 100x what they used to be. Proper help messages, flag parsing, functions, internal variables, using sed all the time (I could never wrap my head around that).
Of course, the main issue is that they’re completely undebbugable now. My bash scripts used to be a sequential list of commands, now they’re 500 lines of variable laden functions.
Is my life any better? Dunno. But it’s satisfying (until there’s a bug)
Not necessarily…. As with all engineering choices there are tradeoffs involved. There can be both advantages and disadvantages to using AI to build scripts. And more likely than not you’ll need AI to help debug the scripts when they have issues. Real tradeoffs to consider.
Just one more data point but this is what I’m seeing too. The extreme ‘ruthless’ prioritization has stopped and devs (who want it) have breathing room to make the enhancements they’ve been wanting all along. Turns out what is good for AI is also good for regular old school SDLC. All you have to do is say it will make AI more effective.
I use CC to do the testing and verifying in multiple deployment environments. It’s faster at things like coming up a plausible test payload or ten, querying a bunch of metrics from datadog, querying and verifying state from a db before and after testing, scanning all the deployed logs for relevant errors. For me that has been more of an uplift than the coding. I still feel like communication it’s worst at. So writing tickets, getting a decision out of a group, documenting only the important things, etc. it’s still not that good at
Agentic tools changed the workflow in our org, we were very tense before with a simple yet careful team process.. Since last winter we're now pushing a lot more but the teamwork (which was brittle before) is now mostly gone, everybody can roll on its own, but the review process didn't scale and now monthly deliveries are full of "seems to work". nobody checks properly, nobody reviews, nobody tighten any bolts.. and nobody cares much anyway, the system allows it.
The way I see it you should calibrate the way you work with LLMs based on how confident you are on that specific area, and if it's your responsibility to own/understand it. Here's how it feels for me:
* Learning stage: 0.5x - 1x. I change my system prompt to teacher mode, taking the productivity hit for actually learning the system/tool pays off dividends later. I change my system prompt to "teacher mode" and slowly loosen it as I get more confident.
* Working-knowledge: 2x - 3x. Once I am ramped up enough I feel like I can get a decent productivity boost. Most of the time is spent at the planning stage. This is my mode for areas I don't really own or care about, just need to get work done.
* Mastered: 10x+ I have been doing web front end for 12+ years, I can quickly review plan/implementations and for my initial prompt I already know most of what I want built.
This really needs to be calibrated to the type of work and complexity.
I can actually believe that LLMs would speed up basic web dev work in small, simple codebases 10X for simple requests.
These conversations usually turn into people talking past each other because they’re working on different things. For other less routine and more complex work, expecting a 10X productivity boost is not realistic at all. It doesn’t matter how good you get at writing prompts and reviewing plans. LLMs just don’t solve everything for you in a good way. Some times the true nature of the problem is revealed while implementing it and by deferring everything to an LLM you spend days throwing tokens at the wrong thing. There is a lot of work where the LLM speed up comes from helping you quickly search docs and codebases and double check your code, but handing the entire thing off to an LLM isn’t reasonable. These tasks aren’t going to reach this mythical 10X productivity boost that is genuinely achievable for much simpler work.
> Some times the true nature of the problem is revealed while implementing it and by deferring everything to an LLM you spend days throwing tokens at the wrong thing.
If you look at the code the LLM spat out (and you really, really should!) you will immediately notice that its shape is not what you thought it should be. You might not notice immediately if you're learning, but if you really "mastered" the domain, you will "just" see it. You will also recognize the problem with your assumptions, and immediately (or after some research) correct the prompt.
Looking at the diffs for everything the LLM does slows you down, of course. Not looking - or looking and not recognizing problems, for one reason or another - can be initially faster, but a single pathological case can eat both the time saving and tokens. For domains you truly "mastered", the current models can generate code as if they read your mind (because you can be that precise in the prompt, and quickly), so it's really glaring and very hard to miss when they somehow misread your mind.
It only works at the "mastered" / "unconscious competence" stages, and only in those narrow domains you truly mastered, but it does seem to work. Is it 10x? No idea, but there is a marked change in the speed boost when crossing from conscious to unconscious competence area, with everything else (harness, model) staying the same.
Some domains don't have the established regularity of architecture that it takes to achieve 10X efficiency. Two I can think of off the top of my head are embedded systems, with a variety of sensors, outputs, processing power, and memory, and novel protocols like ATProto, for which training data is thin on the ground.
Someone posted their AGENTS.md some time ago, that is used in practice to teach their students[1]. That might be useful to you in this context as well.
I'm not sure about the x, but the first thing that arises from that is, I feel like in my case it's way higher than 2.
The 2nd thing is, how do I measure that.
---
In my case, the details of my work (Kinda DevOps, kinda Senior Dev) makes it that having an LLM to do the heavy lifting allows me to do things not only faster, but better, and across domains I do not hold expertise on.
An example of the effect of LLMs in my daily work is that I'm in the middle of a PHP upgrade for a rather large legacy application, and the "heavy lifting" is really out of the scale, letting me concentrate on what really matters, while at the same time if I do keep "the harness" tight I'm certain the results are the correct ones. Also correcting course is just as cheap.
Not having to worry on the tooling on exchange has the incredible desirable result of my velocity being incomparable to what it was before.
Then we have the side effect of how easy to do transfer knowledge: Rather than telling the QA guy how to do the work specific for this task, I defined a set of files (.md documents, skills, an off-the-shelf customised MCP server) that assist QA into doing the work in a way that helps me do my job better and faster.
There's also a clear possibility that what I'm doing will expand to the rest of the team I am in, completely altering the way in which we approach development.
If we take 'x' as 'mileage', yours might vary. Mine has, and I'm baffled at the positive net results I AM getting.
Also, this what I do (coding?) is extremely fun again.
Being able to work close to the speed of thought is the best high.
Because it produces the desired output. The purpose of a program.
There are many domains where an intelligent human can act as a discriminator for output without knowing exactly in precise detail how the process itself works.
I won't argue this title isn't the author's experience, but it's misleading to rest a quantitative claim on a qualitative argument, while also refusing to fully exploit the technology.
From a firm that has fully embraced agentic engineering, they offered topline stats on their measurements of a 4x boost from November to February, and 8x on top from February to May. That's 32x since November 2025, coding with LLMs.
And like whatever, this is second hand and I'm not going to disclose the source. Actual hard, published, and peer-reviewed research is needed here to backup quantitative claims. 2x, 10x, 32x, whatever.
When I approach a new area or task I first explore it, by hand, and implement, by hand, the task.
This way, I get a deeper understanding and it gives me some momentum to llm the rest of the related tasks.
Till I loose enough grip to "re-dive in" that area and to again, do stuff my hand. Some tasks, I only do by hand, and some tasks, I only use llm for.
Really depends on the task, my the project and on, and honesty, my instrest and availability at the moment.
Does the number × depend on how big your thinking is? If you are only 2× then you need to think bigger, aim higher, etc. I think the limit on the 2× is not the ai? Where is the limit now? I think it's unknown.
Definitely feels like a lot more than 2x to me. Any time it feels like the agent is taking a long time I have to look in the mirror and say to myself "What could you have done in 5 minutes?"
I think LLMS have reached to a point where now its just about money. you have to decide whether you really want to spend your tokens to do X or you can just do it yourself and use the tokens for something else.
You still have to know what you’re doing. That came from years of doing it by hand. Where is that necessary experience going to come from for the current and future generations of juniors?
I mean the "x amount" is relative to your existing skill level, right?
It's worth asking ourselves "why does the x amount matter?" I get the desire to define KPIs to estimate productivity gains. But this all seems to be—rather rapidly—leading to an increasingly dehumanized reality (both figuratively and literally) so we can...produce more software? I love building software, and I enjoy using LLMs to help me do it, but there's just this weird vibe I can't quite shake about how we're trying to quantify all of this.
> As for documentation, I've found this simple instruction to vastly improve LLMs' output:
> Never write READMEs, docstrings, or comments. I will write those myself later. And yes, I really mean this.
This is quite validating as I came to the exact same conclusion myself. We’re required to use an LLM for every task at work that touches code†, and I was really struggling to get Claude to stop with the long waffly comments that reiterate the next few lines of code in 3x as many characters, making contextless references to subtasks in whatever harness du jour we’re using this week.
No amount of examples or explanation of what I wanted would make it stop. And then I realised of course, I’m asking something which has no concept of meaning (or, indeed, anything) to only add meaningful comments. More fool me I guess.
Of course, it’s ultimately pointless given all of my colleagues are regularly opening PRs with more comments than code anyway. 80% of my code review responses these days are just increasingly exasperated “pointless comment, please remove”.
† This is just as infantilising as it sounds, by the way
+10, I can't stand AI-generated comments. Half the time they are paragraph-length translations of every 5 lines of code into English prose, or just regurgitating whatever prompt they were given. When they actually try making high level comments they are worse than useless. It's really hard to put into words exactly why. It's something about how they are very verbose, but at the same time vague and stated in a slightly-off manner. Or they will introduce new terminology not reflected in the code's nomenclature, sometimes terms that are just synonyms for terms that the code already uses. Something about it makes me instantly distrustful of any information the comment may actually convey.
locally correct and globally incoherent is how they seem to me. I, and a number of folks I have talked to, find some of them incomprehensible. Like each sentence makes sense, but the overall thing does not.
It's because they don't really have a mental model. RL with Verifiable Rewards can reward working code, but it can't tell you whether the prose make sense or not. In theory RLHF should be good with this. In practice, it appears to be a measure difficult to quantify and steer. I am not an insider though. So it's just a theory.
I have the same impression. It is an interesting phenomenon. Is it one of the ways we will gradually learn that AI in the form of LLMs is fundamentally lacking?
I use AI all the time, it gives a tremendous boost in speed for many tasks and it is most certainly here to stay, but there is something off about certain things, like code comments. Like a weird person.
I kind of wander if the comments are intentionally bad, providing programmers something to bike shed about. A sneakier approach to the flattery the models used to so use so heavily.
You can create a PostToolUse hook [0] which will automatically chastise Claude to shrink comments to one line. It's not perfect but better than the sea of prose it tends to generate by default. Bonus is that it would apply to your colleagues as well!
I have the opposite problem with Code, it barely writes any comments even for fairly complex C++ code blocks (systems programming stuff), have to prompt it to add extra details about why it did something. Request in agents.md doesn't seem to help much.
I'm in the same boat. We have very little documentation where I work, and one of my goals was to improve that as I got onboard. We have a newer dev that started ripping +3K, +5K, +10K line PRs against our documentation repository.
I have tried to shut it down, it immediately nullifies the docs and make it genuine slop.
I truly believe that documentation should be human written and human readable. It's all of the stuff that AI does NOT pick up on. Small inconsistencies, only the necessary details. IE: Don't just rewrite the code that we have into readable english (We are software developers after all. We can read code.) - comments / documentation to me has always been the "in between the lines" stuff that can't be put into the code.
I spent 30min with Opus 5 generating some CSS and components for my Phoenix LiveView template project (which I use for starting new projects). Then I spent around 5 hours parsing and cleaning up the output. I could of course not have done that, but since it's my spare time project I can care about the code and quality.
I did get wins for sure (and I did save time), and I'm sure most people wouldn't even spend five minutes cleaning up, but this is definitely one place where people talk about different things regarding whatever x speedup they get.
This is a dated take to me. I think the next stage in unlocking productivity is so called "loop engineering"; figuring out how to effectively not read all the code while ensuring quality. To me that means implementing statistical quality control and formal methods.
Before I get there I have to figure out how to reliably audit plan adherence. The problem is that when the specs are in natural language, as they are, you need a fallible LLM to verify it.
I agree that removing human from the loop is the next stage, but we're not there yet beyond small programs.
Agents left alone tend to create so much tech debt, that once the program becomes so messy that they can't fix one bug without creating two new bugs, it's too late to even clean that up.
The program will be super tidy in superficial aspects that linters catch (everything neatly formatted and verbosely commented), and roughly appear to do what it's supposed to do, but everything in between will be "I can't even".
We need something else than formal methods, because the problem is usually in lack of simplicity - you get four versions of the problem solved in four times in four in different ways, each uniquely flawed and just incompatible enough with the others that unifying them is too big and hairy for the agent, and will result in eight different glue adapters written in the process.
This will be a deeply unpopular opinion, but I have an 80k line Go project I've created where I haven't read the source, but would absolutely stake its quality against most hand-written projects of a similar size[0].
Recent LLMs struggle with the author's stated issues only inline: they're entirely capable of going back and evaluating codebases to find architectural issues and LLM slop signatures, especially when you use other models to check one model's output. This wasn't true until Fable-class models, but it's true now.
People are still figuring out how to use LLMs/AI and considering how feelings based and hard is to reproduce many of the results people get, a lot of the improved approaches are basically at "trust me bro" reliability.
TBH i expect it'll take several years before people largely settle on what works good and what works bad (or doesn't work) and that's assuming things wont chance in a massive way during that time.
And this isn't really anything new, people rarely come up with some new way of programming and have everyone learn and understand best uses from the start - it takes years (and that is assuming everyone will agree that'd be something worth bothering about).
I found that latest codes don't write comments in code by default. And when they do, they write stupid shit like "This was code that did X, it was now removed".
You have to explcitly prompt them to write comments in code. They are still useful for you, the user. But are arguably useful for the model, too, given how many of them (especially Claude) only reads small chunks of files. So I'd rather have code comments than it reproducing a picture from incomplete data.
A small mention in your user-wide Claude.md (or Agents.md or whatever you use) about the desired types of comments goes a long way. The typical "why, not what", clarifying larger-picture stuff, and everything else you find in every book on code quality
... the official return of pair programming! The best part: LLM's save you from having to argue with a human. Added bonus: now you get to master two tools instead of one.
>As such, I use LLMs mainly to produce a rough draft of the code which I then iterate on heavily, at least until I like the general structure
Tell me you just started with agentic programming without telling me you just started using agentic programming.
Look, don't get me wrong: new folks learning tech should absolutely write articles about it! But their claims might very well change once they learn more .. and I strongly suspect that will be the case here.
Personally i'm only using local LLMs that i can run on my 7 year old PC (that also has a GPU with 24GB VRAM because reasons :-P) so i'm not sure how much that experience matches what others are doing (though roughly speaking what i see people complain about Claude doing doesn't feel that different from what i see the local stuff doing, so i guess the drawbacks aren't scaled down as model sizes increase).
I'm not sure about 1x, 2x or 10x increase as these metrics are about code written but that isn't a productivity metric (something pretty much every half-decent programmer would agree with before LLMs - remember stories about Bill Gates saying that more LoCs being good for software is like more weight is good for airplanes or Bill Atkinson's story about adding -2000 LoCs to improve QuickDraw?).
But they can certainly help "get you going" faster in that if you're stuck on something (for whatever reason - including "that feels too much drudgery") or have issues starting something, you can have an LLM take a stab at it and it'll produce "something". Sometimes it is enough by itself, but more often than not it'll need tweaks (either directly or having the LLM do it). I got to make a bunch of things i couldn't convince myself to do - e.g. an image viewer that doesn't suck (based on my arbitrary judgement), a game database, a script to convert a git repository into static html pages that kinda look like GitHub, etc.
They can also help find (and sometimes fix) bugs or other "code smell" issues. They're not that great for exact results (without tool calling -and knowledge on how to use them effectively- at least) but when it comes to fuzzy / vague stuff like "check out this code <code dump here> can you spot any issues?" they always tend to find some stuff (even if it is hallucinations :-P but sometimes they find actual issues too or whatever hallucination they come up with reveals some actual issues with the code that you didn't spot by yourself). I've been dabbling with Rust recently and asked Qwen 3.6 35B-A3B to judge my code and it wrote "6.5/10, will compile but looks like C in Rust" :-P.
The article says:
> Never write READMEs, docstrings, or comments. I will write those myself later. And yes, I really mean this.
And sure, LLMs aren't that great about those (i do let them leave whatever comments they want though and remove them later myself - i think those comments help during the generation/prediction - basically how they "think", kinda like the reasoning phase), but they can be very good at things like "here is the code, here is the documentation for it, spot discrepancies" (i had Devstral Small 2 do this and it hallucinated a few discrepancies but also found real stuff i missed in the docs).
I've tried to use Devstral Small 2 for some API docs but found it'd sometimes make assumptions about what function do or how. One approach that might work, but i haven't tried yet, is to write the "guide" myself, then have the LLM write the function docs using both the guide and the code as reference. The reason i think this will work is because it got things 95% correct just having access to the function code alone (and without the rest of the codebase), so the "guide" would help it reach 99%. I do not expect it to get to 100% so a manual edit pass will need to be done anyway (and i have an idea for a tool to assist in the manual edit pass for that - a tool that i'll probably get an LLM to write - BTW good luck coming up if all that stuff would increase or decrease productivity for an actual product and not some random stuff i'm toying with :-P).
One other thing i've also found LLMs useful recently is to have them use the stuff you make and see how they try to use it. I have an old project, a GUI toolkit i've been hacking on every now and then since 2011 or so, though it was never a priority. Yesterday i decided to dump all the header files to Qwen 35B-A3B (i use 4bit quantization that gives me a 256k context - it isn't particularly smart but it is neat to not have to micromanage context size like i have to do with Devstral Small 2 or Qwen 27B where both of them aren't very usable speedwise at anything above 32k context sizes).
Then i asked it to just make a few small programs and it did[0] (the shot shows a paint app, a calendar, a todo list and a unit converter). Pretty much every program found bugs in the library :-P and gave me ideas on how to improve things.
In general i get the impression that LLMs aren't great at architecting things but if you do the architecture yourself and write them a framework to use, they should do a fine job at it.
I already convert from multipliers to percentage of increase, so when someone claims 10x they very likely mean +100% productivity, and here 2x means +20% productivity, which seems about right. Nobody that was normally productive before LLMs has suddenly 10x'ed their output now.
The problem is that 20% productivity when it comes to generating code, really doesn't translate in 20% productivity increase overall, when you take into account the fact that the code quality is worse, the fact that writing code is actually not the majority of your time spent, and that people get burnt out from the usage.
I know, and I'm saying that 2x claims should be interpreted as an actual 20% increase, and 10x claims as an actual 100% increase. People are just bad at estimating their own productivity.
I've found a variant of the Gell-Mann Amnesia effect with regards to LLM coding. They're really magical if you're not familiar with the language, domain, or frameworks you are trying to use. But if you're familiar with those things then you'll often catch them lying to your face and producing a lot of plausible nonsense.
My observation is that AI tools generally make people /less/ productive. They are more productive in that they produce more stuff, but they are less productive in that the stuff they are producing has lower inherent value delivery. The primary place where I see this in software teams as a Product person is in document creation. If you are using an AI tool to produce internal documents, this is likely a net-negative value activity that actually reduces the overall productivity of the team. Why? Because writing is thinking. By handing off document creation to an LLM for a document that's supposed to communicate important details between teams (e.g. technical design docs, requirements docs, strategy docs) you are actually handing off thinking, or rather handing off not-thinking as AI tools cannot "think". The outcome of is poorly "thought out" slop that generates more work for everyone involved to resolve /or/ everyone simply ignores the document and the previous processes stop being useful.
If anything, I'd say AI tools in /most/ enterprises as people are trying to use them now are at least a -30% of productivity. If used correctly for taking a human-written/thought requirements doc, converting it into an interactive prototype that can be critiqued and ultimately included as part of the solution alignment within the requirements doc, and then is handed to an engineering team that is effective at using humans and AI to produce code, then it's probably a benefit. But most businesses lack the internal rigor, quality culture, and data governance to support properly applying AI tools in a high context manner internal to their business.
Have you worked with codebases where people produced slop even before LLMs? In such codebases LLMs are a godsend. They can churn around huge amounts of code and find needles in the haystack.
If 60x was a thing, we would see many fantastic products appearing on the market, every product would have improved or goten major new interesting features. Yet it hasn't really happened.
10x or 60x of very little is still not much. "Working programmers" unable to write FizzBuzz exist.
10x or 60x of a negative value is a 10-60x bigger problem than that initial negative value. We all regularly commit things that are actually net negative.
Finally, 10x or 60x more productivity channeled into yet another static site generator, CMS, or programming language doesn't really help in improving products or developing new interesting features.
In short: LLMs offer real productivity gains, but they are still tools, and they are very new, so we misuse them a lot, in various ways (not limited to examples above). Unless they replace programmers and can consistently produce better than average code (even if they fail catastrophically in a few cases), they won't fundamentally change programming. They did change a lot, but not enough to get the result you're describing.
I believe I have seen 60x improvements on certain classes of bugs. Just not on development as a whole
And I've also seen the other side, where vibe coding on projects that had bad code quality to start with leads to bug riddled messes. Projects with beautiful and elaborate test harnesses, that break seconds after contact with a real human user
While I agree with the premise, I think this angle only applies on work one was going to do no matter what. The real power of these tools is that there are so many ideas people would like to try, but never have the time or motivation to pursue.
So the comparison is not only "built with and without LLM" but "would you even build this if you didn't have the LLM?". The gap in productivity in this case is much more wide.
I think the ideas worth pursuing still require substantial time and motivation, even with LLMs. Otherwise you just end up with a half-baked sloppy artifact that nobody will ever use or study, for a net negative benefit to humanity.
Yes but I’ve seen some devs waste a lot of time using AI to build something that was a bad idea to begin with.
Without AI they might have first spent more time validating the idea was worth it.
> Without AI they might have first spent more time validating the idea was worth it.
Seems optimistic
Yes, I work in hardware design ans the cost to code up the RTL for an idea and compare it to the existing code in terms of PPA, its very expensive in terms of engineering time. This week I had 5 different architecites coded, pushed through a functional test bench, bit accurate models created and then run through ppa analysis all done via llms. So its a task tbis project could not afford to do without llms, but can save us 10% in ppa
Those sort of ideas are unlikely to yield major benefit, though.
This is basically replicating the plight of the solo open source dev, writ large. Individual programmers have long built the thing they've cared about on their own time (essentially "for free" because, despite kindergarten economics theory, a programmer cannot usually monetize a marginal hour). And it usually goes that the project never gets adopted anywhere. It might acrue more features and total man-hour effort than most of what FAANG does in open source to drown out the solo devs. But the market will decide that "no organizational buy-in" is a signal the project doesn't matter. Other developers will decide, "if he could do it, so could I" and also not adopt.
Same exact thing is happening and will continue to happen with all these generated "but we wouldn't have done it otherwise" projects. It's just very, very unlikely to go anywhere.
That which took very little effort to create will receive very little effort to promote.
Agree so much. So many small bugs, nits, tweaks I just send off to an LLM agent to figure it out.
that and 2x is "i'm 5'10" and i round up to 6'" low.
This reminds me of themes I recently saw in [Harness Engineering is not Enough: Why Software Factories Fail](https://www.youtube.com/watch?v=Ib5GBkD555M) (Warning: the last 3 slides seem like an advertisement). One thing I liked is how Dex has a little graphic he glossed over showing software development is
- 25% planning & aligning with other teams
- 25% coding
- 25% testing/verifying
- 25% code review/rework
One argument was that agentic coding speeds up that coding part a bunch. So maybe there's 2x speedup in coding. But that's only a small speedup in the totality of everything software engineers do.
AI is great because my job is a lot easier, but it produces some extremely crappy code that would've NEVER passed a code review 1 year ago.
You could argue that it speeds up development by 5X or more, but then it slows down testing/verifying, code review, and in many cases it makes it impossible to review/rework by hand.
I just do what I'm told at work but even though I'm sure I'm in the minority I'm extremely skeptical that LLMs can produce any good-quality code.
Looks to me like we've just lowered the bar--by a lot--and stopped looking at the code that goes out. Every time I look into how Claude implemented something it's completely insane, with no way to refactor it or maintain it in the future.
> it produces some extremely crappy code that would've NEVER passed a code review 1 year ago
The 10x speed up comes from the fact that we have all given up on properly reviewing each others code and we just say “meh, it will be fine, lgtm”
> So maybe there's 2x speedup in coding. But that's only a small speedup in the totality of everything software engineers do.
Amdahl's Law should be familiar to anyone with a 4y computer science/engineering degree. Why aren't they applying it to their own throughput?
Let's apply it:
- 25% planning & aligning with other teams
- 25% coding
- 25% testing/verifying
- 25% code review/rework
I'd say that thanks to LLM assistance I'm 10x faster at coding, code review/rework, and testing/verifying. (LLMs can partially automate testing/verifying too, and the code is higher quality now as well so less testing/verifying is necessary).
So that leaves us with:
- 92.5% planning & aligning with other teams
- 2.5% coding
- 2.5% testing/verifying
- 2.5% code review/rework
Obviously, that makes zero sense as a split. If you saw any organization doing that, you'd suggest having fewer teams, more silos, etc. Maybe you have designers produce code, instead of showing the designs to coders and having the coders implement it. Maybe you force all your engineers to dogfood the product that way they can identify issues themselves rather than needing QA teams to do it. And so on. so the last category, "planning & aligning with other teams", falls too.
Well, there’s probably several reasons:
I’m sure others as well.The other hard to measure part is that I see/hear a lot of LLM usage going towards dev work that was never prioritized before.
Suddenly devs who were cranking out features with no interest in infrastructure are attacking giant refactors to make the code more understandable to the LLM.
Other devs are using LLMs to build themselves quality of life SDLC tools completely separate from the core code base.
Plenty of other examples of this.
The quality of my bash scripts is 100x what they used to be. Proper help messages, flag parsing, functions, internal variables, using sed all the time (I could never wrap my head around that).
Of course, the main issue is that they’re completely undebbugable now. My bash scripts used to be a sequential list of commands, now they’re 500 lines of variable laden functions.
Is my life any better? Dunno. But it’s satisfying (until there’s a bug)
> the main issue is that they’re completely undebbugable now
Well, that means the quality actually dropped then :). Looking impressive isn't equal to quality, understandability and reliability is
Not necessarily…. As with all engineering choices there are tradeoffs involved. There can be both advantages and disadvantages to using AI to build scripts. And more likely than not you’ll need AI to help debug the scripts when they have issues. Real tradeoffs to consider.
I switched to js for dev related scripting after seeing how much python scripting is used for portable c++ projects like chromium.
A node glob() or a regexp string.replace call is probably easier to read than spaghetti shell.
And your llm might do a much better job of creating clean, readable and testable code.
Just one more data point but this is what I’m seeing too. The extreme ‘ruthless’ prioritization has stopped and devs (who want it) have breathing room to make the enhancements they’ve been wanting all along. Turns out what is good for AI is also good for regular old school SDLC. All you have to do is say it will make AI more effective.
I use CC to do the testing and verifying in multiple deployment environments. It’s faster at things like coming up a plausible test payload or ten, querying a bunch of metrics from datadog, querying and verifying state from a db before and after testing, scanning all the deployed logs for relevant errors. For me that has been more of an uplift than the coding. I still feel like communication it’s worst at. So writing tickets, getting a decision out of a group, documenting only the important things, etc. it’s still not that good at
Agentic tools changed the workflow in our org, we were very tense before with a simple yet careful team process.. Since last winter we're now pushing a lot more but the teamwork (which was brittle before) is now mostly gone, everybody can roll on its own, but the review process didn't scale and now monthly deliveries are full of "seems to work". nobody checks properly, nobody reviews, nobody tighten any bolts.. and nobody cares much anyway, the system allows it.
The way I see it you should calibrate the way you work with LLMs based on how confident you are on that specific area, and if it's your responsibility to own/understand it. Here's how it feels for me:
* Learning stage: 0.5x - 1x. I change my system prompt to teacher mode, taking the productivity hit for actually learning the system/tool pays off dividends later. I change my system prompt to "teacher mode" and slowly loosen it as I get more confident.
* Working-knowledge: 2x - 3x. Once I am ramped up enough I feel like I can get a decent productivity boost. Most of the time is spent at the planning stage. This is my mode for areas I don't really own or care about, just need to get work done.
* Mastered: 10x+ I have been doing web front end for 12+ years, I can quickly review plan/implementations and for my initial prompt I already know most of what I want built.
1x == my speed before AI
> * Mastered: 10x+
This really needs to be calibrated to the type of work and complexity.
I can actually believe that LLMs would speed up basic web dev work in small, simple codebases 10X for simple requests.
These conversations usually turn into people talking past each other because they’re working on different things. For other less routine and more complex work, expecting a 10X productivity boost is not realistic at all. It doesn’t matter how good you get at writing prompts and reviewing plans. LLMs just don’t solve everything for you in a good way. Some times the true nature of the problem is revealed while implementing it and by deferring everything to an LLM you spend days throwing tokens at the wrong thing. There is a lot of work where the LLM speed up comes from helping you quickly search docs and codebases and double check your code, but handing the entire thing off to an LLM isn’t reasonable. These tasks aren’t going to reach this mythical 10X productivity boost that is genuinely achievable for much simpler work.
> Some times the true nature of the problem is revealed while implementing it and by deferring everything to an LLM you spend days throwing tokens at the wrong thing.
If you look at the code the LLM spat out (and you really, really should!) you will immediately notice that its shape is not what you thought it should be. You might not notice immediately if you're learning, but if you really "mastered" the domain, you will "just" see it. You will also recognize the problem with your assumptions, and immediately (or after some research) correct the prompt.
Looking at the diffs for everything the LLM does slows you down, of course. Not looking - or looking and not recognizing problems, for one reason or another - can be initially faster, but a single pathological case can eat both the time saving and tokens. For domains you truly "mastered", the current models can generate code as if they read your mind (because you can be that precise in the prompt, and quickly), so it's really glaring and very hard to miss when they somehow misread your mind.
It only works at the "mastered" / "unconscious competence" stages, and only in those narrow domains you truly mastered, but it does seem to work. Is it 10x? No idea, but there is a marked change in the speed boost when crossing from conscious to unconscious competence area, with everything else (harness, model) staying the same.
Some domains don't have the established regularity of architecture that it takes to achieve 10X efficiency. Two I can think of off the top of my head are embedded systems, with a variety of sensors, outputs, processing power, and memory, and novel protocols like ATProto, for which training data is thin on the ground.
Hey, could you share the prompt you're using for "teacher mode"?
Someone posted their AGENTS.md some time ago, that is used in practice to teach their students[1]. That might be useful to you in this context as well.
[1]: https://news.ycombinator.com/item?id=48359858
I'm not sure about the x, but the first thing that arises from that is, I feel like in my case it's way higher than 2.
The 2nd thing is, how do I measure that.
---
In my case, the details of my work (Kinda DevOps, kinda Senior Dev) makes it that having an LLM to do the heavy lifting allows me to do things not only faster, but better, and across domains I do not hold expertise on.
An example of the effect of LLMs in my daily work is that I'm in the middle of a PHP upgrade for a rather large legacy application, and the "heavy lifting" is really out of the scale, letting me concentrate on what really matters, while at the same time if I do keep "the harness" tight I'm certain the results are the correct ones. Also correcting course is just as cheap.
Not having to worry on the tooling on exchange has the incredible desirable result of my velocity being incomparable to what it was before.
Then we have the side effect of how easy to do transfer knowledge: Rather than telling the QA guy how to do the work specific for this task, I defined a set of files (.md documents, skills, an off-the-shelf customised MCP server) that assist QA into doing the work in a way that helps me do my job better and faster.
There's also a clear possibility that what I'm doing will expand to the rest of the team I am in, completely altering the way in which we approach development.
If we take 'x' as 'mileage', yours might vary. Mine has, and I'm baffled at the positive net results I AM getting.
Also, this what I do (coding?) is extremely fun again.
Being able to work close to the speed of thought is the best high.
> allows me to do things not only faster, but better, and across domains I do not hold expertise on.
How would you know it's better when you have no expertise?
Because it produces the desired output. The purpose of a program.
There are many domains where an intelligent human can act as a discriminator for output without knowing exactly in precise detail how the process itself works.
I won't argue this title isn't the author's experience, but it's misleading to rest a quantitative claim on a qualitative argument, while also refusing to fully exploit the technology.
From a firm that has fully embraced agentic engineering, they offered topline stats on their measurements of a 4x boost from November to February, and 8x on top from February to May. That's 32x since November 2025, coding with LLMs.
And like whatever, this is second hand and I'm not going to disclose the source. Actual hard, published, and peer-reviewed research is needed here to backup quantitative claims. 2x, 10x, 32x, whatever.
When I approach a new area or task I first explore it, by hand, and implement, by hand, the task. This way, I get a deeper understanding and it gives me some momentum to llm the rest of the related tasks. Till I loose enough grip to "re-dive in" that area and to again, do stuff my hand. Some tasks, I only do by hand, and some tasks, I only use llm for.
Really depends on the task, my the project and on, and honesty, my instrest and availability at the moment.
Does the number × depend on how big your thinking is? If you are only 2× then you need to think bigger, aim higher, etc. I think the limit on the 2× is not the ai? Where is the limit now? I think it's unknown.
Definitely feels like a lot more than 2x to me. Any time it feels like the agent is taking a long time I have to look in the mirror and say to myself "What could you have done in 5 minutes?"
I think LLMS have reached to a point where now its just about money. you have to decide whether you really want to spend your tokens to do X or you can just do it yourself and use the tokens for something else.
You still have to know what you’re doing. That came from years of doing it by hand. Where is that necessary experience going to come from for the current and future generations of juniors?
> Never write READMEs, docstrings, or comments. I will write those myself later. And yes, I really mean this.
The precise and rigorous practice of “engineering” in 2026.
I mean the "x amount" is relative to your existing skill level, right?
It's worth asking ourselves "why does the x amount matter?" I get the desire to define KPIs to estimate productivity gains. But this all seems to be—rather rapidly—leading to an increasingly dehumanized reality (both figuratively and literally) so we can...produce more software? I love building software, and I enjoy using LLMs to help me do it, but there's just this weird vibe I can't quite shake about how we're trying to quantify all of this.
0.25x because I scope bigger now and then I may get a few "free" React components but largely doing all that work of putting it together.
Wouldn't trade it though. Feel like I can overall do more with less time and energy.
At the end of the day, AI is making me work more (good thing). If you count that as productivity, then sure.
> As for documentation, I've found this simple instruction to vastly improve LLMs' output:
> Never write READMEs, docstrings, or comments. I will write those myself later. And yes, I really mean this.
This is quite validating as I came to the exact same conclusion myself. We’re required to use an LLM for every task at work that touches code†, and I was really struggling to get Claude to stop with the long waffly comments that reiterate the next few lines of code in 3x as many characters, making contextless references to subtasks in whatever harness du jour we’re using this week.
No amount of examples or explanation of what I wanted would make it stop. And then I realised of course, I’m asking something which has no concept of meaning (or, indeed, anything) to only add meaningful comments. More fool me I guess.
Of course, it’s ultimately pointless given all of my colleagues are regularly opening PRs with more comments than code anyway. 80% of my code review responses these days are just increasingly exasperated “pointless comment, please remove”.
† This is just as infantilising as it sounds, by the way
+10, I can't stand AI-generated comments. Half the time they are paragraph-length translations of every 5 lines of code into English prose, or just regurgitating whatever prompt they were given. When they actually try making high level comments they are worse than useless. It's really hard to put into words exactly why. It's something about how they are very verbose, but at the same time vague and stated in a slightly-off manner. Or they will introduce new terminology not reflected in the code's nomenclature, sometimes terms that are just synonyms for terms that the code already uses. Something about it makes me instantly distrustful of any information the comment may actually convey.
locally correct and globally incoherent is how they seem to me. I, and a number of folks I have talked to, find some of them incomprehensible. Like each sentence makes sense, but the overall thing does not.
It's because they don't really have a mental model. RL with Verifiable Rewards can reward working code, but it can't tell you whether the prose make sense or not. In theory RLHF should be good with this. In practice, it appears to be a measure difficult to quantify and steer. I am not an insider though. So it's just a theory.
I have the same impression. It is an interesting phenomenon. Is it one of the ways we will gradually learn that AI in the form of LLMs is fundamentally lacking?
I use AI all the time, it gives a tremendous boost in speed for many tasks and it is most certainly here to stay, but there is something off about certain things, like code comments. Like a weird person.
I kind of wander if the comments are intentionally bad, providing programmers something to bike shed about. A sneakier approach to the flattery the models used to so use so heavily.
You can create a PostToolUse hook [0] which will automatically chastise Claude to shrink comments to one line. It's not perfect but better than the sea of prose it tends to generate by default. Bonus is that it would apply to your colleagues as well!
[0] https://github.com/chrisvariety/branch-fiction/blob/deb37f2b...
I have the opposite problem with Code, it barely writes any comments even for fairly complex C++ code blocks (systems programming stuff), have to prompt it to add extra details about why it did something. Request in agents.md doesn't seem to help much.
> We’re required to use an LLM for every task at work that touches code
Why not just write the code in the prompt so LLM can paste it.
I'm in the same boat. We have very little documentation where I work, and one of my goals was to improve that as I got onboard. We have a newer dev that started ripping +3K, +5K, +10K line PRs against our documentation repository.
I have tried to shut it down, it immediately nullifies the docs and make it genuine slop.
I truly believe that documentation should be human written and human readable. It's all of the stuff that AI does NOT pick up on. Small inconsistencies, only the necessary details. IE: Don't just rewrite the code that we have into readable english (We are software developers after all. We can read code.) - comments / documentation to me has always been the "in between the lines" stuff that can't be put into the code.
I spent 30min with Opus 5 generating some CSS and components for my Phoenix LiveView template project (which I use for starting new projects). Then I spent around 5 hours parsing and cleaning up the output. I could of course not have done that, but since it's my spare time project I can care about the code and quality.
I did get wins for sure (and I did save time), and I'm sure most people wouldn't even spend five minutes cleaning up, but this is definitely one place where people talk about different things regarding whatever x speedup they get.
All these hot takes that LLMs are -10x, 100x, 2x are kind of silly. It really depends on:
What you are using it for? What language, what framework, how many LOC, what domain, how much docs can the LLM read, etc etc.
What are you optimizing for? Cost? Human knowing how your code works? Getting something out the door?
How Good You Are at Prompting the LLM for THAT specific set of parameters?
I find AI can save me like 5x time on something, and in other cases it's wasted 5x time. Overtime I'm hopefully learning how to use it better.
This is a dated take to me. I think the next stage in unlocking productivity is so called "loop engineering"; figuring out how to effectively not read all the code while ensuring quality. To me that means implementing statistical quality control and formal methods.
Before I get there I have to figure out how to reliably audit plan adherence. The problem is that when the specs are in natural language, as they are, you need a fallible LLM to verify it.
I agree that removing human from the loop is the next stage, but we're not there yet beyond small programs.
Agents left alone tend to create so much tech debt, that once the program becomes so messy that they can't fix one bug without creating two new bugs, it's too late to even clean that up.
The program will be super tidy in superficial aspects that linters catch (everything neatly formatted and verbosely commented), and roughly appear to do what it's supposed to do, but everything in between will be "I can't even".
We need something else than formal methods, because the problem is usually in lack of simplicity - you get four versions of the problem solved in four times in four in different ways, each uniquely flawed and just incompatible enough with the others that unifying them is too big and hairy for the agent, and will result in eight different glue adapters written in the process.
This will be a deeply unpopular opinion, but I have an 80k line Go project I've created where I haven't read the source, but would absolutely stake its quality against most hand-written projects of a similar size[0].
Recent LLMs struggle with the author's stated issues only inline: they're entirely capable of going back and evaluating codebases to find architectural issues and LLM slop signatures, especially when you use other models to check one model's output. This wasn't true until Fable-class models, but it's true now.
0: https://github.com/pjlsergeant/byre
> As such, I use LLMs mainly to produce a rough draft of the code which I then iterate on heavily, at least until I like the general structure.
Weird… I would have said this was how it was about 2 years ago, but no way in 2026.
So many of the AI takes posted to HN are outdated.
People are still figuring out how to use LLMs/AI and considering how feelings based and hard is to reproduce many of the results people get, a lot of the improved approaches are basically at "trust me bro" reliability.
TBH i expect it'll take several years before people largely settle on what works good and what works bad (or doesn't work) and that's assuming things wont chance in a massive way during that time.
And this isn't really anything new, people rarely come up with some new way of programming and have everyone learn and understand best uses from the start - it takes years (and that is assuming everyone will agree that'd be something worth bothering about).
> Never write READMEs, docstrings, or comments.
I found that latest codes don't write comments in code by default. And when they do, they write stupid shit like "This was code that did X, it was now removed".
You have to explcitly prompt them to write comments in code. They are still useful for you, the user. But are arguably useful for the model, too, given how many of them (especially Claude) only reads small chunks of files. So I'd rather have code comments than it reproducing a picture from incomplete data.
A small mention in your user-wide Claude.md (or Agents.md or whatever you use) about the desired types of comments goes a long way. The typical "why, not what", clarifying larger-picture stuff, and everything else you find in every book on code quality
200x coding with LLM
... the official return of pair programming! The best part: LLM's save you from having to argue with a human. Added bonus: now you get to master two tools instead of one.
>As such, I use LLMs mainly to produce a rough draft of the code which I then iterate on heavily, at least until I like the general structure
Tell me you just started with agentic programming without telling me you just started using agentic programming.
Look, don't get me wrong: new folks learning tech should absolutely write articles about it! But their claims might very well change once they learn more .. and I strongly suspect that will be the case here.
Tell me you're bad at software engineering without telling me you're bad at software engineering.
Tell me you're bad at people skills without telling me you're bad at people skills
Such is life.
Personally i'm only using local LLMs that i can run on my 7 year old PC (that also has a GPU with 24GB VRAM because reasons :-P) so i'm not sure how much that experience matches what others are doing (though roughly speaking what i see people complain about Claude doing doesn't feel that different from what i see the local stuff doing, so i guess the drawbacks aren't scaled down as model sizes increase).
I'm not sure about 1x, 2x or 10x increase as these metrics are about code written but that isn't a productivity metric (something pretty much every half-decent programmer would agree with before LLMs - remember stories about Bill Gates saying that more LoCs being good for software is like more weight is good for airplanes or Bill Atkinson's story about adding -2000 LoCs to improve QuickDraw?).
But they can certainly help "get you going" faster in that if you're stuck on something (for whatever reason - including "that feels too much drudgery") or have issues starting something, you can have an LLM take a stab at it and it'll produce "something". Sometimes it is enough by itself, but more often than not it'll need tweaks (either directly or having the LLM do it). I got to make a bunch of things i couldn't convince myself to do - e.g. an image viewer that doesn't suck (based on my arbitrary judgement), a game database, a script to convert a git repository into static html pages that kinda look like GitHub, etc.
They can also help find (and sometimes fix) bugs or other "code smell" issues. They're not that great for exact results (without tool calling -and knowledge on how to use them effectively- at least) but when it comes to fuzzy / vague stuff like "check out this code <code dump here> can you spot any issues?" they always tend to find some stuff (even if it is hallucinations :-P but sometimes they find actual issues too or whatever hallucination they come up with reveals some actual issues with the code that you didn't spot by yourself). I've been dabbling with Rust recently and asked Qwen 3.6 35B-A3B to judge my code and it wrote "6.5/10, will compile but looks like C in Rust" :-P.
The article says:
> Never write READMEs, docstrings, or comments. I will write those myself later. And yes, I really mean this.
And sure, LLMs aren't that great about those (i do let them leave whatever comments they want though and remove them later myself - i think those comments help during the generation/prediction - basically how they "think", kinda like the reasoning phase), but they can be very good at things like "here is the code, here is the documentation for it, spot discrepancies" (i had Devstral Small 2 do this and it hallucinated a few discrepancies but also found real stuff i missed in the docs).
I've tried to use Devstral Small 2 for some API docs but found it'd sometimes make assumptions about what function do or how. One approach that might work, but i haven't tried yet, is to write the "guide" myself, then have the LLM write the function docs using both the guide and the code as reference. The reason i think this will work is because it got things 95% correct just having access to the function code alone (and without the rest of the codebase), so the "guide" would help it reach 99%. I do not expect it to get to 100% so a manual edit pass will need to be done anyway (and i have an idea for a tool to assist in the manual edit pass for that - a tool that i'll probably get an LLM to write - BTW good luck coming up if all that stuff would increase or decrease productivity for an actual product and not some random stuff i'm toying with :-P).
One other thing i've also found LLMs useful recently is to have them use the stuff you make and see how they try to use it. I have an old project, a GUI toolkit i've been hacking on every now and then since 2011 or so, though it was never a priority. Yesterday i decided to dump all the header files to Qwen 35B-A3B (i use 4bit quantization that gives me a 256k context - it isn't particularly smart but it is neat to not have to micromanage context size like i have to do with Devstral Small 2 or Qwen 27B where both of them aren't very usable speedwise at anything above 32k context sizes).
Then i asked it to just make a few small programs and it did[0] (the shot shows a paint app, a calendar, a todo list and a unit converter). Pretty much every program found bugs in the library :-P and gave me ideas on how to improve things.
In general i get the impression that LLMs aren't great at architecting things but if you do the architecture yourself and write them a framework to use, they should do a fine job at it.
[0] http://runtimeterror.com/pages/iv/images/d50c436990db203a07f...
Even this is overstating it IMO.
I already convert from multipliers to percentage of increase, so when someone claims 10x they very likely mean +100% productivity, and here 2x means +20% productivity, which seems about right. Nobody that was normally productive before LLMs has suddenly 10x'ed their output now.
The problem is that 20% productivity when it comes to generating code, really doesn't translate in 20% productivity increase overall, when you take into account the fact that the code quality is worse, the fact that writing code is actually not the majority of your time spent, and that people get burnt out from the usage.
2x is 100% increase.
I know, and I'm saying that 2x claims should be interpreted as an actual 20% increase, and 10x claims as an actual 100% increase. People are just bad at estimating their own productivity.
I've found a variant of the Gell-Mann Amnesia effect with regards to LLM coding. They're really magical if you're not familiar with the language, domain, or frameworks you are trying to use. But if you're familiar with those things then you'll often catch them lying to your face and producing a lot of plausible nonsense.
My observation is that AI tools generally make people /less/ productive. They are more productive in that they produce more stuff, but they are less productive in that the stuff they are producing has lower inherent value delivery. The primary place where I see this in software teams as a Product person is in document creation. If you are using an AI tool to produce internal documents, this is likely a net-negative value activity that actually reduces the overall productivity of the team. Why? Because writing is thinking. By handing off document creation to an LLM for a document that's supposed to communicate important details between teams (e.g. technical design docs, requirements docs, strategy docs) you are actually handing off thinking, or rather handing off not-thinking as AI tools cannot "think". The outcome of is poorly "thought out" slop that generates more work for everyone involved to resolve /or/ everyone simply ignores the document and the previous processes stop being useful.
If anything, I'd say AI tools in /most/ enterprises as people are trying to use them now are at least a -30% of productivity. If used correctly for taking a human-written/thought requirements doc, converting it into an interactive prototype that can be critiqued and ultimately included as part of the solution alignment within the requirements doc, and then is handed to an engineering team that is effective at using humans and AI to produce code, then it's probably a benefit. But most businesses lack the internal rigor, quality culture, and data governance to support properly applying AI tools in a high context manner internal to their business.
Have you worked with codebases where people produced slop even before LLMs? In such codebases LLMs are a godsend. They can churn around huge amounts of code and find needles in the haystack.
Does a 60x speedup count? Are you still copying and pasting from chatgpt, because if so your definitely doing it wrong.
Wait, how about an infinite speedup? A friend of mine couldn't code, and now he can. So he is infinity-x better at coding thanks to LLMS!
Take that, you mere 10x-ers! Your days are toast!
Left behind, as the infinite-x rockstar developer roars off on their motorbike, trailing a plume of dust behind
If 60x was a thing, we would see many fantastic products appearing on the market, every product would have improved or goten major new interesting features. Yet it hasn't really happened.
10x or 60x of very little is still not much. "Working programmers" unable to write FizzBuzz exist.
10x or 60x of a negative value is a 10-60x bigger problem than that initial negative value. We all regularly commit things that are actually net negative.
Finally, 10x or 60x more productivity channeled into yet another static site generator, CMS, or programming language doesn't really help in improving products or developing new interesting features.
In short: LLMs offer real productivity gains, but they are still tools, and they are very new, so we misuse them a lot, in various ways (not limited to examples above). Unless they replace programmers and can consistently produce better than average code (even if they fail catastrophically in a few cases), they won't fundamentally change programming. They did change a lot, but not enough to get the result you're describing.
I believe I have seen 60x improvements on certain classes of bugs. Just not on development as a whole
And I've also seen the other side, where vibe coding on projects that had bad code quality to start with leads to bug riddled messes. Projects with beautiful and elaborate test harnesses, that break seconds after contact with a real human user
[flagged]