> LLM’s amplify what you already have: opinions, structure, frameworks.
So far, so agreeable, but…
> If you have thoughts, they come out sharper and faster.
I can’t help but wonder whether constant use of “agent” harnesses will lead to an atrophy of the software engineering (or really any field) muscles.
Actual muscles need exercise to stay in shape (let alone grow), so does the brain. Can we really be sure that thoughts, opinions, taste will still come out sharper and faster after five, ten, 20 years of using these tools almost every day?
Conversely, I also am a user of LLMs (true shocker these days, I know), and am noticing a speedup in areas I was already familiar with, and a quicker introduction to new ones. The obvious benefit cannot be denied, and doing so regardless makes you look uninformed. [0]
So what’s the ideal “middle ground” in this situation? Stoically continuing to sharpen your skills on your own, but risking being left in the dust productivity-wise? Or taking an “agent first” approach and trying to learn and improve more only on the side, as more of an afterthought?
[0] Excluding people who don’t want anything to do with LLMs out of moral principle, which curiously just like the overarching topic I also both respect and understand, but on the other hand don’t do myself.
I will just point out the benefit is not as obvious as you think. Developers have consistently overestimated LLM productivity gains, which still seems true for agentic AI: https://metr.org/blog/2026-05-11-ai-usage-survey/ It is particularly striking how similar the results are to LLMs before agents.
Along with the total absence of long-term data, I think the benefit can be (weakly) denied. Maybe not in the employmemt marketplace, but certainly for myself.
> I will just point out the benefit is not as obvious as you think. Developers have consistently overestimated LLM
I think there are two different claims here:
- developers overestimate productivity gains, which is a solid finding in many of these studies. Skepticism of extremely large productivity gains is warranted and I flatly disbelieve "10x uplift" claims.
- LLMs give no productivity uplift at all, which is much harder to defend. A repeat of the famous METR RCT study did find evidence of improved productivity, and this seems to align with the experience of many experts I trust.
Specifically my claim is "the relatively minor productivity uplift I would personally get out of agentic development is offset by the high cost, along with unresolved questions about long-term code maintainability, so I am not convinced that it is actually beneficial."
IMO the bigger problem is that ~1.5x individual dev productivity uplift seems to translate into 1.05x uplift across the team. People have been waaaaayyyyy too overconfident about this stuff.
I am both a career developer and experienced team manager. from first hand experience the 1.5x im getting from AI is not flowing down to my team / org because why would i output 50% more when the pay environment and leadership are already underwhelming. That additional 50% productivity goes completely to side projects built on my second computer between 9-5 tasks
Actually, the METR report speculates that some of the overreported productivity uplift comes from grabbing unnecessary low-hanging fruit, things like "oh I'll make a web dashboard to keep track of this stuff // wow that would have taken all day without Claude!" But in the olden days they would have just used a notepad. Yet psychologically they built a real thing and saved a lot of time.
> But in the olden days they would have just used a notepad. Yet psychologically they built a real thing and saved a lot of time.
There was an old Onion article about the monetary savings from buying clothes on sale as computed by (a) the women buying the clothes; and (b) their male relatives.
That’s what I’ve heard from my dev team too. They’re using it to give themselves free time while still being on the clock, not to produce more output for the company. Roughly thinking about hours spent on projects I think have gone up per task, the opposite that should be happening.
I'm finding it almost impossible to fill that free time with work, unless it's just reading emails and chat messages.
I can context switch between two or three chats, but doing so speeds up my agent use at the cost of making reviews and discovery harder, so it might come out in the wash.
I'm finding this is a skill which I'm slowwwwwwly improving but, for now, my level means I tend to miss mistakes I'd notice if just working on one task. I also had to make accommodations to the way I work to make it achievable, including telling my employer that a 500 GiB disk just isn't enough any more when I need multiple builds in parallel.
Your comment makes total sense to me but generally, in regards to productivity gains from AI, I can never understand where these are realized for people. Maybe I'm just a laggard but never found myself 25%-50% behind on anything or that much more work/items/tickets available.
Great point. Devs have relatively no incentive to be more productive for their orgs, and all the incentive to be more productive on their own personal work. The AI benefit to devs is real, just not for large enterprises IMO outside of automating clerical/mundane work.
> Devs have relatively no incentive to be more productive for their orgs, and all the incentive to be more productive on their own personal work.
AI definitely supercharges the side project myth, but I think we’re going to see waves of developers quitting or losing their jobs because they’re chasing dreams of running a side code project that isn’t going to turn into a business.
The incentive to be more productive at work is that they get to keep their job and not be replaced by a cheaper junior. I think we’re in for a reckoning across the industry as companies realize that there’s little difference between a lazy senior armed with Claude and a halfway motivated junior who has aspirations of growing into something more. The latter costs less and might grow into a better dev.
> because why would i output 50% more when the pay environment and leadership are already underwhelming
The productivity uplift measured in some of these reports is from more tasks being done, not from developers giving themselves more idle time during the day.
If we do see job losses from AI, my feeling is that it’s going to be concentrated in people with an attitude like this one where workers try to remain anchored to their old delivery pace. It’s becoming easier than ever to replace a worker in that position with a lower paid junior equipped with Claude who has an interest in learning. There’s no reason to keep a higher paid worker around to copy and paste between Jira and Claude.
thats absolutely true regarding wysiwyg projects like a small website. aside from temporary cost savings, systematically ignoring institutional knowledge in favor of mass output is not gonna do humanity any favors ..
Yeah institutional knowledge is hard to capture, even more with agentic coding.
I just compressed what would take a team a few months to deliver in one or two months, the amount of gotcha I spot from the existing system we work on everyday is staggering.
How to deliver and keep these feedback is difficult, the other teams are overwhelmed by other tasks, so any tickets or communications will be forgotten in a few months.
And as an external ressources allowed to use frontier model, I have a hard time to keep up with my own pace, and all that institutional knowledge will be lost both by my employer and client.
I have seen no productivity uplift at all from LLMs. They are at least about neutral these days (they used to be a productivity drain), but no gains at all. By the time I get done reviewing the code to make sure it hasn't done anything crazy, I've spent the same amount of time I would've taken to write the code myself. The only people I personally know who claim productivity gains are getting those gains by completely disregarding quality, or understanding the code, and just YOLO letting the LLM do everything without checking. I'm not willing to do that.
I find Claude is shockingly slow. If it were genuinely instantaneous, it would be worth it, but I often find myself twiddling my thumbs while waiting for it to cook.
Maybe we can make an analogy with taking the plane vs driving to your destination.
A plane literally goes 10x faster than a car, so a 5 hour drive becomes a 30 minutes flight. But if you have to drive to the airport, arrive early, pass check-in, security, boarding, then pick up your luggage on arrival, rent a car and drive to your destination, you may realize that your 30 minute flight took you more than 5 hours in total.
You also have to consider the Amdahl's law: a 10x speedup on 10% of the project is just a 9% speedup overall.
> You also have to consider the Amdahl's law: a 10x speedup on 10% of the project is just a 9% speedup overall.
This is totally true for me. I'm working on more features/bugs at work and getting the coding done faster with claude but the coding part is at maximum half of the process and usually less. The rest of it is clarifying the requirements from our product team or given that we're a very legacy place, what the existing state of our infrastructure and other systems are for the new changes to fit in to, etc.
> You also have to consider the Amdahl's law: a 10x speedup on 10% of the project is just a 9% speedup overall.
It's a 10% speedup, unless you think that a 100% speedup would reduce completion time to zero rather than cutting it in half.
Assuming that "10% of the project" is measured by time requirements, when you get a 10x speedup on that part of the project, you'll end up completing 100 units of work in 91 (formerly 100) units of time. Your work rate is then 100 / 91 = 1.099 times what it was before; that's an improvement of 10%, not 9%.
(Sanity check: suppose you have a project that will take 100 days. That project is reassigned to someone 10% faster than you. They will take 100 / 1.1 = 90.91 days to finish.
If they were 9% faster, they'd take 100 / 1.09 = 91.74 days to finish.
Does a savings of 9 out of 100 days of work look more like the project that saves 9.09 days, or the one that saves 8.26 days?)
> LLMs give no productivity uplift at all, which is much harder to defend
It’s not really hard to defend. Because when people says that productivity is uplifted, they are talking about amount of work, not the ROI. That’s why you keep hearing about LOC, amount of PR and prototypes, and the time taken is actually “time to PR” and not “time to production + time spent on bugs”.
There's another even more abstract liability on the balance sheet here for "deprecated LOC" or something, where the thing they built gets scrapped because no one wants to untangle the rat nest, or the "documentation" was so incoherent that it all had to be rewritten and reviewed.
Coming from legacy enterprise I am using claude to do exactly this for some of my team's more bizarre and poorly documented "inventions" that started out as their pet project and then became part of our official workflow.
In the former case, which seems more likely, does it not fall to reason that like any tool there'll be some that will benefit from it and some that it just won't work as well for, but are equally as good at their job? Like that neovim, unless your an absolute zealot you wouldnt insist that everyone use it, or hire someone based off it, but those who do work well with it do find it to be a boost to productivity.
Also importantly, can it be a neovim? Neovim hasn't had literal trillions invested over it in the space of just 5 years, local llm's aside can those smaller productivity gains justify the huge investment put into them, and which will continue to be needed for further model development?
Developers are cashing in on the productivity gains. Meaning instead of using the increased productivity to do more work, they just become lazier or do fun irrelevant side projects instead, where as before there was just not much time for such things. I have definitely procrastinated on work simply because I know I can swoop in with an LLM and do it all in 15 minutes, whereas before I would have spent a few hours.
This is how LLMs can both result in greater productivity yet still not appear to yield much more benefit than the pre-LLM era.
And there is no benefit to the developer for doing the work first then just sitting idle. That’s how you get people putting more things on your plate.
Some unsolicited, probably obvious advice for you for the future: never do that again.
Managers will abuse you if you expand your workday to make sure deadlines don't slip. They will see that the work is getting done and see no urgency to hire more help.
You have to surface capacity issues to management, learn to say "no" diplomatically, and/or "I have capacity for 2 things out of these 5, prioritize which ones you want first."
The LLMs are better at enforcing their boundaries than developers. Run out of tokens? That's it. No more code. It doesn't care how much the company needs this for the customer call tomorrow.
Odd, it was the opposite for me. Last year, no burnout. But then agents got good and everyone wanted more code more features faster and faster. I got burnt to a crisp. Now? I don’t give a fuck. Ok people want to push tons of code straight to prod? Go ahead, looks good to me… no need to review it if it’s with the latest frontier models right?
Exactly, I finally migrated that old service to typescript and cleaned up some really annoying tooling, for example. I’m more productive and can get quick fixes out very quickly. We’re also seeing agents help identify incidents faster than human investigation can. But big structural things that are hard to integrate still take lots of time.
A few months ago, I heard a keynote speaker claim that LLMs did not give any measurable productivity gains, except in two fields: customer support and software development, where they resulted in about 30% productivity improvement. I forgot if he presented any sources, and if he did, I forgot those, but it sounds plausible to me.
But instead of productivity, I'm much more interested in using it to improve quality. You've got a tireless reviewer who is always ready to review your code and catch any gaps.
I don't know how they're measuring productivity for customer support, but it certainly isn't in terms of customer satisfaction. People have never been happy with chatbots when they're trying to reach a human.
We are getting ai responses from cloud providers. From both aws and google, my teams have received obviously ai generated and non-applicable advice.
The latest was that we need to write in object pooling in their client code, an option that doesn't exist yet in _their_ client code. Like, yo, that is a you-problem on performance bottlenecks; we expect you to provide solutions.
I think the real disaster is that once you let the LLM work on a project for a bit, you start to lose understanding of what exactly is even happening under the hood in the project. You can take steps to mitigate this, but agents don't exactly encourage the behavior required to maintain a good understanding of what's going on.
When a person becomes a manager, they do or do not have enough time and expertise to review all of the code that they trust the team to produce.
Managers usually get into automated testing; unit tests, integration tests, acceptance tests, and maybe also BDD syntax
Managers and developers are responsible for setting a test coverage threshold for merge approval.
If there is 100% branch coverage test coverage for a codebase, what would coverage-guided fuzzing or property testing find? If there is 100% branch coverage test coverage for a codebase, what is the value of spending resources on formal verification?
How does the value of LLM-produced 100% branch coverage compare to no-LLM 100% branch coverage?
I've never really understood this argument. If someone's a manager of an incompetent team, no amount of management skill will save the quality of the resulting software. I don't think "just treat LLMs like smart junior developers" fixes this, because well-functioning teams usually also have senior developers to keep things on track. Like, if we handed a team of genius-but-junior developers to the best "people person" manager in the world (i.e. who doesn't actually read the code), would we really expect decent results? Even if the manager tested the code by hand? I honestly don't think so, at least once the software gets past a certain (fairly low) threshold of size/complexity.
If you have a "this is an incompetent team" attitude, that's what quality software you'll get back; whether or not there's LLM use.
If you have a "I trust my competent team to write good enough tests and test coverage isn't my responsibility" attitude, that's what quality software you'll get back.
There are people producing good and excellent quality software with LLMs. Presumably you must discard low-quality code in order to maintain quality.
There's certainly a limit to code quality with current models. On number of lines of code per unit of time, LLM tools certainly already win.
Can costly automated code review for PRs catch most of the problems before they're under consideration for merge?
For example,
the vscode repo has extensive copilot integration. Every PR gets auto code reviewed. But with their tokens or the contributors'?
If I take poor quality code (AI-assisted or not) and spend a few hundred dollars on tokens for a next gen model and agent to get to 100% coverage and review for security bugs and CWE common weaknesses, what quality code will I have without refactoring with proven patterns and type annotations and polishing docstrings?
I don't think this argument holds. The OP made the very good point that most quality software teams are composed of both seniors and juniors, with the seniors (and even staff members sometimes) drawing the lines so that the system stays on rails.
If you're a manager now and your "team" is a bunch of coding agents, those agents are hardly junior engineers at best. It is equivalent or even worse than hiring a team of 3-5 junior engineers and letting them run rampant with your code.
> If you have a "I trust my competent team to write good enough tests and test coverage isn't my responsibility" attitude, that's what quality software you'll get back.
As much as this sentiment is nice, it is completely divorced from reality, unless the competence is verifiably there. If you take a bunch of juniors and say "yeah I trust them to do everything well enough," you're going to have a disaster on your hands.
> How does the value of LLM-produced 100% branch coverage compare to no-LLM 100% branch coverage?
This is such a salient question. Sometimes (definitely not always) the test suites produced by LLMs are so trivial it's scary. Coverage can be an illusion for sure.
I have been seeing such trivial shit getting submitted and deleting them from the test suites, but it's disheartening to see that the devs didn't have the better taste to see those LLM "tests" are so trivial they border on noop before submitting them...
What new or existing metric would better assess test coverage than line coverage and branch coverage?
I wrote a tool called tert - I guess it's called an agent harness now - to run various test runners and log test output and coverage output to disk. FWIU stripping spaces from JSON does save tokens. It seems like feeding coverage lines-missing maps into the prompt results in better output, better LLM-authored tests.
"Refactor these tests for maintainability and coverage. Use fixtures, mocks, and parametrization"
You can write tests that cover every line/branch without actually verifying the code does what it’s supposed to. Relying on (unreviewed?) LLM written tests that provide coverage may not be a great idea.
We try to use types as much as possible. And if we can't, then lint rules, etc. Basically, get everything as far left in the SDLC as we can.
And, if/where we need tests, we write the source so they are few, high value, and complementary. Like actual unit tests, not complex with stuff like mocks just to generate trivial coverage.
Formal verification is always more valuable than mere testing, but it's hitherto more expensive. The thing that ultimately matters is closing the loop: how well do the tests match the requirements, both as written and as unwritten in the mind of the customer?
Working with an LLM has given me a real eye opener on unwritten requirements. It's like outsourcing. "Yes, you've given me what I wrote down, but I never expected you do to it in that way"
> When a person becomes a manager, they do or do not have enough time and expertise to review all of the code that they trust the team to produce.
> Managers usually get into automated testing; unit tests, integration tests, acceptance tests, and maybe also BDD syntax
I can see managers getting involved into acceptance tests, but never in the other type of tests. And the verification mostly is involved into a quick manual testing/watching a demo. Code is not their concern. When there's a bug, they expect you to investigate and fix it.
Given that it's so easy to generate better than test stubs with a model and manually review what it thinks covers, I think it's fair to expect a manager to run and add to the tests.
If devops has done their job, it should be trivial for a manager to contribute to the tests and run the build on git push (or manually re-run the build with the web UI).
If a manager has deploy rights, they should be able to run the tests.
I think what this fails to cover is that managers rely on other people to do that understanding and keeping things in mind. When there's a problem, the manager isn't just saying to a random person "go fix this", most of the time they're relying on somebody who has a deeper understanding of the problem and is not likely to randomly break something else just to fix this one bug.
Fearless refactoring is possible given test coverage? True in my experience; both with higher-risk human-developed software and with agent-coded software.
I haven't yet made myself learn the new swarm of concurrent agents with different specializations/agent_instructions methods yet.
Are multiple worktrees worth the cognitive burden and merge overhead?
If slop is fine (and sometimes it is), the benefits are undeniable. If the dev was the kind that would have produced slop anyway - again, undeniable boost.
If the quality needs to be high I think it actually can slow you down, though.
Agreed, but also to expand, if the dev is mediocre and ai-assisted coding is a skill, the productivity gains are 10x more mediocre code. Since code is a liability, this is not desirable. Hence, mediocre devs being more productive is an underestimated problem of the age of ai-assisted coding. I see this every day.
The result is a whole bunch of dysfunctional systems unnecessarily dislodging perfectly acceptable processes.
"Code is a liability" is the part that usually gets hand-waved, and there's a number on it now. Veracode's 2025 GenAI report ran 80 tasks across 100+ models: when a task could be solved securely or insecurely, the models picked the insecure path 45% of the time, at roughly 2.7x the vulnerability density of human-written code (Java was worst, at a 72% failure rate).
So the mediocre-dev case may be worse than "10x more mediocre code." It's more code that also skews insecure by default, and that cost shows up downstream in review and incidents, not at the PR. Throughput goes up, and so does the liability per line.
<< So what’s the ideal “middle ground” in this situation?
Putting all this in 2nd paragraph so that you can skip it if you think 'coding' is your primary portion of your job.
I suppose I am in a mildly privileged position in a sense that my work is a weird intersection of tech, finance, and comprehension. In other words, I don't code much, but I absolutely benefit from now being able to play with various projects I would otherwise have no business touching without a bigger support team.
I don't want to invoke Accelenrando, but the muscle imagery and analogy fits. I will give an example. I recently decided to pick up Go for a project ( have experience in some other languages, but I will still be starting fresh ). I could have codex build me what I want, but I am purposefully taking it slow so that I can learn the foundation so that I can have a frame of reference ( because I assume it won't be the only go project for me ).
Otoh, most of my one off python scripts I barely even skim anymore. And honestly,that is the part that scares me more.
Why would you be very careful with a one-off script? The only point is the output.
On the other hand, if you actually care about the output, how do you know it's right, unless you review the script? I mean, if all you care about is plausible-looking output, you could have the LLM produce that, and skip the Python script entirely...
Eh, unlike some of my contemporaries, I am not as interested in merely plausible-looking output. I want good output each time, but I am clearly still trying to find a good balance.
>Conversely, I also am a user of LLMs (true shocker these days, I know), and am noticing a speedup in areas I was already familiar with, and a quicker introduction to new ones. The obvious benefit cannot be denied, and doing so regardless makes you look uninformed.
My largest concern comes from something tangential to this: I'm not sure we're all that good at deciding what should be learned and sticking to it.
Silly example: regex. LLMs are, as far as I know, well above the average dev when it comes to writing regex. Regex is also one of those things that for many people goes unused for months, but then you encounter the occasional perfect regex problem, and it's really easy to just lean on the LLM to write the regex for you rather than spending some time tinkering and testing. Regex can be frustrating and fickle, I think we've all been there.
But then, you just don't learn regex. So where does the intuition for what regex can do come from? Do you just become unable to write regex with no LLM? People stop writing resources for regex I guess?
My concern is that there's stuff I feel I can just chuck onto the LLM but I'm sure my judgement is not perfect. It's still probably worth it, all in all, but I'm not even sure of what I might be losing along the way and that's an uneasy feel.
I've been using regex decades, but it never really stuck to do anything too complex, it was the perfect intersection of difficult and infrequent. ( And also variable - PCRE vs others customisations / non-regular parts, etc ).
I am very glad that I can now just ask claude for a regex to achieve my intent.
Does it mean I'll never master regex? Yes it does, but decades has shown that was unlikely to ever happen anyway.
Regex came up so infrequent that I found myself referring to documentation whenever I needed to use it. But I always wondered, what are the jobs or roles that use it so often that they have mastered it.
That's somewhat alien to me, I suppose because over the years I just never stopped using regex in search/replace tools, which is daily exposure regardless of what code needs to be written.
Sometimes the patterns are dirt-simple, like `/\bfoo/` to find things that start with foo, or a `/foo.?bar/i` to try to find multiple variations of FooBar and fooBar and foo-bar and FOO_BAR. Other contexts outside an IDE include browser dev-tools, exploratory SQL queries on the database, searching centralized logs, and occasionally turning lines of text into spreadsheets via CSV.
None of that guarantees deep expertise of the weirder tricks, but it absolutely keeps the general knowledge "warm."
> But then, you just don't learn regex. So where does the intuition for what regex can do come from?
The training data is there for regex and it's unlikely that regex will experience massive changes but your concern makes sense. I had to learn before LLMs handled that part, the "when to use this" intuition. My guess would be that the logical conclusion is dependence on LLMs to make that determination increasing over time, both for when and how to use regex, for better or worse.
In the era of vibecoding, there are people creating software that haven't ever heard of a regexp. I learned regexps when Perl was popular. It's a useful skill that has served in me well in my career, but if the industry's moved on from a place where regexps and Unix knowledge are useful because this new tool has replaced me, well shit. I'm excited for the future, but also that's not a great feeling to have.
I love AI but one thing that is a bummer is: Skills that used to be straightforward but people generally never took the time to learn are now trivial with AI.
Things like regex and CSS are not that hard if you take some time to study them. People don't though. Now I don't have that advantage anymore since AI can do them so well. Or... Maybe that could be an advantage since there will be even less people learning it.
They were also easy to look up on the fly as one needed them too. So really really knowing them was never a true advantage. For me it wasn't that I never took the time to learn them. I realized long ago that it wasn't worth my time to keep that information in my brain, and I offloaded it to google 100% of the time it came up in my day to day life. Now I offload it to Claude, same thing.
I do agree with that. I wouldn't say I really really know them either. I know enough to sound competent and be relied upon, and then I can google them to remind myself of the details.
I realized later you likely meant the latter anyways and yeah that's much different. It's important to know what these things are, how they work, and why one would need to tweak them. It was never important to have the syntax on instant recall.
If you don't know regex--how to write them, how to use them, what they're good for, what they're not good for--what makes you think you're competent to review a PR containing one?
If you don't know how to regex and you let your imaginary LLM friend write one for you, and you foist that upon your colleagues for review, you've just abused their trust. Because they, being ethical, honest engineers, must now do all the work you neglected except they're doing it at review time instead of at the time of writing.
Or maybe they're dishonest and unethical and they'll just maliciously or negligently approve the PR without actually reviewing the regex.
Either way the situation is completely rotten. You can never ethically use an LLM to do something you're not competent to do yourself.
Knowing what to learn has always been an implicitly vital skill to career growth. Maybe it’s all right fewer people will know regex, just like as each year passes relatively fewer people know about var hoisting in JS
What does the business benefit from you handwriting regex vs a clanker?
The benefits of knowing regex is not about the syntax (that’s what docs/google/books are for), but knowing when it’s right to use it.
Immersion with the codebase is not about coding and syntax (which is actually very easy for me). It’s about being able to intuit that a particular combination can result in a buggy state and provides the wrong result downstream. You lose that you stay in the dreamland of specs.
That's fair. This all feels similar to the shift that happened when Google came on to the search scene. To use your example, people realized they no longer needed to memorize syntax. Instead they only needed to ensure they understood the big picture concepts, and they could fill in the blanks on demand with Google. This is all just moving that layer of abstraction one level higher.
Not really. Before google, we had (and still have) books. What the internet and later google changed was the availability of information (If I had to rely on libraries and bookstores, I'd never be a programmer today). The job was never about memorizing knowledge, only the application of it. That has not changed since the dawn of computing.
Sort of. As a developer I had my stack of reference books, man pages, etc. I’d try hard to memorize syntax as it was a pain to look things up. And I also devoted some memory to remembering where in a book to look, which set of man pages to use, etc
With google that mostly went to the wayside. I just needed to know something existed and I could pull together far more information faster than I was spending before to just do quick lookups
I didn’t say the job was about memorization back then. Rather I found that each of these new abstraction layers made it easier to go broad with one’s on hand knowledge instead of deep
> I can’t help but wonder whether constant use of “agent” harnesses will lead to an atrophy of the software engineering (or really any field) muscles.
Used to think so, but they actually can also be used to train and strengthen skills, and learn new ones.
I had a coding interview, where they kindly sent a brief beforehand to help prepare, presenting a list of topics and concepts that might be useful during the interview, the tech stack, what kind of expectations they would have, and what they’d be paying attention to.
Obviously, it’s not exact list, and there are probably other evaluation dimensions.
But since I was out of practice on some of those, I had Claude generate a dozen sample projects, with each a list of tasks in one document and the solutions in another, and got to it.
Midway, I thought of using codex to role play as an interviewer, to tell it my train of thought and ideas as I went, get feedback, question my choices, etc.
Sure I only went through two and half, maybe three of those projects… but it’s the first time I actually enjoyed prepping for an interview. And I actually learned some things in the process.
Hardest part was probably stopping the LLM from doing the tasks, but nothing unsolvable given a bit more time, clearer instructions, and separation.
> whether constant use of “agent” harnesses will lead to an atrophy of the software engineering (or really any field) muscles
Well, I think most neuropsychologists would agree that the answer is "yes, there will be atrophy" - if you don't use it, you lose it.
> So what’s the ideal “middle ground” in this situation?
I've been thinking a lot about this myself. My current plan is to train myself to get good at recognizing the feeling of "there's potential effort here that I want to outsource to the LLM" and occasionally choosing to not outsource it and do it by hand - especially with personal projects, where there's far less pressure to ship with velocity than work projects - but I'm not settled on this. I'll take any idea!
For what it is worth, my colleagues who use these tools more liberally see a much higher regression rate and are left refactoring more as well. While they definitely put out more PRs, I can confidently say that mine are more valuable and better stand the test of time. Whether that matters or not is up to debate as I guess you could argue there may be some point where the productivity overcomes their regression rate.
My position is, if the technology will be so good and progressing so fast to the point that I don't have to write code anymore and I'm just dictating our needs and what I want architecturally, then I don't really feel the need to put in the time to learn the tools as they are now. That seems way easier to learn than anything computer science related that I had to learn to build robust systems.
> Actual muscles need exercise to stay in shape (let alone grow), so does the brain. Can we really be sure that thoughts, opinions, taste will still come out sharper and faster after five, ten, 20 years of using these tools almost every day?
It occurred to me recently how because the LLMs have to potential to lead to intellectual/skill atrophy, they are a factor which could lead to an Idiocracy-esque [0] situation.
And while it's a well-established Fermi Paradox [1] hypothesis that AI could lead to the violent destruction of a civilization, I think it's also worth considering that they could non-violently lead to their decline due to intellectual atrophy.
I don't think there is necessarily one ideal middle ground here. It still feels to me like what's best is a function that depends on who and when.
I see it as something like a personal gradient descent. You're working on a problem, there are solutions down there somewhere, and you can kind of feel the gradient of the tools-and-techniques ground around you. Any way you walk means you're investing time improving some skill or another. So you should go the way that personally feels to you will best get you moving in the direction that you want to go.
For some people it's obvious LLMs are competent coders, getting better, sticking around... and those people should lean into that gradient. For some people what's obvious is nearly the exact opposites of all that, and I'd encourage those people to also follow their gradient/heart/nose down the path of sharpening their personal traditional coding skills. Some people are in a relatively flat area where nothing is obvious, and need to explore and maybe just keep doing their best to hedge with a bit of both.
> I can’t help but wonder whether constant use of “agent” harnesses will lead to an atrophy of the software engineering (or really any field) muscles.
It will, but I'm not sure the impact of this will be all too great. We suffer from not knowing how to use an abacus because we have a calculator, and people who feel a pull to keep their low-level chops up will do so anyway.
This is just a repackaged argument against SaaS and cloud infrastructure. Every business could be using subscription free spreadsheet software and storing all of their data on prem and employing an IT team to manage the servers and data but the majority choose to pay a third party to handle all of that. Hell, they could even go so far as to do it all manually with written records and filing cabinets. All of those workflows still exist in varying degrees scattered across the myriad of businesses, the existence of SaaS doesn't eliminate the others. AI seems to be playing out in a very similar way with the big providers, local models and die hard human programmers.
The crazy thing is how cheap it is to spin up a cloud server, but how little actual processing power you're getting for your money. Even if the business manages to keep their cloud spend in check it's probably going to be at the expense of a terrible user experience because everything in the web app will feel slow and all that matters is individual P95 request latency is within some arbitrary number of seconds.
The exact same arguments were made against electronic computing in general, in the early days. Pearl clutching is a very human thing to do as new technologies are integrated and become common place. A whole generation or two of developers are going to have to pass away before we stop hearing incessant diatribes about LLMs.
> The parallels you can draw between LLMs and calculators.. just don't make sense.
The technology doesn't matter, you can compare to say the power loom from the 1700's. I'm comparing the reaction of humans; human's haven't changed that much. They always react the same when they feel threatened and emotionally challenged by a technology.
And someone else always cries luddite if anyone says, “Hey guys, this isn’t what the salesman said.”
What matters is the reality of the thing… which is exactly what everyone is discussing.
As they should. We need more conversations like this, early. It could have saved us decades of bad practices with impossible expectations (see: agile).
But people can't separate their emotions from the "reality". The reality is, this technology has strengths and limitations. It has benefits and it has negative consequences. We can and will discuss all of that. But many people aren't operating from a detached analytical viewpoint. They're operating from an emotional, self-interested, defensive stance. And granted, there are just as many people operating from a utopian, rose tinted, self-serving, evangelical position.
In the end, the technology will get used where appropriate. And more importantly many of its weaknesses will be overcome and replaced by new challenges.
It's just a bit tiring to hear the same denouncements repeated over and over. Everyone knows them all by heart now. They're not wrong, they're just not helpful or accomplishing anything. The technology marches forward and will develop naturally. If you personally don't want to use it, then don't.
> They're not wrong, they're just not helpful or accomplishing anything.
I think what it accomplishes is developing a shared understanding of the pros/cons of the tech, the risks associated, how we should mitigate those risks, where the tech is appropriate vs. not appropriate, etc.
It feels very dismissive to say that these conversations aren't worth having. Or that we're thinking about these things too much.
I get that it's tiring, especially in the HN echo chamber. But the conversations are still worth having IMO.
There are conversations worth having. But it feels very dismissive of the obvious power of the technology to constantly focus, in an unbalanced way, on the challenges. People are biased heavily by irrational emotion, on both sides of the debate. It's all just getting a little tedious. I'm all for rational discourse and debate, but it's hard to find through all the vitriol and contempt.
Exactly, looks like I should have written that down for myself instead of relying on LLM or even better, train my memory instead of depending on crutches ;)
I don't think this is equivalent. The calculator won't occasionally hallucinate a wrong answer. LLMs are a far leakier abstraction which means skill atrophy impacts evaluation and verification ability.
We will suffer from not knowing how to add. You could still argue "so what?"
Systems aren't a single addition. They are compounded operations with sprawling complexity. What happens when you can't reason through the system? What happens when you start asking for the wrong things? What happens when saying "fix it" on loop stops working?
>Can we really be sure that thoughts, opinions, taste will still come out sharper and faster after five, ten, 20 years of using these tools almost every day?
After 5 years, I think the thought profile every power user of the LLMs would be an LLM derived carbon copy of each other.
Prepare the world to get even more boringly uniform
If cars did not exist, I would be healthier, able to walk and run many times further due to constant cardio exercise.
I would still travel much less distance.
And just like cars, LLMs will reshape the world to the point that our brains could not even get us to the supermarket because soon it will be 5 miles away and require a car ( or at least a local LLM bike )
FWIW a 5 mile walk is not a long walk. Humans are very much designed to make that trek without too much trouble. Lots of the world has people walking similar distances for fresh water, to go to school, etc.
The point being that cars also reshape your perception of the world. METR had trouble replicating their dev productivity study because devs have already become much whinier about doing things "manually." Things that were slightly tedious in 2022 have become impossibly difficult without AI in 2026.
It is striking to reread Lord of the Rings: so much of the story is spent walking in and observing the Middle-Earth wilderness, and there is a distinct "pantheism" absent from the movies. JRR Tolkien, being one of the first modern fantasy writers, was one of the last who remembered a life without cars.
I had the chance recently to do several of New Zealand's Great Walks (3-4 day scenic wilderness treks with more expensive huts than the rest of the hut system), and I highly recommend doing something similar if you can swing it. Walking 15-30 km a day (9-18 miles) for several days while carrying supplies on your back really gives you new context for what you can do and what life was like without cars.
> So what’s the ideal “middle ground” in this situation?
I use agents to code. But I remember the early days of just AI smart complete in the IDE, where as the programmer I had to be more involved with designing and implementating the solution. This kept me engaged with the implementation as it was being built out. Now with agents, I find myself trying to catch up with what the agent did and spend more time code reviewing. Maybe you end up in the same place in the end. But building the implementation, vs code reviewing, feels more rewarding and I think helps keep your mental tool sharpened.
I think that the onus is on us to get better at using agents and AI to solve the pain points and speed things up while keeping quality high and our mental tools sharpened. I do nto think turning back is an option, but managing the pain points and leveling up is.
> I can’t help but wonder whether constant use of “agent” harnesses will lead to an atrophy of the software engineering (or really any field) muscles.
For sure. You cannot have "only higher level thoughts" without doing lower level work.
Ironically llm themselves prove that because you cannot remove facts like 'paris is capital of france' from llm and have it just retain 'high level thoughts' like 'countries have capitals that you can look up'
> For sure. You cannot have "only higher level thoughts" without doing lower level work
What do you mean? I think people routinely think about things at a very high level with almost no understanding of the lower levels. How many people use a computer each day and reason about them at a very high level while knowing nothing of capacitors, logic gates, or programming languages?
How many people struggle with their computer, or get scammed, because to them it's just icons on a screen, with not even the concept of a process, memory vs. disk, or anything? How much money is lost each year because someone doesn't know what an URL is?
I think they didn't phrase it precisely, but my guess is the underlying idea is actually "high-level software architecture doesn't have a clear abstraction layer you can use to separate it from low-level coding (unlike logic gates, the CPU's ISA, the kernel API, etc), and so delegating the latter leads to delegating the former".
That makes sense but I'm still not so sure—we have things like software architecture patterns that can be discussed at a high level without knowing the intricacies. Like you can be aware of load balancing and even use it but be unaware of how load balancing might work algorithmically.
Let's consider even the original example.
> You cannot remove facts like 'paris is capital of france' from llm and have it just retain 'high level thoughts' like 'countries have capitals that you can look up'
Wouldn't the knowledge that countries have capitals precede the knowledge that Paris is the capital of France?
This says nothing about the accuracy of our own models based on these abstractions that lack the lower-level understanding.
> we have things like software architecture patterns that can be discussed at a high level without knowing the intricacies
I think the counterargument would be "you can't teach people architecture alone and get good architects".
I've observed this myself in "systems engineers" whose job is to connect boxes together without understanding how the boxes work. They, invariably, design ridiculous architectures on their own and need to basically find a domain expert to route their opinions through to come up with anything sane.
> we have things like software architecture patterns that can be discussed at a high level without knowing the intricacies. Like you can be aware of load balancing and even use it but be unaware of how load balancing might work algorithmically.
It can be fine as a “user”, but not really as a “designer”. Because discussion about possible solutions is a matter of tradeoffs and tweaking of parameters, not slinging words around. Abstractions are not appliances that are plug and play. They’re often full of parameters that dictate their usefulness and costs, and not understanding those parameters is just roleplaying.
I think it is because people have preferences for top-down vs bottom-up learning strategies and bottom-up is what the education system is built around.
I suspect top-down learners are who benefit massively from LLMs while bottom-up learners think they are bullshit engines.
As a top-down learner myself, Claude is really the best technology of my lifetime. I can completely understand though how a bottom-up learner could see Claude as an afront to their fundamental epistemology.
For sure. You cannot have "only higher level thoughts" without doing lower level work.
Spend 3 days a week writing Ruby on Rails and 2 days hand rolling x86 assembly. Every web dev I know has been doing this since long before LLMs. Ensures they can keep having high level Rails thoughts.
> I can’t help but wonder whether constant use of “agent” harnesses will lead to an atrophy of the software engineering (or really any field) muscles.
Compiler evolutions really harmed how well software engineers understand or how often they have to drop down to assembly language.
Is that a problem for the 99% of developers around? Probably not.
I view LLMs as the next evolution. Some people will still need to care about the layer below, the shape of the code that is being written. But over time, just as it was with the transition from crafted ASM to higher level languages, the compilers became better, more efficient and trustworthy and I think the same will happen to LLMs, and we probably won't have to check the generated code as much, at least for most of the code around.
Is that a problem? Yes, for code that is intended to interface with humans (most of it still). The quality will probably become better and it won't be much of an issue.
LLMs are not deterministic and I still look at compiler result and I encourage people to look at it. That people don't look at the result of compiler is a problem and not a an argument
> but risking being left in the dust productivity-wise?
What's the risk here? Left behind by who or what?
> Or taking an “agent first” approach and trying to learn and improve more only on the side, as more of an afterthought?
This reads like anxiety resulting from FOMO.
Here's my take: I don't care about LLMs or AI in the sense that I don't feel any need or want to use them. I've only ever tinkered with the free ChatGPT. Never opened an account with any LLM vendor and never even considered it. I program by hand for the joy of it and sometimes for work. Still by hand as I have been doing. MY work gives me that luxury. For now.
Am I obsolete? Am I no longer of any value to society? Of course not. That thinking is just implanted by a group of money hungry individuals who don't give a fuck about me, you or society as a whole. So why would or should I care about LLMs?
Economically if you are vastly outcompeted by other programmers on productivity, yes that is "no longer of value" from a purely employment perspective. Much like an old person who cant use a computer has little value in the job market beyond being a greeter at Walmart, a programmer who hand codes a loop is next to useless on a productivity basis such that it makes zero sense to employ them. It is unfortunate but true. Why pay someone to accomplish less per dollar pf cost. Feels?
Last month, there has been a billing-related incident at work: it has been detected that the GCP bill had increased vastly since early 2026
The related team investigated, found nothing; They escalated to other teams, found nothing; Gemini, claude and whatnot were involved, found nothing;
Somebody raised to topic to me, solution has been found
See my point ? Like GP, I do not use AI. If there is only one person thinking left, being that person does not looks like a bad idea;
Mixing code and "productivity" is trying to remake coder as bluecollars, and most of them probably are (not thinking, replaceable cogs that are overpaid and shall be decommissionned), they will not be missing. The only coder's output is its idea, everything else is a mean to that end.
> Economically if you are vastly outcompeted by other programmers on productivity
That doesn't seem to be the case, at least currently. The credible numbers from from companies that are using LLMs extensively are in the 15-30% productivity improvements for some developers. Those figures don't factor in how that impacts end-to-end SDLC productivity and many of the published results ignore quality which impacts maintenance costs.
Before someone hops in with "Future models will do better", that's also by no means guaranteed. I think the poster you're replying too is safe for a while.
> I can’t help but wonder whether constant use of “agent” harnesses will lead to an atrophy of the software engineering (or really any field) muscles.
This will and is 100% happening. I have a friend who hasn't written code by hand in around a year, but uses LLMs every day, and he tells me he can't remember how to write code by hand anymore. He has been a developer for 10 years. But he's not working for anyone at the moment, so I imagine if he was in a workplace the circumstances would be different and they probably wouldn't settle for this.
I think that as a result of this it likely also atrophies the problem solving and architecture building skills that writing the code manually gives you. It just ends up degrading into a loop of tell the agent to do X and assume it knows what it is doing.
I guess I'm lucky to have taken the career path I have, but I write very little code these days being in pre-sales now. "Productivity" is pretty stark and obvious in sales. You can output all the text you want, maybe it even compiles and passes test suites and looks pretty in a demo. But does it sell? It's nice to be able to have the only true measure that matters actually be measurable. I have yet to use an LLM for anything at all, haven't touched an agent, not because I'm ideologically opposed, but I just haven't seen the need. Part of it is the largely advisory role. I'm largely expected on the customer side to field and answer questions and render technical assistance. I'm not much good at that if I'm just deferring to Claude. The customer can do that themselves. I need to know more than Claude. The other part is just I'm not being paid to generate text. I'm being paid to generate revenue. If there's a convincing argument that LLMs can help me, well, it's an argument, but truth on the ground is I'm performing every bit as good if not better than any of my peers that are blowing a grand a month on tokens. They're definitely having a lot of fun and flooding internal Github orgs with new repos and tooling, but it demonstrably isn't generating new revenue.
In a way, this happened before LLMs with more workforce-fit education, decision tree flowcharts, then software, and so on. If you take most people who started any field, the way they started the field would look very unorthodox, inefficient, etc. "From the margins," as pg might say. Margins where more intuitive skills for the craft are present. Ie, the skills to come up with the model pastry are not the same skills for the pastry line to be baked in a factory.
The part of brain which cares about efficiency and waste have "atrophied" decades ago in most devs. More likely they just never cared...
E.g.
> A 1.6GB Docker image for a Node app. Every code change rebuilds npm install from scratch. CI takes 8 minutes for a one-line fix.
> The team's proposal: add more CI runners.
Would it be surprising if that team makes a React app which does 10x more requests than necessary, has 10x more JS code than necessary, flickers, etc. People just don't care, and it has nothing to do with AI. "Engineering" is too big of a word for this kind of activity.
On the other hand, I think AI can amplify capabilities of those people who still care, and it's a good thing.
I think you can use them critically. Probe the code, see how it behaves when tweaked, can you break it etc.? without slowing you down too much. And in doing so you probably learn 80% of what you would have if you'd written it yourself by hand.
But then there are definitely cases where LLMs are slower and less reliable than someone using tooling by hand, especially like deterministic refactoring operations can be done close to instantly if you already know what needs to be done and how to do it.
Use it or lose it, but the problem is very old. The introduction of the alphabet/literacy almost certainly made people's memories worse -- up to that point they had to hold a lot more in their heads. At the same time it enabled a different better way to share information and learn. The question is thus, will it make the world better, is it a book or a TV?
The amplification is what is causing the skills atrophy. When everyone is chasing max lines of code, we generate more code than what we can realistically write.
And it's not like we can not use LLM because others are using it in a craze to get the next promotion so you have to use it and maximize your output even if your skills are atrophing
You can run to the farm to kill your own chicken so you can stay sharp and fit or figure out what's economically valuable. Or make a bet on what will be valuable later and commit to a brand of yourself
This is not true, though. Technically I don't do C but I used to do Assembly. Yet in Java code paths, to have a decent proof (and satisfactory results) of the optimizations made by JIT, PrintAssembly still works.
Assembly is not just the raw instructions that trigger microcode or knowing the latency of each one of them, or the (un)expected cache misses, it's the understanding the hardware, itself.
I'm frankly baffled by the number of commenters saying they've seen little productivity gains using LLMs/agents. My team got cut in half at the end of last year and we've probably increased our output by 25% primarily due to LLMs. Personally it's probably doubled my productivity. And we generally use a pretty structured spec based approach with consistent human code review, and six months into adoption we've had virtually no bugs attributable to an AI mistake (in fact production issues have gone down, knock on wood). My point being we aren't vibe coding.
I speculate the gains must be heavily context dependent (no pun intended). Different teams, tech stacks, product types, and company policies perhaps have an outsized effect on what gains LLMs give. We are allowed to be pretty independent and don't have much of a restriction on token usage.
What is your metric ? How do you calculate ROI ? Strange that you didn't specify that. Also, are you sure your "25% increase" is due to LLMs ? Every time I see this claim, it is always for a short time period or the increase was not due to LLMs at all
>My team got cut in half at the end of last year and we've probably increased our output by 25% primarily due to LLMs. Personally it's probably doubled my productivity.
> Conversely, I also am a user of LLMs (true shocker these days, I know), and am noticing a speedup in areas I was already familiar with, and a quicker introduction to new ones. The obvious benefit cannot be denied, and doing so regardless makes you look uninformed. [Excluding people who don’t want anything to do with LLMs out of moral principle, which curiously just like the overarching topic I also both respect and understand, but on the other hand don’t do myself.]
Setting aside my moral outrage over the magic token machines. What about me, who gets so tripped up over minor factual errors, that I'm unable to let them go, and it taints the whole conversation such that I'm too wrapped up in my frustration that I can't think about it clearly? Or my innate drive for correctness that's so strong that I eval the minor errors in output, as catastrophically incompatible with my goals?
> Stoically continuing to sharpen your skills on your own, but risking being left in the dust productivity-wise?
I don't believe there's a meaningful productivity increase. Please cite your published (not preprint) peer-reviewed research that proves the productivity improvement. Until then, I'm unconvinced. (Believe me I'd like to be convinced of reality, the answer is still unresolved, and I have my opinions, but I'd rather something conclusive that I can have confidence in)
Then, even if you did show a significant productivity improvement, it wouldn't help me. I have too many qualms over the output quality that I simple can not let go, (I don't think I should, but everyone keeps trying to convince me to lower my quality standards). I don't want something fast, I have plenty of really "fast" things in my life. I exclusively want to add things that are high quality to my life. Things that don't endlessly frustrate me.
The question about where the middle ground is a rhetorically dishonest question. You'd first have to prove/convince me, that there IS a middle ground. Instead of what I believe where that middle ground belongs is quality, and everything emitted by an LLM moves reality in the wrong direction.
Are any of these absolutes? nah, hence my request/demand for peer-review research. All the productivity claims and quality assertions (mine included) are still *exclusively* vibes. But exactly none of them are pristine, (especially not any of the LLM output.)
While I can see that you feel very passionately about this, the reality is that it's the majority experience that will dictate adoption.
There may never be published research on productivity -- blinding in this instance is impossible, so I don't know how you'd ever do fully-controlled behavioural studies that carried any weight. It doesn't matter. If enough of us decide that LLMs are useful to us, then this form of coding will become the norm.
If that ends up causing more harm than good, then eventually there will be a course correction. But for now, for enough people that matter, LLMs are at least giving the perception of productivity increases. And our decisions and choices come down to the perception of reality, not reality itself (for better or worse).
So I think it's far more useful to take a pragmatic approach, as per TFA. Accept that LLMs have issues, but also bring advantages, and that LLM use in coding is here to stay. What we can do is remain aware of the bad, and make better use of the good.
As for you, personally ... if you mentally cannot deal with LLM output, then I think you have two choices. You can either learn to author system prompts, so that LLM output better fits your needs and no longer triggers you; or you can sit more and more on the outer, raging against the machine while the world changes around you.
Eventually, you'll be like a master craftsman in an era of mass-production. But that's potentially highly valuable in niche markets (consider a watchmaker working in Glashutte, for example), so you may yet win from this. Remember that every day, LLMs are making your own coding skills and knowledge more elite and therefore lucrative, sit back, and smile.
Jesus ! You guys are very authoritharian. Even if it is now the norm, I prefer to not use that tool and I bet the more the time past and the more I will be faster than anyone that use LLMs (because of skills atrophy)
LLMs can also be used to improve quality. The approach I tend to encourage, is to write the thing yourself, but let the LLM review your work.
I think that's a far better and more enjoyable way of working than letting the LLM write it and letting users review it. Partially because writing is more enjoyable than reviewing, and I think people are inherently bad at reviewing while LLMs with their infinite patience are much better at it. But also because the writing itself shapes our thinking and we learn much more from it; it makes us better in ways reviewing the LLM's work does not.
That said, I do let AI write a lot for me, because I'm tackling issues that would otherwise be way too big for a single person. Is this a good idea? Maybe not in general, but iny particular context I think it is.
I'm not sure to what extent I'm fooling myself about that last bit.
Peer-reviewed evidence of "productivity" is a pretty high bar to demand.
I don't know if I've ever seen a study that attempted to demonstrate productivity that was ironclad to the point that nobody could find any methodology holes to pick in it.
I won't be as diplomatic as the other reponse to you. You are welcome to your reticence to agentic coding. But as you're no doubt observing, all your peers are saying they're moving ahead leaps and bounds.
Either all of them are wrong, or you will be left behind. Whatever the outcome, it will be on you, and you may make your peace with that.
I appreciate the candor :) But if I'm honest about the size of my ego; the people who write code I admire, the ones who feel remorse when something they built breaks. The people who care about others, and about quality, and don't list TC as something of value. Those are the people who I consider peers. None of them enjoy LLM codegen. They all tell me, that it's annoying because it takes more effort to fix it's output than to just write it by hand. So, no, none of my peers are saying they're moving ahead leaps and bounds. They equally don't understand why everyone thinks it's better. Excluding those people though; I'm more used to everyone around me being wrong. It'll be interesting to see if I, or we, get left behind.
I really appreciate your comment, I think I've discovered part of why I've never felt any anxiety over being replaced by an LLM. I've never tried or wanted to emit low quality code at high velocity, my standards and goals have always listed quality first. Feels more like asking a chef if they're afraid of llms. Why would they be, they're not doing the same thing.
I'm a hobbyist and know the programming trade to a certain extent, but at work I'm "the client" and I have my employer contract developers to address all kinds of needs that I represent as a "business interface".
I'm with you on LLMs being in a different league than good developers curating a fine design with attention to detail, soundness and maintainability. It's just that 1- not all software/components within require that same amount of care: many capabilities are already commodified (as library code or boilerplate, or frameworks, or prior art in the training data that LLMs could easily parrot), and 2- till now, the financial calculus was leaning strongly towards "new developments are expensive, let's use restraint" (irrespective of the actual effort/ratio of commodified code).
So long story short, I don't see LLMs taking out skilled and caring developers: I'll still be reaching out to them for the critical/strategic projects, but on the other hand, I appreciate that the "ancillary" or "fancy" stuff can be cobbled together with the LLM glue gun and remove the impression to my management that engaging with software development is necessarily a costly/risky/exposing endeavour. Some software is to be treated like pets and cherished, and the new reality is to accept that we might have cattle software too, with swaths of prototypes, one-off's, very specific and niche and disposable, and it's fine because this no longer represents an unsustainable financial burden (my bean counting management already likes to kill useful software on a pure spec-sheet matching exercise and shovel everything into large "do it all" systems like ERPs: the fewer things the cheaper, right? Except for those underrepresented users thrown under the bus and their effective productivity).
You need to create your own custom interface. Dive into the harness where the System Prompt lays and write instruction for the LLM to quadruple-check facts before presenting them. Instruct them to check and verify all outputs using tools of your preference before claiming their tasks are finished. Instruct them to review everything, to look for gaps in their reasoning and fix them, to never assume your intent but ask for clarity.. No doubt over the years you have developed a highly tuned arsenal of tools and protocols for work. Treat LLMs in the interface as a tool that you can lay your tuned arsenal over, recreating it as a filter your LLM works inside. Don't use vanilla interfaces and expect them to mirror your niche level of expertise. Over the last year I have developed a custom Claude Code interface using tweakCC, but other interfaces like PI and OpenCode are better documented. I am full on adhd and autistic, and have achieved as a result of access to LLMs things my neurology would never allowed me to achieve otherwise. I can bounce all over the place and still achieve excellence because of the hardcore customizations I have in place that account for my neurology. To me there is no middle ground. Slop is unacceptable. The harness and System Prompt is what moves LLMs to quality outputs.
> a genuinely good tool that enriches your thinking
A smartphone is also a genuinely good all-around tool. Even social media is a genuinely good tool for connecting people.
Yet, I feel like we've been overly optimistic about the impact of said tools on us and our societies in the past two decades.
Smartphones are so good, in fact, in some societies, half of us are addicted to them. Billions of people world-wide.
I ask myself: Will LLMs enrich my thinking in the long run, or will they ruin it?
And what about most people? Will half of us outsource most of our thinking in a decade from now?
Given the speed and global scale that we're running these experiments with, it's fair, I think, to be a bit sceptical of the conclusion that, in the long run, LLMs will enrich our thinking.
I agree with this. Tools are good, betting the farm on a single tool is bad.
Example, cars are good. Betting the farm on cars to the detriment of bikes, buses, and trains is clearly bad. The tool of an llm is clearly handy. Betting trillions of dollars and linking the future of the nation and globe to this tool is clearly bad.
Yes instead of investing everything in AI the USA should invest in electric versions of their vehicles and in trains that can safely carry them because of their lack of range.
There's a saying that the intelligence of an average prehistoric cave man should be, in general, higher than a modern day human simply because the lack of technology required stone age humans to be far more intelligent then we are today. Now you can survive by working as a clerk in McDs, but in the stone age you needed to be on your toes and smart af.
LLMs are just continuing the trend humanity has long been traveling down.
> There's a saying that the intelligence of an average prehistoric cave man should be, in general, higher than a modern day human simply because the lack of technology required stone age humans to be far more intelligent then we are today. Now you can survive by working as a clerk in McDs, but in the stone age you needed to be on your toes and smart af.
This is a funny twist on the glorification of the past.
Modern humans have been getting smarter even over the relatively short time periods that we’ve been measuring.
Modern day humans still can and do survive in the wild if they so desire. It’s not some lost intelligence that was only achievable by past ancient humans.
Flynn effect is just a trend for the past century or so and it's an international measure. For the past decade there's been a trend for the US exclusively that is not aligned with the flynn effect.
No, the CHLCA (Chimpanzee–Human Last Common Ancestor) would be smarter than humans. It could be that chimpanzee intelligence declined faster than human intelligence. Unlikely, but I want to clarify that we did not evolve from chimpanzees. We share a common ancestor.
So you're claim is prehistoric humans are stupider? Also unsubstantiated. And insulting to prehistoric humans. That kind of behavior isn't allowed on HN. Don't disparage a group.
"Flynn effect" claimed that intelligence has been increasing over time. At least we can sort of measure things in the modern era, while objectively assessing the intelligence of the long-dead is turning into skull-measuring territory.
>> I am a little bit scared to say this too: last month I spent almost 10k USD on tokens. It sounds so insane.
No it just sounds completely stupid. You spent $10k to get a program to write text that you can write yourself just as well and for free. "Dissonance" doesn't even begin to describe it. It's like suddenly everyone has turned into the listless blobs of fat that humans have turned into in Wall-E.
I just hope all that will eventually correct itself: open (one way or another) models will reach parity with proprietary ones when diminishing returns eventually make it impossible for LLMs to grow any longer, OpenAI and Anthropic will succeed in bankrupting each other, it will become possible for everyone to run their own, personal, free and open (one way or another) model on their own laptop (assuming we still own our laptops) and at that point most of the problems with LLMs, particularly the ones that come from having to pay someone else to do the things you could always do yourself for free, will go away.
> I am a little bit scared to say this too: last month I spent almost 10k USD on tokens. It sounds so insane.
What are people actually doing with all these tokens? I use LLMs pretty heavily for development, and I'm rarely spending all the tokens that come with a $10/month OpenCode Go subscription...
My heaviest LLM usage month came in at $45. So far this July, $6 and counting (it's been a light month). I can certainly imagine increasing my usage by several orders of magnitude but ... why? If I have something that needs to get done and an LLM can do it, great. But I'm not sitting here inventing reasons to waste money, which is apparently what the tokenmaxers are doing - as evidenced by the astounding lack of value produced by all this vibe coding.
You’re not even trying to use AI if that’s all you spend. I am not a tokenmaxer and it’s easy to max out my 20$ subscription just hacking on a simple web game. Working in a larger enterprise context it’s impossible for me to use AI daily and not spend hundreds of dollars.
I have 3 max20 subscriptions and usually max at least two of them out.
I'm doing lots.
I'm building a social media site that nobody will ever use. I'm printing an ancient game to Linux and expanding the features (such as adding hw acceleration where it isn't needed). I'm troubleshooting crash logs for an Xbox game that has maybe 50 active players. Im building a distributed customer service system that provides LLMs, RAG, tool calls for lots of integrations (Gmail, calendar, sheets) (this one actually has several paying users). I'm building games, and releasing them on the Google marketplace (play store?)
But what used the most tokens was a kind of agent swarm I experimented with to implement a complex desktop application for very specific data manipulation. That.. half worked.. but it's buggy.
I made several internal tools at my company that are actually getting used by pretty much everybody. That's my most successful work I think. Oh, and I created a backup deployment in Azure for my companies product because .. we had the azure credits, so why not?
Same. Granted I have the $100/month Claude sub and barely ever see any of my usage bars go above 50% before they're reset (and that's when I had lined up a bunch of things to test Fable with).
I write specs and review, but the LLM writes all the code, tests, etc and only gets re-prompted when it screws up badly. That said, I'm not running that many agent loops in parallel, so maybe thats where the cost bites
LLM's discussion is always the same, if you don't use it you lose it, although true, the question is why and maybe we don't need to do that thing.
Humans have been gravitating into cognitive thinking and not specialized tasks, some of us, a handful do repetitive tasks and that is ok, but, the majority of us work better in context, in guidance.
LLM offers that, offers that abstraction that we were and are looking into all aspects of our lives.
We once plow the fields, and harvest by hand, and was good, but suddenly became cumbersome, too much to handle and machines came along to do the job of 20, faster and better, and there humans just tweak the machine, teach the small repetitive tasks.
We did that with everything and we change our lives, the way that we upskill ourselves and our communities, we let specialized human tasks as a comfort/skill/romantic/artistic outlet, like some specialized vineyards or specialized farms that do specific very specific things. But the rest we delegate and move to another areas.
I believe the problem is not LLM/AI is how we are going to up skill, repetitive jobs no longer need us, so we need to upskill for context, for management, strategy
The issue that I take on this is ego, we believe that we can do a better job sometimes, an we exhaust ourselves with no reason, Agent delegation works in the same way as Human delegation, the difference is the speed on how that happens, reviews were a good idea when humans toke days to do a product, features, code, document or what ever.
The reward piece I believe is faster to see something working, the issue seems to be the control we had. The approach needs to be the same of those we do with almost everything else. We drive a car but no necessarily knows how to assemble, we might now here and there trouble shooting, change tires, oil, do some level of maintenance, but at the end just a few of us can pull an engine apart and check every piece, cars are not even manufactured manually anymore, QA is part of an automated process, as production rates are far to fast for humans to intervene.
AI brought that to us, insanely fast automation, and with that, Humans should not become gatekeepers but rather guides.
> And then there are the geopolitical tensions. What happens if China or the US cut us off overnight from these technologies? Just a couple of weeks ago the US government showed it was able and willing to cut off non-US citizens from Anthropic’s latest frontier model.
The only evidence you have is US here and all the frontier models from China are free to download. Pure projection.
I might be missing something but what exactly is being projected here? The US did what the author said. It's not farfetched to imagine geopolitical conflicts could, at some point, drive china to do something similar.
> all the frontier models from China are free to download
Maybe? Again, not hard to imagine that they might not be releasing their actual frontier models to the public.
I guess China did not deserve mention. They are the only one releasing open models consitently and not cutting any access. Only US actions show evidence and for the US only.
> I tend not to actually read most LLM output anymore; I skim it, to check if I vibe with it. But a problem statement of three sentences, that I will fact-check really hard. It is like code review: a review with 1,000 lines of code gets an “LGTM”. A review with 100 lines gets 15 comments.
This is where I take issue. I'm in a similar boat to the author. In the last couple of months, I've been experimenting with increasing the use of local and cloud LLMs for my research code. I'll create a prototype, maybe port it to a language I don't use very much like Rust, run some tests... but at the very end when I'm very happy with it, I _need_ to go line by line and understand _everything_ that is happening. Sometimes that means using an LLM to understand it, but even when I do and there is a concept I don't get, I try to read primary resources written by experts.
The least bad thing I've found LLMs good for is ideation because it's super easy to take the good nuggets and leave the bad, but even that carries risks of shaping thought and making everyone reach for and ignore the same ideas in the way the Spotify radio or YouTube autoplay has been shaping/flattening tastes for the worse.
I'm not sure what I'll rule at the end of my experiments with LLMs, but right now I'm enjoying the rush of having prototypes that run quickly. I've always been a top-down learner, being motivated by hacking a cool demo I half understand and progressively tearing it apart.
One thing about open source is a lot of people are throwing low quality PRs that should have been an issue or even a discussion, so you can understand what was the problem the person encountered that motivated such PR. This is hard to get because usually people use LLMs to also answer questions you make about the PR. I am tending towards blocking PRs from people outside the main developers in most of my open source projects. If the person CAN discuss the problem, authoring the PR is easier if I do it myself, even if using an LLM, because reviewing a PR authored by random internet people/bots is hard because of how much the entire code tends to change after minimal questions are asked. What I am sad of this approach is that I did met a few interesting people through receiving PRs and establishing trust and some relationships in the past (eight years ago and before)
Yeah this is where I stopped. I use AI every day for work but 10k spend to me is a signal that OP is doing something extremely stupid with their AI use.
That's the cost of one employee (and not an expensive one). Do you think it's impossible he's getting more value from the AI than he would from a single employee?
If it's an employee, which I doubt but ok, it's an employee you constantly have to probe, talk to, give instructions to, so, not a really good employee.
Neither of us have any idea whether his manufacturing startup is successful. It looks to be a 5 person team. A google search suggests they had a funding round early this year.
It might, in fact, be that you are more successful with your approach. I have no idea. Congrats if so.
I mean, he must be pretty successful to be able to burn 10k in tokens. If he is successful because or despite of the AI, I don't know and I don't really care ? That wasn't the question.
It's not the case that someone successful is doing everything right and someone unsuccessful is doing everything wrong, I'd grant.
But I think it's odd for you to now say it doesn't matter whether or not his AI token use is leading to success or not. I personally think it's actually really important to learn whether/when this approach is smart.
Depends how you use the tokens. It's not 10k, but I have an agent do an investigation automatically when I get paged, so by the time I get to the computer I have a nice summary. But that's costing me like $5/mo at most.
But you could use tons of autonomous agents for all kinds of things. Probably shouldn't, but you can.
I use AI every day, and I use it quite a bit, and I use Opus more than I maybe should, and even then I'm not even at $2000 in tokens this year. I don't understand how people are tokenmaxxing as much as they are.
That seems unlikely given the diverse nature of mutually exclusive opinions that exist out there.
Critics seem to run the gamut from LLMs being incapable of even the most basic of functions to already sentient creatures secretly plotting our destruction with steganographic messages to each other.
It's maybe a bell curve with some wacky at those tails, but there's some fairly significant differences of opinion amongst the positions that are more mainstream.
Just the difference between critics of all LLMs and crutics of all closed weights models are a pretty big gap.
Similarly for those who criticise them for over censorship vs those who criticise them for unrestricted generation.
> And this is where the value is for me: I can simply make things higher quality than I could do them alone.
Yeah, that's the thing for me. LLMs have made my work easier and faster, and they've made my side projects easier and faster. I think there are very sensible and valid critiques but so far the tool works for me.
Same thing here. I was able to finish side projects on my list and even refactor decades old projects with the help of LLMs. It works great so far and yeah, I can still write source code when needed.
I think folks ITT are trivializing the issue here. Many keep drawing parallels to LLMs being akin to the loom or calculator. That seems intellectually dishonest to me. Those tools were not anthropomorphized interfaces, nor were they instant gratification. To do significant things with a calculator, you still had to know maths, order of operations, formulas, etc. We didn't lose mathematicians as a result of the calculator, nor did the calculator play with our emotions.
I have linked this elsewhere in this thread, but the studies on this matter suggest a worse outcome than just losing hand crafted or "artisanal" software: https://arxiv.org/pdf/2604.04721
> These findings are particularly concerning because persistence is foundational to skill acquisition and is one of the strongest predictors of long-term learning.
> We posit that persistence is reduced because AI conditions people to expect immediate answers, thereby denying them the experience of working through challenges on their own.
This issue is not that we are losing the ability to write good software by hand. It is possible we are losing our internal tools to learn new things by circumventing a necessary part of the process required by our biology.
It seems to me this here is the dangerous 'force multiplier' for destructive outcomes.
It's one thing for a kid to become obsessed with a fictional character. It's a whole different playing field when that character can talk back, remember things, and potentially surface all kinds of subject matter which cannot be predicted and could at times be harmful.
Not even remotely the same level of impact. I think we're almost at the point where LLM education (as in, teaching awareness of the basic tech and its shortcomings) needs to start in Kindergarten.
LLM's are replacing your 'brain' i.e. the deepest most amazing part of you. The potential value of this to business is astronomical!
No there won't be an explosion of new software, a minor blip perhaps and even this one of diminishing returns (each cycle of LLM improvements requiring less and less human input).
If there are 25 million software developers today then the ROI by OpenAI, Anthropic et al pays off if in a few years time only 15 million developers are needed. That 10 million is the ROI and that is likely to be you.
I agree with most of the 'llm's are bad' section. However, I find it extremely hard to believe that llm's will make peoples thoughts sharper and faster. There have already been several studies published demonstrating the obvious. Not hard to imagine that long term, the negative cognitive effects of using llm's constantly will compound and be even worse.
How LLM usage affects your thinking process entirely depends on the workflow you use with them.
I can honestly say that while using LLM my thoughts are as sharp and as fast as they were before, if not sharper and faster. But my workflow is that I watch the LLM as it rips, have verbose output turned on so I also read Claude's thinking process (and lots of interesting information is hidden there).
This helps me learn new stuff I wouldn't think of. Helps me stop Claude if he's veering off track. Makes it easier to proofread what's Claude doing and why step-by-step and easier to spot any mistakes (because they often come from wrong assumptions you wouldn't see if you didn't see the thinking process).
If you just let your agent write thousands of lines of code, reviewing them is painful. If you also run multiple agents for multiple tasks at the same time, your focus is divided, so you aren't spending all your brain power on the task at hand.
i was certain of the same worst case scenario until I started to properly read and study the LLMs output. prior to that enlightening, very wise choice, I speedread/skimmed and checked for grave bullshit/hallucinations only.
now i am convinced that it's the same matter as with people who forget most of what they learned after the actual exam vs people who go nuts and can't progress if they didn't understand a foundational part of a formula, fact or theory.
The LLM critics are right, and that's why I try not to use AI in the ways critics point out are bad.
I use AI to code tools for myself, but I don't pretend anything I make is production quality. Duct tape engineering has always been a bit sloppy, and AI just made it faster.
I use AI to troubleshoot issues and plan out strategies, but I basically consider the AI draft of anything to be "draft 0", and use it as a framework for writing my own works for a real first draft of anything I write that will be read by other people. Sometimes the AI spits out a perfect paragraph that I might copy, but I don't ever blindly trust it or let it speak for me. I also double-check everything it says that I don't have existing knowledge of, rather than trust it to be right.
AI images, video, and music are all entertaining, but I only generate these things as a form of self-entertainment and maybe online meming. I could never in good conscience pass these creations off as my own, or publish them online on a personal or business website when something non-synthetic would suffice.
And I am never personally confiding in an LLM like it were a person. I have had it help me brainstorm options for office politics stuff, but I'm not about to ask it for relationship advice or to be my friend.
I do love that it accelerates the tedious stuff, and helps me learn new things pretty quickly if used right. It has definite utility. But I am always really distrustful of it. Sometimes at work we are asked to share how we use AI, and I have actually refused before, on the grounds that I may have found a useful way to use the AI, but I am worried that others will use my same method badly (e.g., not verifying eveything the AI says first), and I would rather not share.
It's like I have a finicky gun. I might be comfortable shooting it since I know its quirks and how to keep it from accidentally discharging, but I'm not loaning it out to anyone I wouldn't want to accidentally shoot themselves with it.
> "Last but not least, even when just researching with LLMs, they have the natural tendency to silently sneak in the thoughts of the majority of the training materials, or sometimes even the political convictions of the ones who created the model."
> "Yet I still write all of my texts with LLMs"
So I'm guessing the author is actually ok with the point they put in the "LLMs are bad" part of the article?
As long as, after it writing his texts, he is reviewing and editing them, I don't see the contradiction there. And I'm somewhat anti-AI (if I don't care enough to make the effort of writing it myself, why do I care for it to be written at all?).
The implication is that LLM critics don't use LLMs at all, or that the author is not an LLM critic, but both of those things are incorrect. We are very good at inventing entire people out of single opinions we read, and the AI arguments are maybe the best example of that I've ever seen in my many years watching internet arguments (not least due to the expansiveness of AI, and the sheer breadth of pros and cons it holds within).
Someone says they like hip hop, and you imagine a young urban male with a baseball cap on backwards, how they view women, that they probably don't like musicals, something like that (not saying you do, but as an example). More than simply the thing they said. You say "I like or do that thing", and instantly get hit with "oh, people like you always complain about X but isn't it weird you never mention Y".
At least that's what I thought of when I read that; because answering in place of other people is another thing I find annoying. So I'm speaking for myself, but with some optimism I may have guessed the intent of OP correctly :)
>as a senior, you don’t need juniors anymore. The mundane tasks, at least I find that a lot of people agree with that one, can be fully outsourced to an LLM
Master craftsmen didn't take on apprentices to give them chores.
A big part of the value to the tenured professor is not having to teach an extra couple of sections of CS 101. The other part is plastering their name on a bunch of extra papers to game their h-index :D
I don't want to be too flippant here, but senior engineers generally gain very little from training juniors, not least because promo processes suck at basically every tech firm, and so the moment you train up a junior, they are going to leap to another company rather than go through promo...
Master craftsmen paid apprentices almost next to nothing, and they were often contractually guaranteed to stick around for many years, so the teaching was a kind of wage and also a cost that could be recuperated later on. (The apprentice even often had to pay the craftsman to take them on.) None of those things are true for junior software engineers, who are paid to contribute and can leave at any moment. Also, yes apprentices often had to do chores. It is just not analogous at all.
Not to mention, the physical trades are not stopping taking on apprentices because... LLMs can't build a house? Not even the mundane bits.
It will be interesting how to get the financial incentives to train juniors in knowledge work to line up with the fact that every new generation needs to be trained.
LLMs are awesome, but sometimes we'd be better off with more friction.
This is because.. sometimes an idea is just bad. I learned the hard way the other day when I filed this PR against prek: https://github.com/j178/prek/pull/2302
My idea was bad.
The implementation was meant to solve my problem. All it did was expose more problems and waste my time and the maintainer's, and god knows how many tokens from Claude and Codex. In the end the PR was useless so I closed it.
The maintainer and I would have both been better off - less time wasted - if I was forced to forever wonder if the idea was good, letting it bounce around the back of my mind.
>If you have thoughts, they come out sharper and faster.
No, they come duller and increasingly dependent on the LLM doing them for you.
And that happens to the generations that learned to code earlier without the LLM. What would happen to the later ones, those getting in uni after LLMs emerged, that would never get a real feel for the basics to begin with?
The one thing that had me reading to the end was the mention near the top of the environmental impact of LLMs, but he never got back to it.
I'm currently writing an onboarding doc for my team, encouraging LLM use for some tasks. (OK, well, I'm actually procrastinating by reading HN).
At the same time, I'm in a darkened office with tinfoil on the windows and a fan pointed at me because it's hell outside and it has been for weeks, and every year it seems to get hotter and hotter and we have longer and longer heatwaves.
This seems ... discordant, at a minimum.
Really, _should_ we be using these things to speed up, say, dependency updates if the cost is the planet? I wanted to know what the author thought about that.
I won't belabour the point, but I think someone needs to correct their misconceptions. You're comparing a single, average person's usage of AI in a day -- not the billions who actually use it. Then you are comparing it to an American's two-hour commute in an oversized SUV. (A commute on the Tube costs about 0.02991 kg per passenger, per kilometre.)
What about training those models? Or the usage of all of the data centres? The projections are that by 2028 a fifth of all energy consumption in the U.S. alone will be for AI.[1]
This is actually a good link. Thank you. It's usage that is the big driver, and LLMs are a big driver in emissions and other damage to the environment.
What a nitemare to read. I stuck with it but wished after I didn’t. Reminds me of a 2.5 star movie that I watch to the end but am so annoyed as the credits roll.
Yes there is a dissonance, using LLMS just took their dot point notes and spread them thin. I personally would prefer just the notes and some personal anecdotes to tie the notes into a cohesive whole. This, this approach here, is not better.
It depends upon why you have issues with LLMs. If you’re just concerned about quality then sure the dissonance isn’t intolerable. If you’re concerned about their ethics then this becomes a much more challenging position to have.
But that's the thing - people can live with the contradictions. You can find people at a gas station who will totally agree that air pollution and CO2 emissions are killing people, but still fill up the tank.
My feeling is that LLMs are an interesting technology. I don't feel currently threatened by them, and they're handy in the hands of people that can use them with discipline. However, I think the people controlling and creating the LLMs are by and large scumbags who want to gamble with everyone's future. It's hard for me to look at their public statements and the actions of people like Altman, Amodei, Musk, and come away with any other conclusion than that these are fairly awful people that have bad intentions. Their message is essentially "we want to summon the machine god to destroy your livelihood because it will enrich us and give us power over the world, and there's a really high probability it gets out of control and causes massive problems [that we're completely going to not do anything about]. But we really like money! And we have bunkers!"
I suspect a lot of people are in this boat. We both use it and cheer against it. But it's not cognitive dissonance, it's just a recognition that there's a useful tool being controlled by vile people.
> I think the core issue here is trust. You should never trust random people on the internet anyway. But before LLMs, there was this base thing: creating a proper PR with proper descriptions would require at least some human time, so it would keep trolls and low quality submissions out. Or at least you could easily filter them out within a couple of seconds. So even if a new person came in, you could trust that this person would have at least spent a couple of hours on that. And then it was probably worth taking a closer look at it.
Ding ding ding. This is my biggest gripe with AI. Even the SEO blogspam, the fluff in front of every recipe, yarnwork or DIY instruction, it all was clearly written by a human. Someone had invested time (and money) in getting something in front of my eyes.
But now, it's all just slop. Everywhere. And hell I'm tired because the onslaught breaks my trust filters.
Maybe I think this is an age thing. Boomers? They trust everything written down somewhere. No matter what, and no matter if they didn't spend half my childhood to "never trust what people write on the Internet", and now they fall for scams left and right. My generation as said grew up with this "never trust, always verify" thing. And the younger generation? They DGAF about anything any more, all they care about is trying to survive.
> And b), the teaching, aka “How do we teach new people?”: previously, there was this balance aka “the junior does some pretty mundane tasks, but for this the senior reviews it together with him and helps him to grow”.
GOD YES YES YES THIS x1000.
There is barely anything more rewarding than teaching someone something, to watch the other person grow - and eventually surpassing your own abilities. That is when you know you did right and well. My wife is the best example, she started out at "can you help me with Excel", and these days, she pulls off stuff that would make more than a few finance people blush.
> There is barely anything more rewarding than teaching someone something, to watch the other person grow
I think many junior devs (or aspiring junior devs) look for exactly this experience. This is a matching problem we haven't solved yet. Is Open Source the solution ? I really think it has to be solved if we want truely reliable software in the future.
There's a tangential problem - companies do not want trainees or juniors any more, they (usually) cost more money than they bring in for half a decade if you don't fraudulently bill your customers... something the "consulting" industry is infamous for. And when the juniors got enough experience to be considered intermediate or senior, they jump ship to get a larger pay rise, leaving you with the need to hire another senior.
The entire economy is broken due to the focus on short term quarterly result instead of the health of a company in 5, 10 or even 20 years.
I think that the use of LLMs to expedite the learning process (especially when it comes to learning new things) is extraordinary and should be practiced by anybody with the means to. Think about how far we as the human race have come. Everything eventually plateaus and either flatlines or else regresses.. With the use of AI we can expand our ways of thinking into a new tier we would have never been able to achieve otherwise. It goes without saying that people who refuse to use the tools we've been given are close-minded and susceptible to fail in the new age that we are truly lucky to be a part of. And its here to stay, whether you like it or not lol
> And then there is the situation about junior engineers. There are actually two different points in there: a) you cannot trust the effort behind your junior’s code anymore, and b) seniors have no incentive left to teach juniors.
If you are bringing in juniors with the expectation that they won’t produce any value for some time then it might be better to block them from using LLMs.
Do this and you will get a better understanding of where they are in the process and they will probably learn the ropes more quickly.
> If you are bringing in juniors with the expectation that they won’t produce any value for some time then it might be better to block them from using LLMs
A) I think many companies no longer view juniors as an investment, they want them to be productive. They view juniors as "cheaper almost-seniors"
B) Blocking them from using LLMs would mean acknowledging that there is a downside to having your employees using LLMs. If they prevent your juniors from improving, what negative effect are they having on your seniors?
I'm saying it in jest, but it's also a bit true. Not necessarily because we use it any differently. But because my use of AI saves me time. But their use of AI adds more to my plate, no matter if it's slop or not.
I noticed very recently I actually prefer reviewing other people’s generated code over handwritten. There’s no ego involved and it usually does a better, more thorough job. We have enough infrastructure that by the time another human lays eyes on it for the first time each pull request has already been through 3+ rounds of automated review. Being the only specialist in my area on the team used to mean more coaching and more involvement in code review. Now my coworkers are sending me idiomatic solutions with modern approaches that I approve right away or with hardly any comments
So much this. When coding I use my LLM to hone the code to perfection before sending it to review, including running AI reviews on it multiple times and fixing obvious bugs so that reviewers have less work to do. I also test all my code manually (LLM can be helpful by giving me a list of test scenarios to consider). When reviewing PRs I carefully read the AI reviewer output, consider which points are relevant, which are irrelevant and only post the relevant issues when requesting changes.
I have coworkers who routinely post PRs with comments stating "all tests green, ready to review" and then I actually start the application, test it manually and it doesn't work. And don't get me started on reviewers who post the raw AI output of their "reviewing skill" and then I have to read through 30 paragraphs of "findings" most of which are irrelevant (just today I got the "valuable" feedback of "View is @api_view(["POST"]); no test asserts GET/PUT/PATCH/DELETE → 405")...
The critics are absolutely right, LLMs have a lot of faults. But they also have a few great benefits. I use them every day for the benefits, fully aware of the faults, and I watch those faults like a hawk.
I think I'd put it this way: if I could flip a switch tomorrow and the AI boom was done, I probably would. The negative effects just super duper outweigh the fact that coders are more efficient. But since it exists, I'm going to use it. Hacking on my own small projects isn't using that many resources. But the benefit simply isn't worth the cost, as much as I personally love using LLMs.
Aren't all conferences just huge piles of cognitive dissonance?
At least the "industry" ones used for "networking" and all that other crap. Where's the news here? And why does the author not take a step back and realize that there is none?
> I strongly agree that written text should be from humans to humans. Yet I still write all of my texts with LLMs. And I don’t find that contradictory. What distinguishes “AI slop” from “good writing” is whether a human has put thoughts behind it. And you cannot outsource thinking.
Writing is thinking. If you're outsourcing your writing to an LLM, you are shortchanging yourself by skipping over much of the refinement of ideas that the process of writing provides.
I wish the author had clarified how he uses LLMs for writing. It's perfectly fine to have an LLM proofread and fix your grammatical errors--that's a mechanical task. But I think it's gross when it starts putting words and content on the page.
The desire to categorize people as right or wrong should be resisted. It's tangential, and distracts by feeding the tribalism part of the brain instead of considering individual things that are explicitly stated.
If the critique itself turns out to be wrong. Using LLMs does not equal endorsement nor does it make valid points to critizise disappear. "using LLMs" has no clear definition and can still be in line with the critique.
Such concerns were present, and rightly so. Concern and critique is important to express so that we are sure new tech is useful and negative side-effects are realised and appropriately mitigated.
Job displacement: When the Internet first arrived in a significant number of non-technical lives, there were concerns about research jobs and library jobs. In fact it increased the need for that sort of thing so increased that sort of job, and even if it hadn't the access to information for the average person would have been worth that cost. As remote work grew initially, then massively in 2020, there were major concerns about that sort of thing again (people working in Manchester on London wages, going into the office very little, it being even easier to outsource jobs to countries with cheaper workforces, etc.). These concerns are still being debated today, and that is quite valid.
Ethics: Hell yes. And again, it is good that we are still refining what we think is acceptable to this very day. There have been ethical concerns almost from the start of the Internet in the public mind, and they have only increased over time as new parts of it take shape.
Environmental: That is a lot less relevant, as the Internet itself didn't directly have the sort of impact that LLMs are having. There have been local concerns though, over the building of data-centers and the power infrastructure to support them, disturbing land to lay cables, and so forth, and again these concerns are still relevant (though have somewhat folded into just being a concern about LLMs & related ATM as they are the main reason for the current rapid growth).
Skill Atrophy: The Internet never really threatened that, except maybe a small amount of concern wrt certain research roles, it increased the base level of those skills in the general population much more than it had any negative effects (but people did raise the concerns). It gave better access to training and knowledge, but wasn't actually doing things for us in the same way LLMs and the agent systems linked to them are being used, you still needed to learn from that information/training in order to complete your task.
The advent of the internet was collaborative and based on introducing shared protocols for a couple of decades. It deserved criticism when globalised capitalism got involved, and monopolies started forming, leading to rent-seeking, excessive centralisation, and enshittification.
The impact is that the internet has a fraction of the value to improve people's lives as it should have. It is a very poor free market, incredibly poor competition because of lack of standards and protocols and interoperability. People's minds are ground down by social media, search engines don't work well any more and so on.
So yes - every new technology deserves many criticisms, so they can be addressed, and as a society we can gain the benefits of that technology and minimise the disadvantages.
The printing press lead to copyright, public libraries, universal literacy... All things which are now widely celebrated. They took centuries to work out, and are all government and regulatory intervention to fix problems critics noticed and campaigned about.
AI is the same, only it is at risk of moving much faster and having a much large negative impact before society reacts.
So no, most of the criticism of LLMs are not wrong - they are correct, as are the people saying the technology of LLMs is useful to people and the economy.
Critics are friends of a new technology - without responding to every criticism in a significant way, AI will rapidly lead to a Butlerian jihad. If you like AI, you should love criticism of AI even more.
Because it's a machine, not an oracle. If you "hold it right" you are more productive. When you catch them being "dumb" you're reinforcing your own deep knowledge on the topic. When it is correct, you're either learning something new or your task is complete. You are still at risk when you know little and trust it to work alone.
Good article! This matches how I feel about the situation.
It's really not incongruent to use LLMs and be in awe of their frankly incredible capabilities while at the same time recognize the risks and frankly real damage we are already seeing to junior training and hiring, open source communities and (in my opinion) very soon the entire fabric of our society.
I respect that people don't want to use agents themselves for whatever personal reason.
I respect maintainers not accepting AI-authored contributions. It's a tradeoff between progress, growing new contributors and maintainer sanity. Though I do feel that categoric opposition to anything AI will likely be futile in the mid-term.
I respect people pushing for regulation of AI or a global pause or whatever.
I don't particularly respect people dismissing everything AI authored as slop. Categorically refusing to read an article because it contains em-dashes or the term "load-bearing" is silly. While this is slowly changing now, many people are still in complete denial as to what the frontier AI is capable of.
Love it, hate it - I don't care, but at least respect it, goddamit.
You're all in denial when you criticize LLMs. It's not necessarily that the criticism isn't true. It's more how self assured the criticism is. That's the biggest problem because AI is a moving target. It is getting better, and it is getting better fast. A lot of the criticism can become outdated in a year or six months. The change is happening in front of your very eyes and yet you can always reliably come on HN and find some sort of self assured criticism to say AI can't design, AI code must always be reviewed. Blah blah blah.
The big thing people used to call AI was that it was a stochastic parrot and all it did was summarize things. Clearly. None of this is/was true anymore. And very likely all the current criticism will be eliminated soon and we have to find new excuses about AI that makes us feel we are superior.
The status quo is about to change. Every 6 months. And you will always think of yourself as superior to LLMs. Your current criticisms will evolve as most of them will be rendered not true pretty soon.
Nope. Not true anymore. Right now the bottleneck is the review because code comes out so fast there is a measurable and adjustable trade off that can be made.
If you review all code, your output will be slow. If you review less code, your output will be faster at the cost of more bugs in production. Bug rate will never go down to zero whether you use AI or not.
That is the trade off, if you review everything then output is really slow. If you start only reviewing certain types of code like model changes, database changes. Or only backend code and not frontend code, you hit a sweet spot of speed and reliability.
As LLMs improve the need for reviews becomes less and less. Companies who don't adjust are just slowing themselves down. That is the trend.
Anyone who lets the LLM put out unreviewed code is shockingly negligent and bad at his job. It certainly might be the case that many people are bad in such a way, but that doesn't make it acceptable.
The above code, if a and b are integers have 2^64 test cases for full coverage. Code being much more complex then that on average will have a magnitude more test cases needed for full coverage... likely more test cases then all of the code humanity has ever wrote since the dawn of programming.
Testing is a statistical sample. You sample a small subset out of the full domain of possible tests and that sample you hope is somewhat representative of the actual population of tests. To put it in a sentence: If my suite of tests is correct then it is more likely that the entire full universe of every single possible test for my program can all be correct.
Realistically though the universe of all possible tests is so large that it's basically impossible for any sample to be a good representation of it. A suite of test does help up confidence that things will be MORE correct, but the sample size is so small it is almost guaranteed that you missed something.
I think both the LLM critics and the LLM advocates are right.
Even this article has some cognitive dissonance in it. What it really comes down to is how much you trust your own verification process. The branches of questions an LLM generates are still trapped within the biases of its training data. Of course, the authority to craft that initial prompt, the very first question, comes from human experience and learning.
But I think thought itself is the easiest resource to outsource. People say the human did the thinking and the LLM just amplified it, but the truth is, the LLM outsources the thinking. Otherwise, when the result is good, people say "human thought was present," and when it's bad, they say "human thought was absent." But a part of the actual thinking really is outsourced. The alternatives, the counterexamples, the sentence structure. In programming terms, the reader's experience gets outsourced. When you write a blog post, you find yourself thinking about how to make something you understand easy for someone else to understand. With an LLM, that part gets outsourced.
But at the same time, I don't get the argument that you shouldn't use it at all. We don't "think" about everything. We have limited cognitive resources. So we study deeply the things we care about, but for the things we don't need, we mostly leave them to "common sense" or prejudice. We just skim the surface.
I think of "common sense" as "the largest collection of prejudice." Because what we call common sense usually just amounts to surface level knowledge, the kind of thing we know just enough about to get by.
That's why I think LLMs are good. The reason is simple. I don't think deeply about everything in the world anyway. For everything else, I'm buried in some kind of bias. You see it on HN all the time, right? People fight over some technology, but they often don't think about its internal structure or why it works the way it does. They just treat it as an identity. They fight over a particular language, a framework, an operating system, but they rarely check how that technology actually works internally or why it was designed that way. Why use MVC, why a different architecture might be better for my case, it's easier to just go with what's popular. Put more elegantly, "job mobility" gets bundled in there too. I use Windows. In my country, if it's not Windows, you literally can't do anything. You can't even do basic online banking. From regional context like that all the way down to personal interests, people are bound to be different. So I'm just going to use LLMs. The most common excuse you hear around this is the whole "reinventing the wheel" thing.
So yeah, I'm going to use LLMs. Because I recognize that I bias myself toward only thinking about what I want to think about. And I know that bias isn't cognitively healthy. But on the flip side, I think what the world values, whether it's knowing a lot or knowing one thing deeply, is going to change.
Honestly, I don't know what's right. I think both the advocates and the critics are making valid points. I respect the people who don't use it, and the people who do just have their own workflow. There's really no reason to fight over whose workflow is superior.
Criticizing use of agents for skill atrophy is valid, it definitely atrophies blank slate coding ability, though I don't think it atrophies engineering abilities unless you just YOLO all decisions to the agent. The data center/oligarchy complaints are also valid.
Saying agents produce shitty code is a bad argument though. They produce shitty codebase organization, but at a micro level their code is solid if not elegant. If you let them turn your codebase into a spaghetti mess, that's on you.
The trust problem feels more important than whether the code was AI-assisted. A small, reproducible change with clear tests is reviewable; a huge opaque diff is not, regardless of who typed it.
This is like meat eaters who wish they were vegan but state that they're not mentally strong enough or whatever. Incredibly annoying. Either shut up or convert.
> LLM’s amplify what you already have: opinions, structure, frameworks.
So far, so agreeable, but…
> If you have thoughts, they come out sharper and faster.
I can’t help but wonder whether constant use of “agent” harnesses will lead to an atrophy of the software engineering (or really any field) muscles.
Actual muscles need exercise to stay in shape (let alone grow), so does the brain. Can we really be sure that thoughts, opinions, taste will still come out sharper and faster after five, ten, 20 years of using these tools almost every day?
Conversely, I also am a user of LLMs (true shocker these days, I know), and am noticing a speedup in areas I was already familiar with, and a quicker introduction to new ones. The obvious benefit cannot be denied, and doing so regardless makes you look uninformed. [0]
So what’s the ideal “middle ground” in this situation? Stoically continuing to sharpen your skills on your own, but risking being left in the dust productivity-wise? Or taking an “agent first” approach and trying to learn and improve more only on the side, as more of an afterthought?
[0] Excluding people who don’t want anything to do with LLMs out of moral principle, which curiously just like the overarching topic I also both respect and understand, but on the other hand don’t do myself.
I will just point out the benefit is not as obvious as you think. Developers have consistently overestimated LLM productivity gains, which still seems true for agentic AI: https://metr.org/blog/2026-05-11-ai-usage-survey/ It is particularly striking how similar the results are to LLMs before agents.
Along with the total absence of long-term data, I think the benefit can be (weakly) denied. Maybe not in the employmemt marketplace, but certainly for myself.
> I will just point out the benefit is not as obvious as you think. Developers have consistently overestimated LLM
I think there are two different claims here:
- developers overestimate productivity gains, which is a solid finding in many of these studies. Skepticism of extremely large productivity gains is warranted and I flatly disbelieve "10x uplift" claims.
- LLMs give no productivity uplift at all, which is much harder to defend. A repeat of the famous METR RCT study did find evidence of improved productivity, and this seems to align with the experience of many experts I trust.
Specifically my claim is "the relatively minor productivity uplift I would personally get out of agentic development is offset by the high cost, along with unresolved questions about long-term code maintainability, so I am not convinced that it is actually beneficial."
IMO the bigger problem is that ~1.5x individual dev productivity uplift seems to translate into 1.05x uplift across the team. People have been waaaaayyyyy too overconfident about this stuff.
I am both a career developer and experienced team manager. from first hand experience the 1.5x im getting from AI is not flowing down to my team / org because why would i output 50% more when the pay environment and leadership are already underwhelming. That additional 50% productivity goes completely to side projects built on my second computer between 9-5 tasks
Actually, the METR report speculates that some of the overreported productivity uplift comes from grabbing unnecessary low-hanging fruit, things like "oh I'll make a web dashboard to keep track of this stuff // wow that would have taken all day without Claude!" But in the olden days they would have just used a notepad. Yet psychologically they built a real thing and saved a lot of time.
True I have made a few low tier apps that just hit apis that were previously obfuscated deep in menus that have had an outsized impact
> But in the olden days they would have just used a notepad. Yet psychologically they built a real thing and saved a lot of time.
There was an old Onion article about the monetary savings from buying clothes on sale as computed by (a) the women buying the clothes; and (b) their male relatives.
That’s what I’ve heard from my dev team too. They’re using it to give themselves free time while still being on the clock, not to produce more output for the company. Roughly thinking about hours spent on projects I think have gone up per task, the opposite that should be happening.
I'm finding it almost impossible to fill that free time with work, unless it's just reading emails and chat messages.
I can context switch between two or three chats, but doing so speeds up my agent use at the cost of making reviews and discovery harder, so it might come out in the wash.
>I can context switch between two or three chats
I'm finding this is a skill which I'm slowwwwwwly improving but, for now, my level means I tend to miss mistakes I'd notice if just working on one task. I also had to make accommodations to the way I work to make it achievable, including telling my employer that a 500 GiB disk just isn't enough any more when I need multiple builds in parallel.
Your comment makes total sense to me but generally, in regards to productivity gains from AI, I can never understand where these are realized for people. Maybe I'm just a laggard but never found myself 25%-50% behind on anything or that much more work/items/tickets available.
Great point. Devs have relatively no incentive to be more productive for their orgs, and all the incentive to be more productive on their own personal work. The AI benefit to devs is real, just not for large enterprises IMO outside of automating clerical/mundane work.
> Devs have relatively no incentive to be more productive for their orgs, and all the incentive to be more productive on their own personal work.
AI definitely supercharges the side project myth, but I think we’re going to see waves of developers quitting or losing their jobs because they’re chasing dreams of running a side code project that isn’t going to turn into a business.
The incentive to be more productive at work is that they get to keep their job and not be replaced by a cheaper junior. I think we’re in for a reckoning across the industry as companies realize that there’s little difference between a lazy senior armed with Claude and a halfway motivated junior who has aspirations of growing into something more. The latter costs less and might grow into a better dev.
If the salaries fall then there would be no highly motivated juniors.
Lazy senior vs motivated junior tension has always existed. I was one then became the other. I don’t see llm changing this dynamic!
> because why would i output 50% more when the pay environment and leadership are already underwhelming
The productivity uplift measured in some of these reports is from more tasks being done, not from developers giving themselves more idle time during the day.
If we do see job losses from AI, my feeling is that it’s going to be concentrated in people with an attitude like this one where workers try to remain anchored to their old delivery pace. It’s becoming easier than ever to replace a worker in that position with a lower paid junior equipped with Claude who has an interest in learning. There’s no reason to keep a higher paid worker around to copy and paste between Jira and Claude.
thats absolutely true regarding wysiwyg projects like a small website. aside from temporary cost savings, systematically ignoring institutional knowledge in favor of mass output is not gonna do humanity any favors ..
Yeah institutional knowledge is hard to capture, even more with agentic coding.
I just compressed what would take a team a few months to deliver in one or two months, the amount of gotcha I spot from the existing system we work on everyday is staggering.
How to deliver and keep these feedback is difficult, the other teams are overwhelmed by other tasks, so any tickets or communications will be forgotten in a few months.
And as an external ressources allowed to use frontier model, I have a hard time to keep up with my own pace, and all that institutional knowledge will be lost both by my employer and client.
Could it also be that the phrase “using an LLM” is loaded in itself? Who is getting more velocity out of it:
- chat box interaction
- vendor harness user (the cursor / antigravity crowd)
- SDD Claude users
- bespoke harness creators who let it goooooooo
I mean we already know who’s using all the tokens and driving enough cost to make penny pinchers consider removing other “costs”
I have seen no productivity uplift at all from LLMs. They are at least about neutral these days (they used to be a productivity drain), but no gains at all. By the time I get done reviewing the code to make sure it hasn't done anything crazy, I've spent the same amount of time I would've taken to write the code myself. The only people I personally know who claim productivity gains are getting those gains by completely disregarding quality, or understanding the code, and just YOLO letting the LLM do everything without checking. I'm not willing to do that.
I find Claude is shockingly slow. If it were genuinely instantaneous, it would be worth it, but I often find myself twiddling my thumbs while waiting for it to cook.
Yeah that's why you do multiple tasks and projects simultaneously.
Humans notoriously suck at context-switching. I doubt you can really do multiple projects simultaneously without compromising quality.
You're already vibe coding, how worse can it get?
That sounds like a really efficient way to very quickly produce a mountain of crap.
Maybe we can make an analogy with taking the plane vs driving to your destination.
A plane literally goes 10x faster than a car, so a 5 hour drive becomes a 30 minutes flight. But if you have to drive to the airport, arrive early, pass check-in, security, boarding, then pick up your luggage on arrival, rent a car and drive to your destination, you may realize that your 30 minute flight took you more than 5 hours in total.
You also have to consider the Amdahl's law: a 10x speedup on 10% of the project is just a 9% speedup overall.
> You also have to consider the Amdahl's law: a 10x speedup on 10% of the project is just a 9% speedup overall.
This is totally true for me. I'm working on more features/bugs at work and getting the coding done faster with claude but the coding part is at maximum half of the process and usually less. The rest of it is clarifying the requirements from our product team or given that we're a very legacy place, what the existing state of our infrastructure and other systems are for the new changes to fit in to, etc.
> You also have to consider the Amdahl's law: a 10x speedup on 10% of the project is just a 9% speedup overall.
It's a 10% speedup, unless you think that a 100% speedup would reduce completion time to zero rather than cutting it in half.
Assuming that "10% of the project" is measured by time requirements, when you get a 10x speedup on that part of the project, you'll end up completing 100 units of work in 91 (formerly 100) units of time. Your work rate is then 100 / 91 = 1.099 times what it was before; that's an improvement of 10%, not 9%.
(Sanity check: suppose you have a project that will take 100 days. That project is reassigned to someone 10% faster than you. They will take 100 / 1.1 = 90.91 days to finish.
If they were 9% faster, they'd take 100 / 1.09 = 91.74 days to finish.
Does a savings of 9 out of 100 days of work look more like the project that saves 9.09 days, or the one that saves 8.26 days?)
> LLMs give no productivity uplift at all, which is much harder to defend
It’s not really hard to defend. Because when people says that productivity is uplifted, they are talking about amount of work, not the ROI. That’s why you keep hearing about LOC, amount of PR and prototypes, and the time taken is actually “time to PR” and not “time to production + time spent on bugs”.
There's another even more abstract liability on the balance sheet here for "deprecated LOC" or something, where the thing they built gets scrapped because no one wants to untangle the rat nest, or the "documentation" was so incoherent that it all had to be rewritten and reviewed.
Coming from legacy enterprise I am using claude to do exactly this for some of my team's more bizarre and poorly documented "inventions" that started out as their pet project and then became part of our official workflow.
In the former case, which seems more likely, does it not fall to reason that like any tool there'll be some that will benefit from it and some that it just won't work as well for, but are equally as good at their job? Like that neovim, unless your an absolute zealot you wouldnt insist that everyone use it, or hire someone based off it, but those who do work well with it do find it to be a boost to productivity.
Also importantly, can it be a neovim? Neovim hasn't had literal trillions invested over it in the space of just 5 years, local llm's aside can those smaller productivity gains justify the huge investment put into them, and which will continue to be needed for further model development?
Developers are cashing in on the productivity gains. Meaning instead of using the increased productivity to do more work, they just become lazier or do fun irrelevant side projects instead, where as before there was just not much time for such things. I have definitely procrastinated on work simply because I know I can swoop in with an LLM and do it all in 15 minutes, whereas before I would have spent a few hours.
This is how LLMs can both result in greater productivity yet still not appear to yield much more benefit than the pre-LLM era.
And there is no benefit to the developer for doing the work first then just sitting idle. That’s how you get people putting more things on your plate.
I would frame it differently ... Last year I was burning the fuck out hard. This year I am not.
My gain is that I produce good work in a normal workday and I am absolutely not writing code to 4 am anymore...
Overall for me it is a win for me and my company.
Some unsolicited, probably obvious advice for you for the future: never do that again.
Managers will abuse you if you expand your workday to make sure deadlines don't slip. They will see that the work is getting done and see no urgency to hire more help.
You have to surface capacity issues to management, learn to say "no" diplomatically, and/or "I have capacity for 2 things out of these 5, prioritize which ones you want first."
The LLMs are better at enforcing their boundaries than developers. Run out of tokens? That's it. No more code. It doesn't care how much the company needs this for the customer call tomorrow.
I feel that’s the opposite of the LLM-business models.
If I run out of tokens I’m encouraged to spend money to buy more tokens, Claude has an overage setup, and Anthropic runs occasional token sales.
So, to that example: the more often customers are hitting token limits under duress the more likely to open up the wallet and pay to finish.
So, its drug dealer economics.
Run out of your fix? (Tap tap) Buy some more!
Odd, it was the opposite for me. Last year, no burnout. But then agents got good and everyone wanted more code more features faster and faster. I got burnt to a crisp. Now? I don’t give a fuck. Ok people want to push tons of code straight to prod? Go ahead, looks good to me… no need to review it if it’s with the latest frontier models right?
Exactly, I finally migrated that old service to typescript and cleaned up some really annoying tooling, for example. I’m more productive and can get quick fixes out very quickly. We’re also seeing agents help identify incidents faster than human investigation can. But big structural things that are hard to integrate still take lots of time.
A few months ago, I heard a keynote speaker claim that LLMs did not give any measurable productivity gains, except in two fields: customer support and software development, where they resulted in about 30% productivity improvement. I forgot if he presented any sources, and if he did, I forgot those, but it sounds plausible to me.
But instead of productivity, I'm much more interested in using it to improve quality. You've got a tireless reviewer who is always ready to review your code and catch any gaps.
I don't know how they're measuring productivity for customer support, but it certainly isn't in terms of customer satisfaction. People have never been happy with chatbots when they're trying to reach a human.
We are getting ai responses from cloud providers. From both aws and google, my teams have received obviously ai generated and non-applicable advice.
The latest was that we need to write in object pooling in their client code, an option that doesn't exist yet in _their_ client code. Like, yo, that is a you-problem on performance bottlenecks; we expect you to provide solutions.
[flagged]
I think the real disaster is that once you let the LLM work on a project for a bit, you start to lose understanding of what exactly is even happening under the hood in the project. You can take steps to mitigate this, but agents don't exactly encourage the behavior required to maintain a good understanding of what's going on.
Like becoming a manager?
When a person becomes a manager, they do or do not have enough time and expertise to review all of the code that they trust the team to produce.
Managers usually get into automated testing; unit tests, integration tests, acceptance tests, and maybe also BDD syntax
Managers and developers are responsible for setting a test coverage threshold for merge approval.
If there is 100% branch coverage test coverage for a codebase, what would coverage-guided fuzzing or property testing find? If there is 100% branch coverage test coverage for a codebase, what is the value of spending resources on formal verification?
How does the value of LLM-produced 100% branch coverage compare to no-LLM 100% branch coverage?
I've never really understood this argument. If someone's a manager of an incompetent team, no amount of management skill will save the quality of the resulting software. I don't think "just treat LLMs like smart junior developers" fixes this, because well-functioning teams usually also have senior developers to keep things on track. Like, if we handed a team of genius-but-junior developers to the best "people person" manager in the world (i.e. who doesn't actually read the code), would we really expect decent results? Even if the manager tested the code by hand? I honestly don't think so, at least once the software gets past a certain (fairly low) threshold of size/complexity.
If you have a "this is an incompetent team" attitude, that's what quality software you'll get back; whether or not there's LLM use.
If you have a "I trust my competent team to write good enough tests and test coverage isn't my responsibility" attitude, that's what quality software you'll get back.
There are people producing good and excellent quality software with LLMs. Presumably you must discard low-quality code in order to maintain quality.
There's certainly a limit to code quality with current models. On number of lines of code per unit of time, LLM tools certainly already win.
Can costly automated code review for PRs catch most of the problems before they're under consideration for merge?
For example, the vscode repo has extensive copilot integration. Every PR gets auto code reviewed. But with their tokens or the contributors'?
If I take poor quality code (AI-assisted or not) and spend a few hundred dollars on tokens for a next gen model and agent to get to 100% coverage and review for security bugs and CWE common weaknesses, what quality code will I have without refactoring with proven patterns and type annotations and polishing docstrings?
I don't think this argument holds. The OP made the very good point that most quality software teams are composed of both seniors and juniors, with the seniors (and even staff members sometimes) drawing the lines so that the system stays on rails.
If you're a manager now and your "team" is a bunch of coding agents, those agents are hardly junior engineers at best. It is equivalent or even worse than hiring a team of 3-5 junior engineers and letting them run rampant with your code.
> If you have a "I trust my competent team to write good enough tests and test coverage isn't my responsibility" attitude, that's what quality software you'll get back.
As much as this sentiment is nice, it is completely divorced from reality, unless the competence is verifiably there. If you take a bunch of juniors and say "yeah I trust them to do everything well enough," you're going to have a disaster on your hands.
> How does the value of LLM-produced 100% branch coverage compare to no-LLM 100% branch coverage?
This is such a salient question. Sometimes (definitely not always) the test suites produced by LLMs are so trivial it's scary. Coverage can be an illusion for sure.
I have been seeing such trivial shit getting submitted and deleting them from the test suites, but it's disheartening to see that the devs didn't have the better taste to see those LLM "tests" are so trivial they border on noop before submitting them...
What new or existing metric would better assess test coverage than line coverage and branch coverage?
I wrote a tool called tert - I guess it's called an agent harness now - to run various test runners and log test output and coverage output to disk. FWIU stripping spaces from JSON does save tokens. It seems like feeding coverage lines-missing maps into the prompt results in better output, better LLM-authored tests.
"Refactor these tests for maintainability and coverage. Use fixtures, mocks, and parametrization"
>What new or existing metric would better assess test coverage than line coverage and branch coverage?
Substance coverage - testing the actual logic, edge cases, etc. Not mere lines.
You can write tests that cover every line/branch without actually verifying the code does what it’s supposed to. Relying on (unreviewed?) LLM written tests that provide coverage may not be a great idea.
So, a project can have 100% test coverage (the tests run each line or branch at least once without any errors or failures) but fail acceptance tests.
If there is prompt insufficiency, there is probably acceptance test insufficiency.
A more assuming agent could automatically develop a plan that includes presumptive acceptance tests and request feedback before spending tokens
We try to use types as much as possible. And if we can't, then lint rules, etc. Basically, get everything as far left in the SDLC as we can.
And, if/where we need tests, we write the source so they are few, high value, and complementary. Like actual unit tests, not complex with stuff like mocks just to generate trivial coverage.
Formal verification is always more valuable than mere testing, but it's hitherto more expensive. The thing that ultimately matters is closing the loop: how well do the tests match the requirements, both as written and as unwritten in the mind of the customer?
Working with an LLM has given me a real eye opener on unwritten requirements. It's like outsourcing. "Yes, you've given me what I wrote down, but I never expected you do to it in that way"
Causes of failure; Operator skill, prompt insufficiency, model insufficiency, agent insufficiency, insufficient code review, overloaded merge maintainer
> When a person becomes a manager, they do or do not have enough time and expertise to review all of the code that they trust the team to produce.
> Managers usually get into automated testing; unit tests, integration tests, acceptance tests, and maybe also BDD syntax
I can see managers getting involved into acceptance tests, but never in the other type of tests. And the verification mostly is involved into a quick manual testing/watching a demo. Code is not their concern. When there's a bug, they expect you to investigate and fix it.
Given that it's so easy to generate better than test stubs with a model and manually review what it thinks covers, I think it's fair to expect a manager to run and add to the tests.
If devops has done their job, it should be trivial for a manager to contribute to the tests and run the build on git push (or manually re-run the build with the web UI).
If a manager has deploy rights, they should be able to run the tests.
Managers often still have code and team-lead responsibilities IME. First-line managers, that is.
And then sometimes you report a memory leak and it gets fixed by a VP and you wonder if he doesn't have something better to do.
If the memory leak could cost the firm its reputation with important customers, then I doubt the VP had something better to do.
I think what this fails to cover is that managers rely on other people to do that understanding and keeping things in mind. When there's a problem, the manager isn't just saying to a random person "go fix this", most of the time they're relying on somebody who has a deeper understanding of the problem and is not likely to randomly break something else just to fix this one bug.
Fearless refactoring is possible given test coverage? True in my experience; both with higher-risk human-developed software and with agent-coded software.
I haven't yet made myself learn the new swarm of concurrent agents with different specializations/agent_instructions methods yet.
Are multiple worktrees worth the cognitive burden and merge overhead?
A merge maintainer is always in code review mode
The productivity depends upon the requirements.
If slop is fine (and sometimes it is), the benefits are undeniable. If the dev was the kind that would have produced slop anyway - again, undeniable boost.
If the quality needs to be high I think it actually can slow you down, though.
Agreed, but also to expand, if the dev is mediocre and ai-assisted coding is a skill, the productivity gains are 10x more mediocre code. Since code is a liability, this is not desirable. Hence, mediocre devs being more productive is an underestimated problem of the age of ai-assisted coding. I see this every day.
The result is a whole bunch of dysfunctional systems unnecessarily dislodging perfectly acceptable processes.
"Code is a liability" is the part that usually gets hand-waved, and there's a number on it now. Veracode's 2025 GenAI report ran 80 tasks across 100+ models: when a task could be solved securely or insecurely, the models picked the insecure path 45% of the time, at roughly 2.7x the vulnerability density of human-written code (Java was worst, at a 72% failure rate).
So the mediocre-dev case may be worse than "10x more mediocre code." It's more code that also skews insecure by default, and that cost shows up downstream in review and incidents, not at the PR. Throughput goes up, and so does the liability per line.
<< So what’s the ideal “middle ground” in this situation?
Putting all this in 2nd paragraph so that you can skip it if you think 'coding' is your primary portion of your job.
I suppose I am in a mildly privileged position in a sense that my work is a weird intersection of tech, finance, and comprehension. In other words, I don't code much, but I absolutely benefit from now being able to play with various projects I would otherwise have no business touching without a bigger support team.
I don't want to invoke Accelenrando, but the muscle imagery and analogy fits. I will give an example. I recently decided to pick up Go for a project ( have experience in some other languages, but I will still be starting fresh ). I could have codex build me what I want, but I am purposefully taking it slow so that I can learn the foundation so that I can have a frame of reference ( because I assume it won't be the only go project for me ).
Otoh, most of my one off python scripts I barely even skim anymore. And honestly,that is the part that scares me more.
Why would you be very careful with a one-off script? The only point is the output.
On the other hand, if you actually care about the output, how do you know it's right, unless you review the script? I mean, if all you care about is plausible-looking output, you could have the LLM produce that, and skip the Python script entirely...
Eh, unlike some of my contemporaries, I am not as interested in merely plausible-looking output. I want good output each time, but I am clearly still trying to find a good balance.
>Conversely, I also am a user of LLMs (true shocker these days, I know), and am noticing a speedup in areas I was already familiar with, and a quicker introduction to new ones. The obvious benefit cannot be denied, and doing so regardless makes you look uninformed.
My largest concern comes from something tangential to this: I'm not sure we're all that good at deciding what should be learned and sticking to it.
Silly example: regex. LLMs are, as far as I know, well above the average dev when it comes to writing regex. Regex is also one of those things that for many people goes unused for months, but then you encounter the occasional perfect regex problem, and it's really easy to just lean on the LLM to write the regex for you rather than spending some time tinkering and testing. Regex can be frustrating and fickle, I think we've all been there.
But then, you just don't learn regex. So where does the intuition for what regex can do come from? Do you just become unable to write regex with no LLM? People stop writing resources for regex I guess?
My concern is that there's stuff I feel I can just chuck onto the LLM but I'm sure my judgement is not perfect. It's still probably worth it, all in all, but I'm not even sure of what I might be losing along the way and that's an uneasy feel.
I've been using regex decades, but it never really stuck to do anything too complex, it was the perfect intersection of difficult and infrequent. ( And also variable - PCRE vs others customisations / non-regular parts, etc ).
I am very glad that I can now just ask claude for a regex to achieve my intent.
Does it mean I'll never master regex? Yes it does, but decades has shown that was unlikely to ever happen anyway.
How do you know that what it produces is correct?
I’m wondering if by the time you have been explicit enough in your prompt you might as well have looked up what you needed.
The expression is always terse so it’s not as if you are saving much typing.
Automation. And i think that vastly overestimating the time it takes to write explicit prompts.
[dead]
Regex came up so infrequent that I found myself referring to documentation whenever I needed to use it. But I always wondered, what are the jobs or roles that use it so often that they have mastered it.
That's somewhat alien to me, I suppose because over the years I just never stopped using regex in search/replace tools, which is daily exposure regardless of what code needs to be written.
Sometimes the patterns are dirt-simple, like `/\bfoo/` to find things that start with foo, or a `/foo.?bar/i` to try to find multiple variations of FooBar and fooBar and foo-bar and FOO_BAR. Other contexts outside an IDE include browser dev-tools, exploratory SQL queries on the database, searching centralized logs, and occasionally turning lines of text into spreadsheets via CSV.
None of that guarantees deep expertise of the weirder tricks, but it absolutely keeps the general knowledge "warm."
anything with large amounts of user text input.
> But then, you just don't learn regex. So where does the intuition for what regex can do come from? The training data is there for regex and it's unlikely that regex will experience massive changes but your concern makes sense. I had to learn before LLMs handled that part, the "when to use this" intuition. My guess would be that the logical conclusion is dependence on LLMs to make that determination increasing over time, both for when and how to use regex, for better or worse.
We're talking about validating the regex, which you have to do with your own skills, while LLMs are still wrong, some of the time.
In the era of vibecoding, there are people creating software that haven't ever heard of a regexp. I learned regexps when Perl was popular. It's a useful skill that has served in me well in my career, but if the industry's moved on from a place where regexps and Unix knowledge are useful because this new tool has replaced me, well shit. I'm excited for the future, but also that's not a great feeling to have.
I love AI but one thing that is a bummer is: Skills that used to be straightforward but people generally never took the time to learn are now trivial with AI.
Things like regex and CSS are not that hard if you take some time to study them. People don't though. Now I don't have that advantage anymore since AI can do them so well. Or... Maybe that could be an advantage since there will be even less people learning it.
They were also easy to look up on the fly as one needed them too. So really really knowing them was never a true advantage. For me it wasn't that I never took the time to learn them. I realized long ago that it wasn't worth my time to keep that information in my brain, and I offloaded it to google 100% of the time it came up in my day to day life. Now I offload it to Claude, same thing.
I do agree with that. I wouldn't say I really really know them either. I know enough to sound competent and be relied upon, and then I can google them to remind myself of the details.
I realized later you likely meant the latter anyways and yeah that's much different. It's important to know what these things are, how they work, and why one would need to tweak them. It was never important to have the syntax on instant recall.
If you don't know regex--how to write them, how to use them, what they're good for, what they're not good for--what makes you think you're competent to review a PR containing one?
If you don't know how to regex and you let your imaginary LLM friend write one for you, and you foist that upon your colleagues for review, you've just abused their trust. Because they, being ethical, honest engineers, must now do all the work you neglected except they're doing it at review time instead of at the time of writing.
Or maybe they're dishonest and unethical and they'll just maliciously or negligently approve the PR without actually reviewing the regex.
Either way the situation is completely rotten. You can never ethically use an LLM to do something you're not competent to do yourself.
Knowing what to learn has always been an implicitly vital skill to career growth. Maybe it’s all right fewer people will know regex, just like as each year passes relatively fewer people know about var hoisting in JS
What does the business benefit from you handwriting regex vs a clanker?
The benefits of knowing regex is not about the syntax (that’s what docs/google/books are for), but knowing when it’s right to use it.
Immersion with the codebase is not about coding and syntax (which is actually very easy for me). It’s about being able to intuit that a particular combination can result in a buggy state and provides the wrong result downstream. You lose that you stay in the dreamland of specs.
That's fair. This all feels similar to the shift that happened when Google came on to the search scene. To use your example, people realized they no longer needed to memorize syntax. Instead they only needed to ensure they understood the big picture concepts, and they could fill in the blanks on demand with Google. This is all just moving that layer of abstraction one level higher.
Not really. Before google, we had (and still have) books. What the internet and later google changed was the availability of information (If I had to rely on libraries and bookstores, I'd never be a programmer today). The job was never about memorizing knowledge, only the application of it. That has not changed since the dawn of computing.
Sort of. As a developer I had my stack of reference books, man pages, etc. I’d try hard to memorize syntax as it was a pain to look things up. And I also devoted some memory to remembering where in a book to look, which set of man pages to use, etc
With google that mostly went to the wayside. I just needed to know something existed and I could pull together far more information faster than I was spending before to just do quick lookups
I didn’t say the job was about memorization back then. Rather I found that each of these new abstraction layers made it easier to go broad with one’s on hand knowledge instead of deep
> I can’t help but wonder whether constant use of “agent” harnesses will lead to an atrophy of the software engineering (or really any field) muscles.
Used to think so, but they actually can also be used to train and strengthen skills, and learn new ones.
I had a coding interview, where they kindly sent a brief beforehand to help prepare, presenting a list of topics and concepts that might be useful during the interview, the tech stack, what kind of expectations they would have, and what they’d be paying attention to.
Obviously, it’s not exact list, and there are probably other evaluation dimensions.
But since I was out of practice on some of those, I had Claude generate a dozen sample projects, with each a list of tasks in one document and the solutions in another, and got to it.
Midway, I thought of using codex to role play as an interviewer, to tell it my train of thought and ideas as I went, get feedback, question my choices, etc.
Sure I only went through two and half, maybe three of those projects… but it’s the first time I actually enjoyed prepping for an interview. And I actually learned some things in the process.
Hardest part was probably stopping the LLM from doing the tasks, but nothing unsolvable given a bit more time, clearer instructions, and separation.
> whether constant use of “agent” harnesses will lead to an atrophy of the software engineering (or really any field) muscles
Well, I think most neuropsychologists would agree that the answer is "yes, there will be atrophy" - if you don't use it, you lose it.
> So what’s the ideal “middle ground” in this situation?
I've been thinking a lot about this myself. My current plan is to train myself to get good at recognizing the feeling of "there's potential effort here that I want to outsource to the LLM" and occasionally choosing to not outsource it and do it by hand - especially with personal projects, where there's far less pressure to ship with velocity than work projects - but I'm not settled on this. I'll take any idea!
Your suspicion is already being studied with increased effort, and the starting point has solid bones:
https://arxiv.org/pdf/2604.04721
For what it is worth, my colleagues who use these tools more liberally see a much higher regression rate and are left refactoring more as well. While they definitely put out more PRs, I can confidently say that mine are more valuable and better stand the test of time. Whether that matters or not is up to debate as I guess you could argue there may be some point where the productivity overcomes their regression rate.
My position is, if the technology will be so good and progressing so fast to the point that I don't have to write code anymore and I'm just dictating our needs and what I want architecturally, then I don't really feel the need to put in the time to learn the tools as they are now. That seems way easier to learn than anything computer science related that I had to learn to build robust systems.
> Actual muscles need exercise to stay in shape (let alone grow), so does the brain. Can we really be sure that thoughts, opinions, taste will still come out sharper and faster after five, ten, 20 years of using these tools almost every day?
It occurred to me recently how because the LLMs have to potential to lead to intellectual/skill atrophy, they are a factor which could lead to an Idiocracy-esque [0] situation.
And while it's a well-established Fermi Paradox [1] hypothesis that AI could lead to the violent destruction of a civilization, I think it's also worth considering that they could non-violently lead to their decline due to intellectual atrophy.
[0]: https://www.imdb.com/title/tt0387808/
[1]: https://en.wikipedia.org/wiki/Fermi_paradox
The doctor from Idiocracy: it says your shit's fucked up.
I don't think there is necessarily one ideal middle ground here. It still feels to me like what's best is a function that depends on who and when.
I see it as something like a personal gradient descent. You're working on a problem, there are solutions down there somewhere, and you can kind of feel the gradient of the tools-and-techniques ground around you. Any way you walk means you're investing time improving some skill or another. So you should go the way that personally feels to you will best get you moving in the direction that you want to go.
For some people it's obvious LLMs are competent coders, getting better, sticking around... and those people should lean into that gradient. For some people what's obvious is nearly the exact opposites of all that, and I'd encourage those people to also follow their gradient/heart/nose down the path of sharpening their personal traditional coding skills. Some people are in a relatively flat area where nothing is obvious, and need to explore and maybe just keep doing their best to hedge with a bit of both.
> I can’t help but wonder whether constant use of “agent” harnesses will lead to an atrophy of the software engineering (or really any field) muscles.
It will, but I'm not sure the impact of this will be all too great. We suffer from not knowing how to use an abacus because we have a calculator, and people who feel a pull to keep their low-level chops up will do so anyway.
Now imagine if your calculator billed per button press.
And imagine you can't own a calculator because owning one outright requires too much hardware (or whatever).
This is just a repackaged argument against SaaS and cloud infrastructure. Every business could be using subscription free spreadsheet software and storing all of their data on prem and employing an IT team to manage the servers and data but the majority choose to pay a third party to handle all of that. Hell, they could even go so far as to do it all manually with written records and filing cabinets. All of those workflows still exist in varying degrees scattered across the myriad of businesses, the existence of SaaS doesn't eliminate the others. AI seems to be playing out in a very similar way with the big providers, local models and die hard human programmers.
> This is just a repackaged argument against SaaS and cloud infrastructure.
Which, ironically enough, is still a terrible idea for a lot of businesses. But it's trendy, so they go with it.
The crazy thing is how cheap it is to spin up a cloud server, but how little actual processing power you're getting for your money. Even if the business manages to keep their cloud spend in check it's probably going to be at the expense of a terrible user experience because everything in the web app will feel slow and all that matters is individual P95 request latency is within some arbitrary number of seconds.
The exact same arguments were made against electronic computing in general, in the early days. Pearl clutching is a very human thing to do as new technologies are integrated and become common place. A whole generation or two of developers are going to have to pass away before we stop hearing incessant diatribes about LLMs.
Don't use calculator as an analogy, then.
Calculators are a simple machine that implement very basic rules.
LLMs are in a different category. The parallels you can draw between LLMs and calculators.. just don't make sense.
> The parallels you can draw between LLMs and calculators.. just don't make sense.
The technology doesn't matter, you can compare to say the power loom from the 1700's. I'm comparing the reaction of humans; human's haven't changed that much. They always react the same when they feel threatened and emotionally challenged by a technology.
And someone else always cries luddite if anyone says, “Hey guys, this isn’t what the salesman said.”
What matters is the reality of the thing… which is exactly what everyone is discussing.
As they should. We need more conversations like this, early. It could have saved us decades of bad practices with impossible expectations (see: agile).
But people can't separate their emotions from the "reality". The reality is, this technology has strengths and limitations. It has benefits and it has negative consequences. We can and will discuss all of that. But many people aren't operating from a detached analytical viewpoint. They're operating from an emotional, self-interested, defensive stance. And granted, there are just as many people operating from a utopian, rose tinted, self-serving, evangelical position.
In the end, the technology will get used where appropriate. And more importantly many of its weaknesses will be overcome and replaced by new challenges.
It's just a bit tiring to hear the same denouncements repeated over and over. Everyone knows them all by heart now. They're not wrong, they're just not helpful or accomplishing anything. The technology marches forward and will develop naturally. If you personally don't want to use it, then don't.
> They're not wrong, they're just not helpful or accomplishing anything.
I think what it accomplishes is developing a shared understanding of the pros/cons of the tech, the risks associated, how we should mitigate those risks, where the tech is appropriate vs. not appropriate, etc.
It feels very dismissive to say that these conversations aren't worth having. Or that we're thinking about these things too much.
I get that it's tiring, especially in the HN echo chamber. But the conversations are still worth having IMO.
There are conversations worth having. But it feels very dismissive of the obvious power of the technology to constantly focus, in an unbalanced way, on the challenges. People are biased heavily by irrational emotion, on both sides of the debate. It's all just getting a little tedious. I'm all for rational discourse and debate, but it's hard to find through all the vitriol and contempt.
> The technology marches forward and will develop naturally. If you personally don't want to use it, then don't.
The debate marches forward and will develop naturally. If you personally don't want to read it, then don't.
One up.
To Aristotle and Plato. Where Plato was against writing because people will become forgetful if they don’t train their minds remembering things.
It was Socrates that was skeptical about writing. We only know this because his student Plato wrote it down.
Exactly, looks like I should have written that down for myself instead of relying on LLM or even better, train my memory instead of depending on crutches ;)
I don't think this is equivalent. The calculator won't occasionally hallucinate a wrong answer. LLMs are a far leakier abstraction which means skill atrophy impacts evaluation and verification ability.
They're not an abstraction at all until we're committing prompts instead of code.
We might as well be committing prompts given how little review happens to much of this code.
But many people are doing just that, automating workflows through prompt templates and skills.
We will suffer from not knowing how to add. You could still argue "so what?"
Systems aren't a single addition. They are compounded operations with sprawling complexity. What happens when you can't reason through the system? What happens when you start asking for the wrong things? What happens when saying "fix it" on loop stops working?
>Can we really be sure that thoughts, opinions, taste will still come out sharper and faster after five, ten, 20 years of using these tools almost every day?
After 5 years, I think the thought profile every power user of the LLMs would be an LLM derived carbon copy of each other.
Prepare the world to get even more boringly uniform
If cars did not exist, I would be healthier, able to walk and run many times further due to constant cardio exercise.
I would still travel much less distance.
And just like cars, LLMs will reshape the world to the point that our brains could not even get us to the supermarket because soon it will be 5 miles away and require a car ( or at least a local LLM bike )
FWIW a 5 mile walk is not a long walk. Humans are very much designed to make that trek without too much trouble. Lots of the world has people walking similar distances for fresh water, to go to school, etc.
The point being that cars also reshape your perception of the world. METR had trouble replicating their dev productivity study because devs have already become much whinier about doing things "manually." Things that were slightly tedious in 2022 have become impossibly difficult without AI in 2026.
It is striking to reread Lord of the Rings: so much of the story is spent walking in and observing the Middle-Earth wilderness, and there is a distinct "pantheism" absent from the movies. JRR Tolkien, being one of the first modern fantasy writers, was one of the last who remembered a life without cars.
I had the chance recently to do several of New Zealand's Great Walks (3-4 day scenic wilderness treks with more expensive huts than the rest of the hut system), and I highly recommend doing something similar if you can swing it. Walking 15-30 km a day (9-18 miles) for several days while carrying supplies on your back really gives you new context for what you can do and what life was like without cars.
[dead]
[dead]
> So what’s the ideal “middle ground” in this situation?
I use agents to code. But I remember the early days of just AI smart complete in the IDE, where as the programmer I had to be more involved with designing and implementating the solution. This kept me engaged with the implementation as it was being built out. Now with agents, I find myself trying to catch up with what the agent did and spend more time code reviewing. Maybe you end up in the same place in the end. But building the implementation, vs code reviewing, feels more rewarding and I think helps keep your mental tool sharpened.
I think that the onus is on us to get better at using agents and AI to solve the pain points and speed things up while keeping quality high and our mental tools sharpened. I do nto think turning back is an option, but managing the pain points and leveling up is.
I think it's the opposite. Being only the code reviewer devolves into being lazy and cutting corners. The sharpening comes from hands-on practice.
That's what he was saying
> I can’t help but wonder whether constant use of “agent” harnesses will lead to an atrophy of the software engineering (or really any field) muscles.
For sure. You cannot have "only higher level thoughts" without doing lower level work.
Ironically llm themselves prove that because you cannot remove facts like 'paris is capital of france' from llm and have it just retain 'high level thoughts' like 'countries have capitals that you can look up'
> For sure. You cannot have "only higher level thoughts" without doing lower level work
What do you mean? I think people routinely think about things at a very high level with almost no understanding of the lower levels. How many people use a computer each day and reason about them at a very high level while knowing nothing of capacitors, logic gates, or programming languages?
How many people struggle with their computer, or get scammed, because to them it's just icons on a screen, with not even the concept of a process, memory vs. disk, or anything? How much money is lost each year because someone doesn't know what an URL is?
I think they didn't phrase it precisely, but my guess is the underlying idea is actually "high-level software architecture doesn't have a clear abstraction layer you can use to separate it from low-level coding (unlike logic gates, the CPU's ISA, the kernel API, etc), and so delegating the latter leads to delegating the former".
That makes sense but I'm still not so sure—we have things like software architecture patterns that can be discussed at a high level without knowing the intricacies. Like you can be aware of load balancing and even use it but be unaware of how load balancing might work algorithmically.
Let's consider even the original example. > You cannot remove facts like 'paris is capital of france' from llm and have it just retain 'high level thoughts' like 'countries have capitals that you can look up'
Wouldn't the knowledge that countries have capitals precede the knowledge that Paris is the capital of France?
This says nothing about the accuracy of our own models based on these abstractions that lack the lower-level understanding.
> we have things like software architecture patterns that can be discussed at a high level without knowing the intricacies
I think the counterargument would be "you can't teach people architecture alone and get good architects".
I've observed this myself in "systems engineers" whose job is to connect boxes together without understanding how the boxes work. They, invariably, design ridiculous architectures on their own and need to basically find a domain expert to route their opinions through to come up with anything sane.
> we have things like software architecture patterns that can be discussed at a high level without knowing the intricacies. Like you can be aware of load balancing and even use it but be unaware of how load balancing might work algorithmically.
It can be fine as a “user”, but not really as a “designer”. Because discussion about possible solutions is a matter of tradeoffs and tweaking of parameters, not slinging words around. Abstractions are not appliances that are plug and play. They’re often full of parameters that dictate their usefulness and costs, and not understanding those parameters is just roleplaying.
I think it is because people have preferences for top-down vs bottom-up learning strategies and bottom-up is what the education system is built around.
I suspect top-down learners are who benefit massively from LLMs while bottom-up learners think they are bullshit engines.
As a top-down learner myself, Claude is really the best technology of my lifetime. I can completely understand though how a bottom-up learner could see Claude as an afront to their fundamental epistemology.
Because it is.
For sure. You cannot have "only higher level thoughts" without doing lower level work.
Spend 3 days a week writing Ruby on Rails and 2 days hand rolling x86 assembly. Every web dev I know has been doing this since long before LLMs. Ensures they can keep having high level Rails thoughts.
Honestly, horrible example.
> I can’t help but wonder whether constant use of “agent” harnesses will lead to an atrophy of the software engineering (or really any field) muscles.
Compiler evolutions really harmed how well software engineers understand or how often they have to drop down to assembly language.
Is that a problem for the 99% of developers around? Probably not.
I view LLMs as the next evolution. Some people will still need to care about the layer below, the shape of the code that is being written. But over time, just as it was with the transition from crafted ASM to higher level languages, the compilers became better, more efficient and trustworthy and I think the same will happen to LLMs, and we probably won't have to check the generated code as much, at least for most of the code around.
Is that a problem? Yes, for code that is intended to interface with humans (most of it still). The quality will probably become better and it won't be much of an issue.
LLMs are not deterministic and I still look at compiler result and I encourage people to look at it. That people don't look at the result of compiler is a problem and not a an argument
> but risking being left in the dust productivity-wise?
What's the risk here? Left behind by who or what?
> Or taking an “agent first” approach and trying to learn and improve more only on the side, as more of an afterthought?
This reads like anxiety resulting from FOMO.
Here's my take: I don't care about LLMs or AI in the sense that I don't feel any need or want to use them. I've only ever tinkered with the free ChatGPT. Never opened an account with any LLM vendor and never even considered it. I program by hand for the joy of it and sometimes for work. Still by hand as I have been doing. MY work gives me that luxury. For now.
Am I obsolete? Am I no longer of any value to society? Of course not. That thinking is just implanted by a group of money hungry individuals who don't give a fuck about me, you or society as a whole. So why would or should I care about LLMs?
Economically if you are vastly outcompeted by other programmers on productivity, yes that is "no longer of value" from a purely employment perspective. Much like an old person who cant use a computer has little value in the job market beyond being a greeter at Walmart, a programmer who hand codes a loop is next to useless on a productivity basis such that it makes zero sense to employ them. It is unfortunate but true. Why pay someone to accomplish less per dollar pf cost. Feels?
Last month, there has been a billing-related incident at work: it has been detected that the GCP bill had increased vastly since early 2026
The related team investigated, found nothing; They escalated to other teams, found nothing; Gemini, claude and whatnot were involved, found nothing;
Somebody raised to topic to me, solution has been found
See my point ? Like GP, I do not use AI. If there is only one person thinking left, being that person does not looks like a bad idea;
Mixing code and "productivity" is trying to remake coder as bluecollars, and most of them probably are (not thinking, replaceable cogs that are overpaid and shall be decommissionned), they will not be missing. The only coder's output is its idea, everything else is a mean to that end.
> Economically if you are vastly outcompeted by other programmers on productivity
That doesn't seem to be the case, at least currently. The credible numbers from from companies that are using LLMs extensively are in the 15-30% productivity improvements for some developers. Those figures don't factor in how that impacts end-to-end SDLC productivity and many of the published results ignore quality which impacts maintenance costs.
Before someone hops in with "Future models will do better", that's also by no means guaranteed. I think the poster you're replying too is safe for a while.
> I can’t help but wonder whether constant use of “agent” harnesses will lead to an atrophy of the software engineering (or really any field) muscles.
This will and is 100% happening. I have a friend who hasn't written code by hand in around a year, but uses LLMs every day, and he tells me he can't remember how to write code by hand anymore. He has been a developer for 10 years. But he's not working for anyone at the moment, so I imagine if he was in a workplace the circumstances would be different and they probably wouldn't settle for this.
I think that as a result of this it likely also atrophies the problem solving and architecture building skills that writing the code manually gives you. It just ends up degrading into a loop of tell the agent to do X and assume it knows what it is doing.
> and am noticing a speedup in areas I was already familiar with, and a quicker introduction to new ones.
Only problem is that, like a LLM, you don't retain anything.
Hobby projects may become a lot more important now because if you do them without LLMs you may retain a brain cell or two.
I guess I'm lucky to have taken the career path I have, but I write very little code these days being in pre-sales now. "Productivity" is pretty stark and obvious in sales. You can output all the text you want, maybe it even compiles and passes test suites and looks pretty in a demo. But does it sell? It's nice to be able to have the only true measure that matters actually be measurable. I have yet to use an LLM for anything at all, haven't touched an agent, not because I'm ideologically opposed, but I just haven't seen the need. Part of it is the largely advisory role. I'm largely expected on the customer side to field and answer questions and render technical assistance. I'm not much good at that if I'm just deferring to Claude. The customer can do that themselves. I need to know more than Claude. The other part is just I'm not being paid to generate text. I'm being paid to generate revenue. If there's a convincing argument that LLMs can help me, well, it's an argument, but truth on the ground is I'm performing every bit as good if not better than any of my peers that are blowing a grand a month on tokens. They're definitely having a lot of fun and flooding internal Github orgs with new repos and tooling, but it demonstrably isn't generating new revenue.
"The customer can do that themselves."
>> You would probably be surprised how few actually have the agency to do even this small step.
In a way, this happened before LLMs with more workforce-fit education, decision tree flowcharts, then software, and so on. If you take most people who started any field, the way they started the field would look very unorthodox, inefficient, etc. "From the margins," as pg might say. Margins where more intuitive skills for the craft are present. Ie, the skills to come up with the model pastry are not the same skills for the pastry line to be baked in a factory.
The part of brain which cares about efficiency and waste have "atrophied" decades ago in most devs. More likely they just never cared...
E.g.
> A 1.6GB Docker image for a Node app. Every code change rebuilds npm install from scratch. CI takes 8 minutes for a one-line fix.
> The team's proposal: add more CI runners.
Would it be surprising if that team makes a React app which does 10x more requests than necessary, has 10x more JS code than necessary, flickers, etc. People just don't care, and it has nothing to do with AI. "Engineering" is too big of a word for this kind of activity.
On the other hand, I think AI can amplify capabilities of those people who still care, and it's a good thing.
I think you can use them critically. Probe the code, see how it behaves when tweaked, can you break it etc.? without slowing you down too much. And in doing so you probably learn 80% of what you would have if you'd written it yourself by hand.
But then there are definitely cases where LLMs are slower and less reliable than someone using tooling by hand, especially like deterministic refactoring operations can be done close to instantly if you already know what needs to be done and how to do it.
Use it or lose it, but the problem is very old. The introduction of the alphabet/literacy almost certainly made people's memories worse -- up to that point they had to hold a lot more in their heads. At the same time it enabled a different better way to share information and learn. The question is thus, will it make the world better, is it a book or a TV?
The amplification is what is causing the skills atrophy. When everyone is chasing max lines of code, we generate more code than what we can realistically write.
And it's not like we can not use LLM because others are using it in a craze to get the next promotion so you have to use it and maximize your output even if your skills are atrophing
I have some hobby projects I write for fun without an LLM, just to learn. And also have hobby projects that use LLM extensively, also just to learn.
Get some hobby projects.
You can run to the farm to kill your own chicken so you can stay sharp and fit or figure out what's economically valuable. Or make a bet on what will be valuable later and commit to a brand of yourself
C led to an atrophy of assembly muscles. Though this is mostly different.
A spec is NOT CODE.
Someone still needs to understand what the code means.
C optimizers are not the same as the radical departure of an LLM.
>C led to an atrophy of assembly muscles.
This is not true, though. Technically I don't do C but I used to do Assembly. Yet in Java code paths, to have a decent proof (and satisfactory results) of the optimizations made by JIT, PrintAssembly still works.
Assembly is not just the raw instructions that trigger microcode or knowing the latency of each one of them, or the (un)expected cache misses, it's the understanding the hardware, itself.
I'm frankly baffled by the number of commenters saying they've seen little productivity gains using LLMs/agents. My team got cut in half at the end of last year and we've probably increased our output by 25% primarily due to LLMs. Personally it's probably doubled my productivity. And we generally use a pretty structured spec based approach with consistent human code review, and six months into adoption we've had virtually no bugs attributable to an AI mistake (in fact production issues have gone down, knock on wood). My point being we aren't vibe coding.
I speculate the gains must be heavily context dependent (no pun intended). Different teams, tech stacks, product types, and company policies perhaps have an outsized effect on what gains LLMs give. We are allowed to be pretty independent and don't have much of a restriction on token usage.
What is your metric ? How do you calculate ROI ? Strange that you didn't specify that. Also, are you sure your "25% increase" is due to LLMs ? Every time I see this claim, it is always for a short time period or the increase was not due to LLMs at all
>My team got cut in half at the end of last year and we've probably increased our output by 25% primarily due to LLMs. Personally it's probably doubled my productivity.
Love to see worker unity in action.
[dead]
> Conversely, I also am a user of LLMs (true shocker these days, I know), and am noticing a speedup in areas I was already familiar with, and a quicker introduction to new ones. The obvious benefit cannot be denied, and doing so regardless makes you look uninformed. [Excluding people who don’t want anything to do with LLMs out of moral principle, which curiously just like the overarching topic I also both respect and understand, but on the other hand don’t do myself.]
Setting aside my moral outrage over the magic token machines. What about me, who gets so tripped up over minor factual errors, that I'm unable to let them go, and it taints the whole conversation such that I'm too wrapped up in my frustration that I can't think about it clearly? Or my innate drive for correctness that's so strong that I eval the minor errors in output, as catastrophically incompatible with my goals?
> Stoically continuing to sharpen your skills on your own, but risking being left in the dust productivity-wise?
I don't believe there's a meaningful productivity increase. Please cite your published (not preprint) peer-reviewed research that proves the productivity improvement. Until then, I'm unconvinced. (Believe me I'd like to be convinced of reality, the answer is still unresolved, and I have my opinions, but I'd rather something conclusive that I can have confidence in)
Then, even if you did show a significant productivity improvement, it wouldn't help me. I have too many qualms over the output quality that I simple can not let go, (I don't think I should, but everyone keeps trying to convince me to lower my quality standards). I don't want something fast, I have plenty of really "fast" things in my life. I exclusively want to add things that are high quality to my life. Things that don't endlessly frustrate me.
The question about where the middle ground is a rhetorically dishonest question. You'd first have to prove/convince me, that there IS a middle ground. Instead of what I believe where that middle ground belongs is quality, and everything emitted by an LLM moves reality in the wrong direction.
Are any of these absolutes? nah, hence my request/demand for peer-review research. All the productivity claims and quality assertions (mine included) are still *exclusively* vibes. But exactly none of them are pristine, (especially not any of the LLM output.)
> What about me ...
While I can see that you feel very passionately about this, the reality is that it's the majority experience that will dictate adoption.
There may never be published research on productivity -- blinding in this instance is impossible, so I don't know how you'd ever do fully-controlled behavioural studies that carried any weight. It doesn't matter. If enough of us decide that LLMs are useful to us, then this form of coding will become the norm.
If that ends up causing more harm than good, then eventually there will be a course correction. But for now, for enough people that matter, LLMs are at least giving the perception of productivity increases. And our decisions and choices come down to the perception of reality, not reality itself (for better or worse).
So I think it's far more useful to take a pragmatic approach, as per TFA. Accept that LLMs have issues, but also bring advantages, and that LLM use in coding is here to stay. What we can do is remain aware of the bad, and make better use of the good.
As for you, personally ... if you mentally cannot deal with LLM output, then I think you have two choices. You can either learn to author system prompts, so that LLM output better fits your needs and no longer triggers you; or you can sit more and more on the outer, raging against the machine while the world changes around you.
Eventually, you'll be like a master craftsman in an era of mass-production. But that's potentially highly valuable in niche markets (consider a watchmaker working in Glashutte, for example), so you may yet win from this. Remember that every day, LLMs are making your own coding skills and knowledge more elite and therefore lucrative, sit back, and smile.
Jesus ! You guys are very authoritharian. Even if it is now the norm, I prefer to not use that tool and I bet the more the time past and the more I will be faster than anyone that use LLMs (because of skills atrophy)
LLMs can also be used to improve quality. The approach I tend to encourage, is to write the thing yourself, but let the LLM review your work.
I think that's a far better and more enjoyable way of working than letting the LLM write it and letting users review it. Partially because writing is more enjoyable than reviewing, and I think people are inherently bad at reviewing while LLMs with their infinite patience are much better at it. But also because the writing itself shapes our thinking and we learn much more from it; it makes us better in ways reviewing the LLM's work does not.
That said, I do let AI write a lot for me, because I'm tackling issues that would otherwise be way too big for a single person. Is this a good idea? Maybe not in general, but iny particular context I think it is.
I'm not sure to what extent I'm fooling myself about that last bit.
Peer-reviewed evidence of "productivity" is a pretty high bar to demand.
I don't know if I've ever seen a study that attempted to demonstrate productivity that was ironclad to the point that nobody could find any methodology holes to pick in it.
I won't be as diplomatic as the other reponse to you. You are welcome to your reticence to agentic coding. But as you're no doubt observing, all your peers are saying they're moving ahead leaps and bounds.
Either all of them are wrong, or you will be left behind. Whatever the outcome, it will be on you, and you may make your peace with that.
I appreciate the candor :) But if I'm honest about the size of my ego; the people who write code I admire, the ones who feel remorse when something they built breaks. The people who care about others, and about quality, and don't list TC as something of value. Those are the people who I consider peers. None of them enjoy LLM codegen. They all tell me, that it's annoying because it takes more effort to fix it's output than to just write it by hand. So, no, none of my peers are saying they're moving ahead leaps and bounds. They equally don't understand why everyone thinks it's better. Excluding those people though; I'm more used to everyone around me being wrong. It'll be interesting to see if I, or we, get left behind.
I really appreciate your comment, I think I've discovered part of why I've never felt any anxiety over being replaced by an LLM. I've never tried or wanted to emit low quality code at high velocity, my standards and goals have always listed quality first. Feels more like asking a chef if they're afraid of llms. Why would they be, they're not doing the same thing.
I'm a hobbyist and know the programming trade to a certain extent, but at work I'm "the client" and I have my employer contract developers to address all kinds of needs that I represent as a "business interface".
I'm with you on LLMs being in a different league than good developers curating a fine design with attention to detail, soundness and maintainability. It's just that 1- not all software/components within require that same amount of care: many capabilities are already commodified (as library code or boilerplate, or frameworks, or prior art in the training data that LLMs could easily parrot), and 2- till now, the financial calculus was leaning strongly towards "new developments are expensive, let's use restraint" (irrespective of the actual effort/ratio of commodified code).
So long story short, I don't see LLMs taking out skilled and caring developers: I'll still be reaching out to them for the critical/strategic projects, but on the other hand, I appreciate that the "ancillary" or "fancy" stuff can be cobbled together with the LLM glue gun and remove the impression to my management that engaging with software development is necessarily a costly/risky/exposing endeavour. Some software is to be treated like pets and cherished, and the new reality is to accept that we might have cattle software too, with swaths of prototypes, one-off's, very specific and niche and disposable, and it's fine because this no longer represents an unsustainable financial burden (my bean counting management already likes to kill useful software on a pure spec-sheet matching exercise and shovel everything into large "do it all" systems like ERPs: the fewer things the cheaper, right? Except for those underrepresented users thrown under the bus and their effective productivity).
You need to create your own custom interface. Dive into the harness where the System Prompt lays and write instruction for the LLM to quadruple-check facts before presenting them. Instruct them to check and verify all outputs using tools of your preference before claiming their tasks are finished. Instruct them to review everything, to look for gaps in their reasoning and fix them, to never assume your intent but ask for clarity.. No doubt over the years you have developed a highly tuned arsenal of tools and protocols for work. Treat LLMs in the interface as a tool that you can lay your tuned arsenal over, recreating it as a filter your LLM works inside. Don't use vanilla interfaces and expect them to mirror your niche level of expertise. Over the last year I have developed a custom Claude Code interface using tweakCC, but other interfaces like PI and OpenCode are better documented. I am full on adhd and autistic, and have achieved as a result of access to LLMs things my neurology would never allowed me to achieve otherwise. I can bounce all over the place and still achieve excellence because of the hardcore customizations I have in place that account for my neurology. To me there is no middle ground. Slop is unacceptable. The harness and System Prompt is what moves LLMs to quality outputs.
Beautifully put. Agree to the last letter.
> a genuinely good tool that enriches your thinking
A smartphone is also a genuinely good all-around tool. Even social media is a genuinely good tool for connecting people.
Yet, I feel like we've been overly optimistic about the impact of said tools on us and our societies in the past two decades.
Smartphones are so good, in fact, in some societies, half of us are addicted to them. Billions of people world-wide.
I ask myself: Will LLMs enrich my thinking in the long run, or will they ruin it?
And what about most people? Will half of us outsource most of our thinking in a decade from now?
Given the speed and global scale that we're running these experiments with, it's fair, I think, to be a bit sceptical of the conclusion that, in the long run, LLMs will enrich our thinking.
I agree with this. Tools are good, betting the farm on a single tool is bad.
Example, cars are good. Betting the farm on cars to the detriment of bikes, buses, and trains is clearly bad. The tool of an llm is clearly handy. Betting trillions of dollars and linking the future of the nation and globe to this tool is clearly bad.
Yes instead of investing everything in AI the USA should invest in electric versions of their vehicles and in trains that can safely carry them because of their lack of range.
You're being sarcastic, but literally every other developed country invests in trains and tens of millions of people drive EV's daily with no issues.
If you want to argue that LLMs are 100x more useful than a functioning railroad system I've a bridge for you to subscribe to.
There's a saying that the intelligence of an average prehistoric cave man should be, in general, higher than a modern day human simply because the lack of technology required stone age humans to be far more intelligent then we are today. Now you can survive by working as a clerk in McDs, but in the stone age you needed to be on your toes and smart af.
LLMs are just continuing the trend humanity has long been traveling down.
> There's a saying that the intelligence of an average prehistoric cave man should be, in general, higher than a modern day human simply because the lack of technology required stone age humans to be far more intelligent then we are today. Now you can survive by working as a clerk in McDs, but in the stone age you needed to be on your toes and smart af.
This is a funny twist on the glorification of the past.
Modern humans have been getting smarter even over the relatively short time periods that we’ve been measuring.
It’s called the Flynn effect: https://en.wikipedia.org/wiki/Flynn_effect
Modern day humans still can and do survive in the wild if they so desire. It’s not some lost intelligence that was only achievable by past ancient humans.
https://news.northwestern.edu/stories/2023/03/americans-iq-s...
1. No IQ tests for prehistoric humans. 2. IQ is going down in the US.
> 2. IQ is going down in the US.
Flynn effect research says otherwise.
If you think prehistoric humans were geniuses because They could survive in the wild, by that measure monkeys and gorillas must be geniuses too.
>Flynn effect research says otherwise.
Flynn effect is just a trend for the past century or so and it's an international measure. For the past decade there's been a trend for the US exclusively that is not aligned with the flynn effect.
https://news.northwestern.edu/stories/2023/03/americans-iq-s...
That doesn't pass the sniff test, or chimpanzees would be smarter than humans.
No, the CHLCA (Chimpanzee–Human Last Common Ancestor) would be smarter than humans. It could be that chimpanzee intelligence declined faster than human intelligence. Unlikely, but I want to clarify that we did not evolve from chimpanzees. We share a common ancestor.
Stone age humans. Not pre hominids.
How do you know that? What do you know about prehistory? Nothing. You might as well be picturing the Flintstones in your head.
You're using broadly shared but totally unsubstantiated about past humans just to argue that things are good right now.
So you're claim is prehistoric humans are stupider? Also unsubstantiated. And insulting to prehistoric humans. That kind of behavior isn't allowed on HN. Don't disparage a group.
> Now you can survive by working as a clerk in McDs, but in the stone age you needed to be on your toes and smart af.
In stone age, you had to focus on other things than thinking to have enough food for the next day. And to survive otherwise.
"Flynn effect" claimed that intelligence has been increasing over time. At least we can sort of measure things in the modern era, while objectively assessing the intelligence of the long-dead is turning into skull-measuring territory.
https://news.northwestern.edu/stories/2023/03/americans-iq-s...
>> I am a little bit scared to say this too: last month I spent almost 10k USD on tokens. It sounds so insane.
No it just sounds completely stupid. You spent $10k to get a program to write text that you can write yourself just as well and for free. "Dissonance" doesn't even begin to describe it. It's like suddenly everyone has turned into the listless blobs of fat that humans have turned into in Wall-E.
I just hope all that will eventually correct itself: open (one way or another) models will reach parity with proprietary ones when diminishing returns eventually make it impossible for LLMs to grow any longer, OpenAI and Anthropic will succeed in bankrupting each other, it will become possible for everyone to run their own, personal, free and open (one way or another) model on their own laptop (assuming we still own our laptops) and at that point most of the problems with LLMs, particularly the ones that come from having to pay someone else to do the things you could always do yourself for free, will go away.
I just hope.
> You spent $10k to get a program to write text that you can write yourself just as well and for free.
"You spent $1k on a flight ticket from SFO to NYC when you could have just walked yourself just as well and for free"
0.25 day flight vs 120+ day walk doesn't really seem like a fair comparison, nobody is hundreds of times more productive from this.
5 minutes assigning the task to an agent and doing something more productive vs. 2 days of tedious typing you've done a billion times already.
As if programming by hand was sooooooo inconvenient
"Why spend $1k on a helicopter ride to Costco when you can bike there?"
Actually.
> I am a little bit scared to say this too: last month I spent almost 10k USD on tokens. It sounds so insane.
What are people actually doing with all these tokens? I use LLMs pretty heavily for development, and I'm rarely spending all the tokens that come with a $10/month OpenCode Go subscription...
My heaviest LLM usage month came in at $45. So far this July, $6 and counting (it's been a light month). I can certainly imagine increasing my usage by several orders of magnitude but ... why? If I have something that needs to get done and an LLM can do it, great. But I'm not sitting here inventing reasons to waste money, which is apparently what the tokenmaxers are doing - as evidenced by the astounding lack of value produced by all this vibe coding.
You’re not even trying to use AI if that’s all you spend. I am not a tokenmaxer and it’s easy to max out my 20$ subscription just hacking on a simple web game. Working in a larger enterprise context it’s impossible for me to use AI daily and not spend hundreds of dollars.
I have 3 max20 subscriptions and usually max at least two of them out.
I'm doing lots.
I'm building a social media site that nobody will ever use. I'm printing an ancient game to Linux and expanding the features (such as adding hw acceleration where it isn't needed). I'm troubleshooting crash logs for an Xbox game that has maybe 50 active players. Im building a distributed customer service system that provides LLMs, RAG, tool calls for lots of integrations (Gmail, calendar, sheets) (this one actually has several paying users). I'm building games, and releasing them on the Google marketplace (play store?)
But what used the most tokens was a kind of agent swarm I experimented with to implement a complex desktop application for very specific data manipulation. That.. half worked.. but it's buggy.
I made several internal tools at my company that are actually getting used by pretty much everybody. That's my most successful work I think. Oh, and I created a backup deployment in Azure for my companies product because .. we had the azure credits, so why not?
Same. Granted I have the $100/month Claude sub and barely ever see any of my usage bars go above 50% before they're reset (and that's when I had lined up a bunch of things to test Fable with).
A lot of their usage was fable which is crazy expensive.
If you are reviewing the changes and keep giving input, it is not that token consuming. But many people go with full yolo with agent mode.
I honestly felt I was going pretty YOLO :)
I write specs and review, but the LLM writes all the code, tests, etc and only gets re-prompted when it screws up badly. That said, I'm not running that many agent loops in parallel, so maybe thats where the cost bites
LLM's discussion is always the same, if you don't use it you lose it, although true, the question is why and maybe we don't need to do that thing.
Humans have been gravitating into cognitive thinking and not specialized tasks, some of us, a handful do repetitive tasks and that is ok, but, the majority of us work better in context, in guidance. LLM offers that, offers that abstraction that we were and are looking into all aspects of our lives. We once plow the fields, and harvest by hand, and was good, but suddenly became cumbersome, too much to handle and machines came along to do the job of 20, faster and better, and there humans just tweak the machine, teach the small repetitive tasks. We did that with everything and we change our lives, the way that we upskill ourselves and our communities, we let specialized human tasks as a comfort/skill/romantic/artistic outlet, like some specialized vineyards or specialized farms that do specific very specific things. But the rest we delegate and move to another areas. I believe the problem is not LLM/AI is how we are going to up skill, repetitive jobs no longer need us, so we need to upskill for context, for management, strategy
The issue that I take on this is ego, we believe that we can do a better job sometimes, an we exhaust ourselves with no reason, Agent delegation works in the same way as Human delegation, the difference is the speed on how that happens, reviews were a good idea when humans toke days to do a product, features, code, document or what ever. The reward piece I believe is faster to see something working, the issue seems to be the control we had. The approach needs to be the same of those we do with almost everything else. We drive a car but no necessarily knows how to assemble, we might now here and there trouble shooting, change tires, oil, do some level of maintenance, but at the end just a few of us can pull an engine apart and check every piece, cars are not even manufactured manually anymore, QA is part of an automated process, as production rates are far to fast for humans to intervene. AI brought that to us, insanely fast automation, and with that, Humans should not become gatekeepers but rather guides.
> And then there are the geopolitical tensions. What happens if China or the US cut us off overnight from these technologies? Just a couple of weeks ago the US government showed it was able and willing to cut off non-US citizens from Anthropic’s latest frontier model.
The only evidence you have is US here and all the frontier models from China are free to download. Pure projection.
You can read China's AI strategy here: https://ipc.court.gov.cn/zh-cn/news/view-5766.html
I might be missing something but what exactly is being projected here? The US did what the author said. It's not farfetched to imagine geopolitical conflicts could, at some point, drive china to do something similar.
> all the frontier models from China are free to download
Maybe? Again, not hard to imagine that they might not be releasing their actual frontier models to the public.
I guess China did not deserve mention. They are the only one releasing open models consitently and not cutting any access. Only US actions show evidence and for the US only.
Fair enough, that makes sense
The link I shared mentions their AI strategy.
Short version is this: putting export controls on AI open-source will hinder their own internal open-source ecosystem
> I tend not to actually read most LLM output anymore; I skim it, to check if I vibe with it. But a problem statement of three sentences, that I will fact-check really hard. It is like code review: a review with 1,000 lines of code gets an “LGTM”. A review with 100 lines gets 15 comments.
This is where I take issue. I'm in a similar boat to the author. In the last couple of months, I've been experimenting with increasing the use of local and cloud LLMs for my research code. I'll create a prototype, maybe port it to a language I don't use very much like Rust, run some tests... but at the very end when I'm very happy with it, I _need_ to go line by line and understand _everything_ that is happening. Sometimes that means using an LLM to understand it, but even when I do and there is a concept I don't get, I try to read primary resources written by experts.
The least bad thing I've found LLMs good for is ideation because it's super easy to take the good nuggets and leave the bad, but even that carries risks of shaping thought and making everyone reach for and ignore the same ideas in the way the Spotify radio or YouTube autoplay has been shaping/flattening tastes for the worse.
I'm not sure what I'll rule at the end of my experiments with LLMs, but right now I'm enjoying the rush of having prototypes that run quickly. I've always been a top-down learner, being motivated by hacking a cool demo I half understand and progressively tearing it apart.
One thing about open source is a lot of people are throwing low quality PRs that should have been an issue or even a discussion, so you can understand what was the problem the person encountered that motivated such PR. This is hard to get because usually people use LLMs to also answer questions you make about the PR. I am tending towards blocking PRs from people outside the main developers in most of my open source projects. If the person CAN discuss the problem, authoring the PR is easier if I do it myself, even if using an LLM, because reviewing a PR authored by random internet people/bots is hard because of how much the entire code tends to change after minimal questions are asked. What I am sad of this approach is that I did met a few interesting people through receiving PRs and establishing trust and some relationships in the past (eight years ago and before)
Do I understand that spend report right ? 10k USD in a month for AI tokens ? After talking about the environmental cost of AI ?
Yeah this is where I stopped. I use AI every day for work but 10k spend to me is a signal that OP is doing something extremely stupid with their AI use.
That's the cost of one employee (and not an expensive one). Do you think it's impossible he's getting more value from the AI than he would from a single employee?
If it's an employee, which I doubt but ok, it's an employee you constantly have to probe, talk to, give instructions to, so, not a really good employee.
Neither of us have any idea whether his manufacturing startup is successful. It looks to be a 5 person team. A google search suggests they had a funding round early this year.
It might, in fact, be that you are more successful with your approach. I have no idea. Congrats if so.
I mean, he must be pretty successful to be able to burn 10k in tokens. If he is successful because or despite of the AI, I don't know and I don't really care ? That wasn't the question.
It's not the case that someone successful is doing everything right and someone unsuccessful is doing everything wrong, I'd grant.
But I think it's odd for you to now say it doesn't matter whether or not his AI token use is leading to success or not. I personally think it's actually really important to learn whether/when this approach is smart.
Depends how you use the tokens. It's not 10k, but I have an agent do an investigation automatically when I get paged, so by the time I get to the computer I have a nice summary. But that's costing me like $5/mo at most.
But you could use tons of autonomous agents for all kinds of things. Probably shouldn't, but you can.
I use AI every day, and I use it quite a bit, and I use Opus more than I maybe should, and even then I'm not even at $2000 in tokens this year. I don't understand how people are tokenmaxxing as much as they are.
Doesn't seem that wild if youre doing large green field dev with frontier models.
Im spending about 2-3K a week on personal projects and 5K a week on corporate stuff.
Crazy. I use claude pro and I barely hit the quota working on greenfield stuff.
You could realistically be burning through ~5k of tokens without hitting the quota. The subscriptions are crazy subsidized relative to the API rates.
I did the napkin math (https://news.ycombinator.com/item?id=46984659) -- LLMs + humans are way more resource efficient than humans alone, even if they speed you up by only 5%.
It's not because LLMs are that efficient, it's because humans are that inefficient.
Yes, this dissonance is the point of the post. The first two sentences use the word, and then it's repeated throughout.
It's nice to see a person who actually acknowledges tradeoff.
For the curious, this seems to be his company: https://www.umh.app
>I almost agree with all of the LLM critics
That seems unlikely given the diverse nature of mutually exclusive opinions that exist out there.
Critics seem to run the gamut from LLMs being incapable of even the most basic of functions to already sentient creatures secretly plotting our destruction with steganographic messages to each other.
It's maybe a bell curve with some wacky at those tails, but there's some fairly significant differences of opinion amongst the positions that are more mainstream.
Just the difference between critics of all LLMs and crutics of all closed weights models are a pretty big gap.
Similarly for those who criticise them for over censorship vs those who criticise them for unrestricted generation.
> And this is where the value is for me: I can simply make things higher quality than I could do them alone.
Yeah, that's the thing for me. LLMs have made my work easier and faster, and they've made my side projects easier and faster. I think there are very sensible and valid critiques but so far the tool works for me.
Same thing here. I was able to finish side projects on my list and even refactor decades old projects with the help of LLMs. It works great so far and yeah, I can still write source code when needed.
I think folks ITT are trivializing the issue here. Many keep drawing parallels to LLMs being akin to the loom or calculator. That seems intellectually dishonest to me. Those tools were not anthropomorphized interfaces, nor were they instant gratification. To do significant things with a calculator, you still had to know maths, order of operations, formulas, etc. We didn't lose mathematicians as a result of the calculator, nor did the calculator play with our emotions.
I have linked this elsewhere in this thread, but the studies on this matter suggest a worse outcome than just losing hand crafted or "artisanal" software: https://arxiv.org/pdf/2604.04721
> These findings are particularly concerning because persistence is foundational to skill acquisition and is one of the strongest predictors of long-term learning.
> We posit that persistence is reduced because AI conditions people to expect immediate answers, thereby denying them the experience of working through challenges on their own.
This issue is not that we are losing the ability to write good software by hand. It is possible we are losing our internal tools to learn new things by circumventing a necessary part of the process required by our biology.
> nor did the calculator play with our emotions
It seems to me this here is the dangerous 'force multiplier' for destructive outcomes.
It's one thing for a kid to become obsessed with a fictional character. It's a whole different playing field when that character can talk back, remember things, and potentially surface all kinds of subject matter which cannot be predicted and could at times be harmful.
Not even remotely the same level of impact. I think we're almost at the point where LLM education (as in, teaching awareness of the basic tech and its shortcomings) needs to start in Kindergarten.
Exactly this!
LLM's are replacing your 'brain' i.e. the deepest most amazing part of you. The potential value of this to business is astronomical!
No there won't be an explosion of new software, a minor blip perhaps and even this one of diminishing returns (each cycle of LLM improvements requiring less and less human input).
If there are 25 million software developers today then the ROI by OpenAI, Anthropic et al pays off if in a few years time only 15 million developers are needed. That 10 million is the ROI and that is likely to be you.
I agree with most of the 'llm's are bad' section. However, I find it extremely hard to believe that llm's will make peoples thoughts sharper and faster. There have already been several studies published demonstrating the obvious. Not hard to imagine that long term, the negative cognitive effects of using llm's constantly will compound and be even worse.
How LLM usage affects your thinking process entirely depends on the workflow you use with them.
I can honestly say that while using LLM my thoughts are as sharp and as fast as they were before, if not sharper and faster. But my workflow is that I watch the LLM as it rips, have verbose output turned on so I also read Claude's thinking process (and lots of interesting information is hidden there).
This helps me learn new stuff I wouldn't think of. Helps me stop Claude if he's veering off track. Makes it easier to proofread what's Claude doing and why step-by-step and easier to spot any mistakes (because they often come from wrong assumptions you wouldn't see if you didn't see the thinking process).
If you just let your agent write thousands of lines of code, reviewing them is painful. If you also run multiple agents for multiple tasks at the same time, your focus is divided, so you aren't spending all your brain power on the task at hand.
i was certain of the same worst case scenario until I started to properly read and study the LLMs output. prior to that enlightening, very wise choice, I speedread/skimmed and checked for grave bullshit/hallucinations only.
now i am convinced that it's the same matter as with people who forget most of what they learned after the actual exam vs people who go nuts and can't progress if they didn't understand a foundational part of a formula, fact or theory.
The LLM critics are right, and that's why I try not to use AI in the ways critics point out are bad.
I use AI to code tools for myself, but I don't pretend anything I make is production quality. Duct tape engineering has always been a bit sloppy, and AI just made it faster.
I use AI to troubleshoot issues and plan out strategies, but I basically consider the AI draft of anything to be "draft 0", and use it as a framework for writing my own works for a real first draft of anything I write that will be read by other people. Sometimes the AI spits out a perfect paragraph that I might copy, but I don't ever blindly trust it or let it speak for me. I also double-check everything it says that I don't have existing knowledge of, rather than trust it to be right.
AI images, video, and music are all entertaining, but I only generate these things as a form of self-entertainment and maybe online meming. I could never in good conscience pass these creations off as my own, or publish them online on a personal or business website when something non-synthetic would suffice.
And I am never personally confiding in an LLM like it were a person. I have had it help me brainstorm options for office politics stuff, but I'm not about to ask it for relationship advice or to be my friend.
I do love that it accelerates the tedious stuff, and helps me learn new things pretty quickly if used right. It has definite utility. But I am always really distrustful of it. Sometimes at work we are asked to share how we use AI, and I have actually refused before, on the grounds that I may have found a useful way to use the AI, but I am worried that others will use my same method badly (e.g., not verifying eveything the AI says first), and I would rather not share.
It's like I have a finicky gun. I might be comfortable shooting it since I know its quirks and how to keep it from accidentally discharging, but I'm not loaning it out to anyone I wouldn't want to accidentally shoot themselves with it.
> "Last but not least, even when just researching with LLMs, they have the natural tendency to silently sneak in the thoughts of the majority of the training materials, or sometimes even the political convictions of the ones who created the model."
> "Yet I still write all of my texts with LLMs"
So I'm guessing the author is actually ok with the point they put in the "LLMs are bad" part of the article?
As long as, after it writing his texts, he is reviewing and editing them, I don't see the contradiction there. And I'm somewhat anti-AI (if I don't care enough to make the effort of writing it myself, why do I care for it to be written at all?).
That's the problem with human written text, sometimes it hallucinates.
The implication is that LLM critics don't use LLMs at all, or that the author is not an LLM critic, but both of those things are incorrect. We are very good at inventing entire people out of single opinions we read, and the AI arguments are maybe the best example of that I've ever seen in my many years watching internet arguments (not least due to the expansiveness of AI, and the sheer breadth of pros and cons it holds within).
> We are very good at inventing entire people out of single opinions we read
Can you clarify what this is supposed to mean?
Someone says they like hip hop, and you imagine a young urban male with a baseball cap on backwards, how they view women, that they probably don't like musicals, something like that (not saying you do, but as an example). More than simply the thing they said. You say "I like or do that thing", and instantly get hit with "oh, people like you always complain about X but isn't it weird you never mention Y".
At least that's what I thought of when I read that; because answering in place of other people is another thing I find annoying. So I'm speaking for myself, but with some optimism I may have guessed the intent of OP correctly :)
>as a senior, you don’t need juniors anymore. The mundane tasks, at least I find that a lot of people agree with that one, can be fully outsourced to an LLM
Master craftsmen didn't take on apprentices to give them chores.
>Master craftsmen didn't take on apprentices to give them chores.
Is today opposite day?
So the purpose of a graduate degree is TAing a few classes?
A big part of the value to the tenured professor is not having to teach an extra couple of sections of CS 101. The other part is plastering their name on a bunch of extra papers to game their h-index :D
I don't want to be too flippant here, but senior engineers generally gain very little from training juniors, not least because promo processes suck at basically every tech firm, and so the moment you train up a junior, they are going to leap to another company rather than go through promo...
Master craftsmen paid apprentices almost next to nothing, and they were often contractually guaranteed to stick around for many years, so the teaching was a kind of wage and also a cost that could be recuperated later on. (The apprentice even often had to pay the craftsman to take them on.) None of those things are true for junior software engineers, who are paid to contribute and can leave at any moment. Also, yes apprentices often had to do chores. It is just not analogous at all.
Not to mention, the physical trades are not stopping taking on apprentices because... LLMs can't build a house? Not even the mundane bits.
It will be interesting how to get the financial incentives to train juniors in knowledge work to line up with the fact that every new generation needs to be trained.
LLMs are awesome, but sometimes we'd be better off with more friction.
This is because.. sometimes an idea is just bad. I learned the hard way the other day when I filed this PR against prek: https://github.com/j178/prek/pull/2302
My idea was bad.
The implementation was meant to solve my problem. All it did was expose more problems and waste my time and the maintainer's, and god knows how many tokens from Claude and Codex. In the end the PR was useless so I closed it.
The maintainer and I would have both been better off - less time wasted - if I was forced to forever wonder if the idea was good, letting it bounce around the back of my mind.
>If you have thoughts, they come out sharper and faster.
No, they come duller and increasingly dependent on the LLM doing them for you.
And that happens to the generations that learned to code earlier without the LLM. What would happen to the later ones, those getting in uni after LLMs emerged, that would never get a real feel for the basics to begin with?
The one thing that had me reading to the end was the mention near the top of the environmental impact of LLMs, but he never got back to it.
I'm currently writing an onboarding doc for my team, encouraging LLM use for some tasks. (OK, well, I'm actually procrastinating by reading HN).
At the same time, I'm in a darkened office with tinfoil on the windows and a fan pointed at me because it's hell outside and it has been for weeks, and every year it seems to get hotter and hotter and we have longer and longer heatwaves.
This seems ... discordant, at a minimum.
Really, _should_ we be using these things to speed up, say, dependency updates if the cost is the planet? I wanted to know what the author thought about that.
Your commute is worse for the environment than your day's token usage, by a magnitude.
Hey, I'm happy to work from home. I'm not the one choosing to have a commute, companies are forcing that.
Wait a sec, aren't companies also making LLM usage compulsory?
Wow it's starting to sound like they don't care about the environment!
This checks out 50-250kg CO2 are common guesses for 10k spend.
That would be in the same order of magnitude of an average daily commute with an older car.
Well, I doubt that. I mostly work from home. But also I live in a big city (London) with a working public transport system.
I dont know what to tell you. I guess - correct your misconceptions?
Carbon footprint of commute is in KGs of CO2. A day's worth of AI, depending on model, is ~15-100g of CO2.
Heck food you eat has more than a few Kgs of footprint everyday.
I won't belabour the point, but I think someone needs to correct their misconceptions. You're comparing a single, average person's usage of AI in a day -- not the billions who actually use it. Then you are comparing it to an American's two-hour commute in an oversized SUV. (A commute on the Tube costs about 0.02991 kg per passenger, per kilometre.)
What about training those models? Or the usage of all of the data centres? The projections are that by 2028 a fifth of all energy consumption in the U.S. alone will be for AI.[1]
[1] https://www.technologyreview.com/2025/05/20/1116327/ai-energ...
Does that include the training?
yes. training is a magnitude lower of energy of the lifecycle of frontier models. with current use of opus etc, the math is closer to 1%.
https://libguides.usc.edu/blogs/USC-AI-Beat/hidden-costs-of-...
This is actually a good link. Thank you. It's usage that is the big driver, and LLMs are a big driver in emissions and other damage to the environment.
What a nitemare to read. I stuck with it but wished after I didn’t. Reminds me of a 2.5 star movie that I watch to the end but am so annoyed as the credits roll.
Yes there is a dissonance, using LLMS just took their dot point notes and spread them thin. I personally would prefer just the notes and some personal anecdotes to tie the notes into a cohesive whole. This, this approach here, is not better.
It depends upon why you have issues with LLMs. If you’re just concerned about quality then sure the dissonance isn’t intolerable. If you’re concerned about their ethics then this becomes a much more challenging position to have.
But that's the thing - people can live with the contradictions. You can find people at a gas station who will totally agree that air pollution and CO2 emissions are killing people, but still fill up the tank.
My feeling is that LLMs are an interesting technology. I don't feel currently threatened by them, and they're handy in the hands of people that can use them with discipline. However, I think the people controlling and creating the LLMs are by and large scumbags who want to gamble with everyone's future. It's hard for me to look at their public statements and the actions of people like Altman, Amodei, Musk, and come away with any other conclusion than that these are fairly awful people that have bad intentions. Their message is essentially "we want to summon the machine god to destroy your livelihood because it will enrich us and give us power over the world, and there's a really high probability it gets out of control and causes massive problems [that we're completely going to not do anything about]. But we really like money! And we have bunkers!"
I suspect a lot of people are in this boat. We both use it and cheer against it. But it's not cognitive dissonance, it's just a recognition that there's a useful tool being controlled by vile people.
> I think the core issue here is trust. You should never trust random people on the internet anyway. But before LLMs, there was this base thing: creating a proper PR with proper descriptions would require at least some human time, so it would keep trolls and low quality submissions out. Or at least you could easily filter them out within a couple of seconds. So even if a new person came in, you could trust that this person would have at least spent a couple of hours on that. And then it was probably worth taking a closer look at it.
Ding ding ding. This is my biggest gripe with AI. Even the SEO blogspam, the fluff in front of every recipe, yarnwork or DIY instruction, it all was clearly written by a human. Someone had invested time (and money) in getting something in front of my eyes.
But now, it's all just slop. Everywhere. And hell I'm tired because the onslaught breaks my trust filters.
Maybe I think this is an age thing. Boomers? They trust everything written down somewhere. No matter what, and no matter if they didn't spend half my childhood to "never trust what people write on the Internet", and now they fall for scams left and right. My generation as said grew up with this "never trust, always verify" thing. And the younger generation? They DGAF about anything any more, all they care about is trying to survive.
> And b), the teaching, aka “How do we teach new people?”: previously, there was this balance aka “the junior does some pretty mundane tasks, but for this the senior reviews it together with him and helps him to grow”.
GOD YES YES YES THIS x1000.
There is barely anything more rewarding than teaching someone something, to watch the other person grow - and eventually surpassing your own abilities. That is when you know you did right and well. My wife is the best example, she started out at "can you help me with Excel", and these days, she pulls off stuff that would make more than a few finance people blush.
> There is barely anything more rewarding than teaching someone something, to watch the other person grow
I think many junior devs (or aspiring junior devs) look for exactly this experience. This is a matching problem we haven't solved yet. Is Open Source the solution ? I really think it has to be solved if we want truely reliable software in the future.
There's a tangential problem - companies do not want trainees or juniors any more, they (usually) cost more money than they bring in for half a decade if you don't fraudulently bill your customers... something the "consulting" industry is infamous for. And when the juniors got enough experience to be considered intermediate or senior, they jump ship to get a larger pay rise, leaving you with the need to hire another senior.
The entire economy is broken due to the focus on short term quarterly result instead of the health of a company in 5, 10 or even 20 years.
I think that the use of LLMs to expedite the learning process (especially when it comes to learning new things) is extraordinary and should be practiced by anybody with the means to. Think about how far we as the human race have come. Everything eventually plateaus and either flatlines or else regresses.. With the use of AI we can expand our ways of thinking into a new tier we would have never been able to achieve otherwise. It goes without saying that people who refuse to use the tools we've been given are close-minded and susceptible to fail in the new age that we are truly lucky to be a part of. And its here to stay, whether you like it or not lol
> And then there is the situation about junior engineers. There are actually two different points in there: a) you cannot trust the effort behind your junior’s code anymore, and b) seniors have no incentive left to teach juniors.
If you are bringing in juniors with the expectation that they won’t produce any value for some time then it might be better to block them from using LLMs.
Do this and you will get a better understanding of where they are in the process and they will probably learn the ropes more quickly.
> If you are bringing in juniors with the expectation that they won’t produce any value for some time then it might be better to block them from using LLMs
A) I think many companies no longer view juniors as an investment, they want them to be productive. They view juniors as "cheaper almost-seniors"
B) Blocking them from using LLMs would mean acknowledging that there is a downside to having your employees using LLMs. If they prevent your juniors from improving, what negative effect are they having on your seniors?
Block juniors from LLM usage. But then how will we show metrics of AI usage??!?!
My use of AI is correct. My coworkers use is not.
I'm saying it in jest, but it's also a bit true. Not necessarily because we use it any differently. But because my use of AI saves me time. But their use of AI adds more to my plate, no matter if it's slop or not.
I noticed very recently I actually prefer reviewing other people’s generated code over handwritten. There’s no ego involved and it usually does a better, more thorough job. We have enough infrastructure that by the time another human lays eyes on it for the first time each pull request has already been through 3+ rounds of automated review. Being the only specialist in my area on the team used to mean more coaching and more involvement in code review. Now my coworkers are sending me idiomatic solutions with modern approaches that I approve right away or with hardly any comments
So much this. When coding I use my LLM to hone the code to perfection before sending it to review, including running AI reviews on it multiple times and fixing obvious bugs so that reviewers have less work to do. I also test all my code manually (LLM can be helpful by giving me a list of test scenarios to consider). When reviewing PRs I carefully read the AI reviewer output, consider which points are relevant, which are irrelevant and only post the relevant issues when requesting changes.
I have coworkers who routinely post PRs with comments stating "all tests green, ready to review" and then I actually start the application, test it manually and it doesn't work. And don't get me started on reviewers who post the raw AI output of their "reviewing skill" and then I have to read through 30 paragraphs of "findings" most of which are irrelevant (just today I got the "valuable" feedback of "View is @api_view(["POST"]); no test asserts GET/PUT/PATCH/DELETE → 405")...
The critics are absolutely right, LLMs have a lot of faults. But they also have a few great benefits. I use them every day for the benefits, fully aware of the faults, and I watch those faults like a hawk.
I think I'd put it this way: if I could flip a switch tomorrow and the AI boom was done, I probably would. The negative effects just super duper outweigh the fact that coders are more efficient. But since it exists, I'm going to use it. Hacking on my own small projects isn't using that many resources. But the benefit simply isn't worth the cost, as much as I personally love using LLMs.
Aren't all conferences just huge piles of cognitive dissonance?
At least the "industry" ones used for "networking" and all that other crap. Where's the news here? And why does the author not take a step back and realize that there is none?
> And this is where the value is for me: I can simply make things higher quality than I could do them alone.
It's sad that this can be true because if you did them alone the quality would be non-existent.
> I strongly agree that written text should be from humans to humans. Yet I still write all of my texts with LLMs. And I don’t find that contradictory. What distinguishes “AI slop” from “good writing” is whether a human has put thoughts behind it. And you cannot outsource thinking.
Writing is thinking. If you're outsourcing your writing to an LLM, you are shortchanging yourself by skipping over much of the refinement of ideas that the process of writing provides.
I wish the author had clarified how he uses LLMs for writing. It's perfectly fine to have an LLM proofread and fix your grammatical errors--that's a mechanical task. But I think it's gross when it starts putting words and content on the page.
> last month I spent almost 10k USD on tokens. It sounds so insane
It is. That kind of money would change my life and this guy spends it on prompts that "really force him to think".
I use LLMs. So far, 20 bucks per month has me covered. I hit the limit sometimes. I just take a walk. Forces me to think.
At what point would one say that the LLM critics were wrong in their load-bearing (yes I used it) claims?
The desire to categorize people as right or wrong should be resisted. It's tangential, and distracts by feeding the tribalism part of the brain instead of considering individual things that are explicitly stated.
> The desire to categorize people as right or wrong should be resisted
kinda shih people say after their Load Bearing Claims (thank you Opus 4.6) turned out wrong
When AI frontier labs fire all their engineers.
why
... Because AI is more capable?
If the critique itself turns out to be wrong. Using LLMs does not equal endorsement nor does it make valid points to critizise disappear. "using LLMs" has no clear definition and can still be in line with the critique.
If you have no critiques about a piece of technology in an area of technology that has existed for less than 10 years I think that speaks for itself.
so you are saying that the advent of internet also deserved criticism in much the same way as LLM's?
- job displacement
- ethics
- environmental
- skill atrophy
Such concerns were present, and rightly so. Concern and critique is important to express so that we are sure new tech is useful and negative side-effects are realised and appropriately mitigated.
Job displacement: When the Internet first arrived in a significant number of non-technical lives, there were concerns about research jobs and library jobs. In fact it increased the need for that sort of thing so increased that sort of job, and even if it hadn't the access to information for the average person would have been worth that cost. As remote work grew initially, then massively in 2020, there were major concerns about that sort of thing again (people working in Manchester on London wages, going into the office very little, it being even easier to outsource jobs to countries with cheaper workforces, etc.). These concerns are still being debated today, and that is quite valid.
Ethics: Hell yes. And again, it is good that we are still refining what we think is acceptable to this very day. There have been ethical concerns almost from the start of the Internet in the public mind, and they have only increased over time as new parts of it take shape.
Environmental: That is a lot less relevant, as the Internet itself didn't directly have the sort of impact that LLMs are having. There have been local concerns though, over the building of data-centers and the power infrastructure to support them, disturbing land to lay cables, and so forth, and again these concerns are still relevant (though have somewhat folded into just being a concern about LLMs & related ATM as they are the main reason for the current rapid growth).
Skill Atrophy: The Internet never really threatened that, except maybe a small amount of concern wrt certain research roles, it increased the base level of those skills in the general population much more than it had any negative effects (but people did raise the concerns). It gave better access to training and knowledge, but wasn't actually doing things for us in the same way LLMs and the agent systems linked to them are being used, you still needed to learn from that information/training in order to complete your task.
What about the impact of internet itself - its utility. Thats the primary mode of criticism on LLM
The advent of the internet was collaborative and based on introducing shared protocols for a couple of decades. It deserved criticism when globalised capitalism got involved, and monopolies started forming, leading to rent-seeking, excessive centralisation, and enshittification.
The impact is that the internet has a fraction of the value to improve people's lives as it should have. It is a very poor free market, incredibly poor competition because of lack of standards and protocols and interoperability. People's minds are ground down by social media, search engines don't work well any more and so on.
So yes - every new technology deserves many criticisms, so they can be addressed, and as a society we can gain the benefits of that technology and minimise the disadvantages.
The printing press lead to copyright, public libraries, universal literacy... All things which are now widely celebrated. They took centuries to work out, and are all government and regulatory intervention to fix problems critics noticed and campaigned about.
AI is the same, only it is at risk of moving much faster and having a much large negative impact before society reacts.
So no, most of the criticism of LLMs are not wrong - they are correct, as are the people saying the technology of LLMs is useful to people and the economy.
Critics are friends of a new technology - without responding to every criticism in a significant way, AI will rapidly lead to a Butlerian jihad. If you like AI, you should love criticism of AI even more.
Are you assuming that I believe the advent of the Internet and LLMs are the same thing?
"Dissonance" is a funny way to spell hypocrisy.
Because it's a machine, not an oracle. If you "hold it right" you are more productive. When you catch them being "dumb" you're reinforcing your own deep knowledge on the topic. When it is correct, you're either learning something new or your task is complete. You are still at risk when you know little and trust it to work alone.
> I strongly agree that written text should be from humans to humans. Yet I still write all of my texts with LLMs.
Hypocritical garbage. Into the blacklist you go.
If you can’t even use your own words for basic communication then there’s absolutely no reason for me to take you seriously.
Good article! This matches how I feel about the situation.
It's really not incongruent to use LLMs and be in awe of their frankly incredible capabilities while at the same time recognize the risks and frankly real damage we are already seeing to junior training and hiring, open source communities and (in my opinion) very soon the entire fabric of our society.
I respect that people don't want to use agents themselves for whatever personal reason.
I respect maintainers not accepting AI-authored contributions. It's a tradeoff between progress, growing new contributors and maintainer sanity. Though I do feel that categoric opposition to anything AI will likely be futile in the mid-term.
I respect people pushing for regulation of AI or a global pause or whatever.
I don't particularly respect people dismissing everything AI authored as slop. Categorically refusing to read an article because it contains em-dashes or the term "load-bearing" is silly. While this is slowly changing now, many people are still in complete denial as to what the frontier AI is capable of.
Love it, hate it - I don't care, but at least respect it, goddamit.
You're all in denial when you criticize LLMs. It's not necessarily that the criticism isn't true. It's more how self assured the criticism is. That's the biggest problem because AI is a moving target. It is getting better, and it is getting better fast. A lot of the criticism can become outdated in a year or six months. The change is happening in front of your very eyes and yet you can always reliably come on HN and find some sort of self assured criticism to say AI can't design, AI code must always be reviewed. Blah blah blah.
The big thing people used to call AI was that it was a stochastic parrot and all it did was summarize things. Clearly. None of this is/was true anymore. And very likely all the current criticism will be eliminated soon and we have to find new excuses about AI that makes us feel we are superior.
The status quo is about to change. Every 6 months. And you will always think of yourself as superior to LLMs. Your current criticisms will evolve as most of them will be rendered not true pretty soon.
> AI code must always be reviewed
Yeah, of course AI code must always be reviewed. All code must be reviewed.
Nope. Not true anymore. Right now the bottleneck is the review because code comes out so fast there is a measurable and adjustable trade off that can be made.
If you review all code, your output will be slow. If you review less code, your output will be faster at the cost of more bugs in production. Bug rate will never go down to zero whether you use AI or not.
That is the trade off, if you review everything then output is really slow. If you start only reviewing certain types of code like model changes, database changes. Or only backend code and not frontend code, you hit a sweet spot of speed and reliability.
As LLMs improve the need for reviews becomes less and less. Companies who don't adjust are just slowing themselves down. That is the trend.
With that logic, you would let an intern force push his way to prod, that's not a smart move.
no force pushes. And probably not an intern.
But code that never gets looked at and only tested? Yes. And as AI gets better this will be happening in all companies... more and more.
Anyone who lets the LLM put out unreviewed code is shockingly negligent and bad at his job. It certainly might be the case that many people are bad in such a way, but that doesn't make it acceptable.
I accept it. You will too. Eventually. As AI improves, the reviews will just become more and more just time wasting blockers.
How do you know your testing is complete if you're not sure what the underlying code does?
Testing will never be complete.
The above code, if a and b are integers have 2^64 test cases for full coverage. Code being much more complex then that on average will have a magnitude more test cases needed for full coverage... likely more test cases then all of the code humanity has ever wrote since the dawn of programming.Testing is a statistical sample. You sample a small subset out of the full domain of possible tests and that sample you hope is somewhat representative of the actual population of tests. To put it in a sentence: If my suite of tests is correct then it is more likely that the entire full universe of every single possible test for my program can all be correct.
Realistically though the universe of all possible tests is so large that it's basically impossible for any sample to be a good representation of it. A suite of test does help up confidence that things will be MORE correct, but the sample size is so small it is almost guaranteed that you missed something.
I think both the LLM critics and the LLM advocates are right.
Even this article has some cognitive dissonance in it. What it really comes down to is how much you trust your own verification process. The branches of questions an LLM generates are still trapped within the biases of its training data. Of course, the authority to craft that initial prompt, the very first question, comes from human experience and learning.
But I think thought itself is the easiest resource to outsource. People say the human did the thinking and the LLM just amplified it, but the truth is, the LLM outsources the thinking. Otherwise, when the result is good, people say "human thought was present," and when it's bad, they say "human thought was absent." But a part of the actual thinking really is outsourced. The alternatives, the counterexamples, the sentence structure. In programming terms, the reader's experience gets outsourced. When you write a blog post, you find yourself thinking about how to make something you understand easy for someone else to understand. With an LLM, that part gets outsourced.
But at the same time, I don't get the argument that you shouldn't use it at all. We don't "think" about everything. We have limited cognitive resources. So we study deeply the things we care about, but for the things we don't need, we mostly leave them to "common sense" or prejudice. We just skim the surface.
I think of "common sense" as "the largest collection of prejudice." Because what we call common sense usually just amounts to surface level knowledge, the kind of thing we know just enough about to get by.
That's why I think LLMs are good. The reason is simple. I don't think deeply about everything in the world anyway. For everything else, I'm buried in some kind of bias. You see it on HN all the time, right? People fight over some technology, but they often don't think about its internal structure or why it works the way it does. They just treat it as an identity. They fight over a particular language, a framework, an operating system, but they rarely check how that technology actually works internally or why it was designed that way. Why use MVC, why a different architecture might be better for my case, it's easier to just go with what's popular. Put more elegantly, "job mobility" gets bundled in there too. I use Windows. In my country, if it's not Windows, you literally can't do anything. You can't even do basic online banking. From regional context like that all the way down to personal interests, people are bound to be different. So I'm just going to use LLMs. The most common excuse you hear around this is the whole "reinventing the wheel" thing.
So yeah, I'm going to use LLMs. Because I recognize that I bias myself toward only thinking about what I want to think about. And I know that bias isn't cognitively healthy. But on the flip side, I think what the world values, whether it's knowing a lot or knowing one thing deeply, is going to change.
Honestly, I don't know what's right. I think both the advocates and the critics are making valid points. I respect the people who don't use it, and the people who do just have their own workflow. There's really no reason to fight over whose workflow is superior.
I spend my days yelling at various AI agents for lying, misunderstanding, or misrepresentation of facts. It feels like a modern spiritual process.
Criticizing use of agents for skill atrophy is valid, it definitely atrophies blank slate coding ability, though I don't think it atrophies engineering abilities unless you just YOLO all decisions to the agent. The data center/oligarchy complaints are also valid.
Saying agents produce shitty code is a bad argument though. They produce shitty codebase organization, but at a micro level their code is solid if not elegant. If you let them turn your codebase into a spaghetti mess, that's on you.
[flagged]
[flagged]
[flagged]
The trust problem feels more important than whether the code was AI-assisted. A small, reproducible change with clear tests is reviewable; a huge opaque diff is not, regardless of who typed it.
This sounds like typical german small mindedness and self-importance.
The tech world does not care about woke ideology, german technical illiteracy and self importance.
LLMs are useful and here to stay.
[dead]
This is like meat eaters who wish they were vegan but state that they're not mentally strong enough or whatever. Incredibly annoying. Either shut up or convert.