Anecdotally, my 81 year old father was able to point his phone at the boiler and ask Gemini what the error message meant. It correctly identified the boiler model, identified the error code being displayed on the screen, explained what it meant, provided him with a way to confirm the problem by checking the water pressure guage and then asked to be shown the underside. It again correctly identified and described to him the position and colour of the filling loop lever and how to adjust it to start and stop the water flow. He was able to do all this instead of spending hundreds on getting a plumber out at night because it was during a cold spell and he needed the heating on. That kind of thing is no fantasy and was amazing to witness.
edit: I had already looked up the error code the "old fashioned" way using Google to find the boiler manual, so I would have stepped in if needed, but it was literally flawless and I can't think of any time anything like that hasn't worked when I've used it for similar diagnostics.
Usefulness depends on intended job to be done. A hair dryer isn't very useful at drying clothes.
If someone spent a lifetime mastering woodworking with hand tools, and then was shown a couple very early rudimentary power tools (lacking safety features, crude features, etc) they would rightly conclude they weren't useful. The artisan can do better work faster with less risk of dismemberment without them.
Prior to LLMs, the world's demand for good software was bottlenecked by access to competent software engineers. The people who want software just want it, they don't care about the craft. They have a different job to be done than the engineer.
An example this reminds me of is a jobs to be done theory thought exercise:
Two different first time home owners need to store yard working tools in their backyard, and determine they need a shed. The first one cares most about minimizing the time it takes to get the shed. The second one has some special constraints to deal with AND also wants to start developing their amateur construction skills. They both need sheds, but they have different values, so:
- the first one buys a shed-kit made of plastic panels that can easily be assembled in 20 minutes.
- the second one buys a power saw, power drill, tool belt, saw horses, lumber, screws, metal roofing, etc and builds a custom shed from relative scratch over a few weekends.
Another example is getting take out vs cooking the meal yourself. There are many many examples.
LLMs are already useful to many. They are also not useful to many others. To assume they aren't useful to anyone just because they aren't useful to you is a sign of absent cognitive empathy. Not acknowledging that other people have other priorities and values they are equally valid to your own.
I agree. Although the author didn't delve into it he seems to hint that it may not prove useful enough to the companies that spend money on tokens to justify the huge investments in AI.
For me yes, it's very useful. But what remains to be seen is is it useful to the entities and people willing and able to spend huge amounts of money in AI in exchange for increased productivity or profit... And if so is it useful to generate a cash return on the huge cost investments in datacenters that AI demands
The author of the article mainly talks about agentic programming, code generation, and reasoning. Ans very rightly identifies a big problem with agentic programming, in my experience. If developers can't maintain the software without AI, it's doubtful they can steer AI to maintain it either. Maybe this is not true and we can tell AI something like 'reduce the number of lines of code' until the essential software is exposed and pared down to a quantity and modularity that humans can then participate.
No doubt, most of the value creation is outside of creating software. But if Nvidia and Anthropic do succeed in making better hardware and better software, then the positive reinforcement loop does seem like it could take off. And coding is a big part of that.
Maybe we don't need to understand the code at all? Hard to fathom.
> Maybe we don't need to understand the code at all? Hard to fathom.
Doesn't seem too crazy. This is the life of every customer (and most managers) of a software dev. To fathom it, think of yourself as the unicorn customer that can define projects, specs, and methods with competence, and even make architectural calls, with the AI being the dev.
To me, this is the incredibly clear eventuality of our profession, within the next 5-10 years. You writing code will be the same as most devs who try to hand write ASM: you will very very rarely be able to beat the compiler at its own game. And, just as now, there's a very small chance you'll go read what compiler produces.
I'm a decent full stack web developer, who knows almost nothing about kernel architecture or machine code. Before LLMs I could make a functional app, despite not understanding several layers the app ran on top of.
Determinism is NOT a requirement, as proven by your existence of software devs. Again, shift the perspective to the people you work for, or to your fellow dev. They get along fine with you, who are also not deterministic. They check your output by setting requirements that you can prove are true in a way that satisfies them. You yourself guarantee correctness by tests, because you learned long ago that you yourself are not to be trusted, and are not capable of writing 100% correct code on first pass, except for the most trivial tasks. Your lack of determinism is built into your workflow, with tools like linters, code analysis, etc, to help deal with it all.
Not really. It is true that supply chain attack concerns has put increased pressure on compiler makers to make compilers deterministic, but historically they haven't been, generally speaking. It was long thought that determinism didn't matter. Where compilers do differ from LLMs is that compiler output correctness is binary, while LLM output correctness is probabilistic.
However, compilers have long suffered from inputs not being correct, so we have put a lot of effort into finding ways to validate that the input is correct relative to the output. Which turns out works just as well for LLMs, to the point that validating that the input is correct relative to the output ends up offering the same assurances for both compilers and LLMs alike. Compiler output correctness alone is not actually all that beneficial in practice because, again, it is only as useful as the correctness of the input and inputs are not trustworthy without "double entry accounting" to go along with it. Once you have "double entry accounting" in place then you already have what you need to turn LLM correctness into being binary as well.
> A hair dryer isn't very useful at drying clothes.
More useful for drying clothes than a blender would be though. Or an LLM for that matter. No real point here I just thought it was an interesting example you used because it's still something that does work if you're lacking better options
> Another example is getting take out vs cooking the meal yourself
Interesting analogy
Buying ingredients and Cooking for yourself is almost guaranteed to be cheaper and healthier for your body in the long run. It's pretty difficult to eat out cheaply and very difficult to eat out healthily.
I dunno, might turn out to be some parallels there. I suspect that much like eating too much takeout, LLM usage makes us less fit.
As someone who cooks most of my own meals (and almost never eats out) I agree. I also know quite a few people who don't know how to cook at all and the thought of cooking stresses them out.
So far it seems that we can't all be good/capable at everything.
Maybe you write all your own code and cook your own meals, but do you do your own taxes? would you defend yourself in court? would you fix your car yourself?
Some people may answer yes to all the above, but most people outsource most things. Many people can't afford to outsource to other people, but can afford to outsource to LLMs.
Again it comes down to the job to be done:
- does the person want to develop or practice a skill, or do they just want it done? What is their budget, and how important is it to them to minimize certain risks?
> does the person want to develop or practice a skill, or do they just want it done
I'm probably biased, but I think when it comes to professional work it should always, 100% of the time be the former. People who "just want it done" are the reason every piece of commercial software is becoming dogshit
I seriously don’t understand the naysayers around here. Have they just not used anything past gpt4? You can’t just outsource all of your thinking to them, but they’re obviously useful.
> At first this seems far-fetched, but consider what happened to Facebook’s Metaverse. For a brief window of time it actually seemed reasonable to believe that we would all be spending most of our waking hours with high-tech ski goggles strapped to our heads. That we would work, relax, and socialize with these bulky headsets tricking our brains into thinking they were in a different world.
It's so interesting that people are so divided on this. He even said he generated multiple applications. Yet he has found a way to just about dismiss it somehow.
AI is not going to go away and it's not going to stop improving. That would go against the entire history of computing.
We have levels of improvements in the R&D pipeline in every area: hardware, software, model architecture and training. The models will get larger, architecture more sophisticated, computation dramatically more efficient. New materials, paradigms, more efficient nano-devices, scaling up manufacturing for better devices that are already out of the lab, etc. are in progress pointed towards multiple orders of magnitude efficiency gains. And by the way, that is not at all unusual -- we have been making large and small innovations in computing efficiency for decades.
There are still some things lacking in AI -- it still is jagged intelligence. Give it a few years, people will be nostalgic about the time when humans could still point to some victories here and there. That doesn't require any major breakthroughs -- just continuing to increase the size of the models and improving the training.
I guess, just to play devils advocate, what if code turns out to just be the minimally expressive communication medium required for a human to maintain agency over the function of a system?
If we consider (and I’m not saying this is true, just for arguments sake) that this agency is important as other humans enjoy talking to humans about a goal, rather than machines, and we think this will never change.
In this peculiar case, it actually doesn’t matter how “good” llms get - they will never cross a barrier fundamental to human nature?
There are already extreme edge case uses for AI that are incredibly, freakishly efficient compared to literally any human. Just for say, book or film recommendations... If you want obscure in the sense that no critical re-appraisal or any kind of cult following or curiosity towards it doesn't exist at all period, I'm talking about movies with literally 7 comments across the entire internet. Great films, loved by noone, usually too old to catch any meaningful wave by now... Nothing will ever compare. I've not been able to read nearly as many books as I've watched films but every single recommendation has been fantastic and tailored to my taste in such a way that only very few people have ever achieved, and even then those people are not capable of giving you a brand new list of books to read or movies to watch every single night.
In a broader sense, computational chemistry and drug discovery is already exploding- very, very hard to see that genie being put in the bottle. They've probably created such a large backlog of potentially useful outputs that it would take centuries for humans to sift through.
Useful AI is already here! Get over it. Your term "jagged intelligence" is very good, I will have to write that one down. Jagged indeed.
Sounds like author is currently at the point of learning to use AI to develop mental models (simply continue?)
I think one of the issues with LLMs that confuses people is that they're such a broad open-ended tool that people don't know what to do with them and so conclude they're not sure if they are useful, but in a meta way you could just keep asking the AI questions like "how do I make good use of LLMs in this case" and so on
They seem like an upgraded kind of search engine for text prompts, if nothing else
Personally, I've found that understanding what the AI generates well enough to make substantial changes to it hasn't been too difficult, though it'll obviously depend on how complex the project you're working on is. Effectively speaking, it's like using any sort of off-the-shelf solution and then modifying that solution; you don't need to understand it to use it for your project, but you'll need to do so to make significant changes or add extra functionality.
Practically speaking, is there really a difference between prompting an LLM to create a CMS for a website and installing something like WordPress or Ghost? The setup requires no understanding of the underlying software, but you'll have to build up a mental model from scratch if you want to extend or modify it later.
Of course, it's not going to please anyone that likes the process of programming or knows enough to tell that the output is mediocre at best. But it's very useful for people that don't know how something works and want a 'workable' solution that functionally does what they need. It's basically the next iteration of the WYSIWYG web design tool, the CMS, the site builder, etc. A professional software engineer would be horrified by the code they output, but a non-technical manager type wouldn't notice or care.
> Practically speaking, is there really a difference between prompting an LLM to create a CMS for a website and installing something like WordPress or Ghost?
There is some difference; when you install WordPress or Ghost, your problems are shared. There's a community that goes along with the software and a "shared understanding in the world" of how it works and where the rough edges and limitations are. A lot of the time, LLMs will actually be better at modifying WordPress or Ghost to do what you want than they will be at fixing the CMS you built last Tuesday.
When you custom build, you can get an exact fit for your needs but your misery is yours alone.
I find it baffling that some people had to find this out the hard way. You already knew that it's — if not more work — then at least more tedious to study existing code than write your own.
People have been choosing greenfield rewrites over grokking legacy code since forever.
AI has useful features but the majority of use cases are not useful but are thrust upon us as an excuse to extract more data.
Even if the worst happens in the AI market some uses will still remain. Far fewer, I suspect, than will be demand the computational capacity of all the DCs popping up everywhere.
The hypothetical is that everyone will eventually agree with the author that AI is not useful. I don't see what could possibly change everyone's minds at this point. People have too much direct experience with it. At best, it might remain controversial.
It is most certainly not obviously useful. To anyone who actually cares about quality and not just turning out slop, you have to spend so much time reviewing the code the machine generated that you wind up not saving any time. There are lots of people who don't give a damn about quality, but those were terrible developers before LLMs and remain so now.
Even if you have a moral opposition to a single line of LLM-generated code marring your codebase (not even unit tests!) it can be incredibly useful to assist with reading code.
Take a complex, sophisticated codebase you wrote, that only you have deep understanding of, and throw Codex or Claude at it and ask it questions. How does the Foo work? What happens if the user tries to Frozzle a Bar? Give me a guided tour of how the Baz feature was implemented, top to bottom of the stack.
Personally, I was amazed at how accurate the results were, on a codebase where I knew the answers already. Try to pretend that's not a useful tool to use when you have to fix bugs or add features to a large codebase you aren't already familiar with.
It's delusional for someone to contemplate that millions of developers out there are just churning out 'non usable stuff' and don't care about quality and that somehow you have some 'magic insight' they don't have.
And lacking in self awareness to publicize that.
Obviously some people are using it recklessly, it's the same with everything.
Do you think it's more likely that 'the entire world is wrong and you are right' ... OR ... that you just personally have not figured out how to use the technology in way that makes sense and/or you have a bad sense of of what quality means in most contexts.
AI produces easily 'better than human' code where it has a lot of guardrails, and is probably 10x better than humans at hunting down and fixing bugs.
Software is 90% 'maintenance' , and a well maintained code base provide ample basis for the AI to understand what design and formatting to use for a given solution.
It definitely falls down on it's own architecture, it's not that hard to work along with the AI.
It doesn't matter what we think of Altman or whoever, or whatever or AGI or 'the bubble' - that's all noise. It's literally the most blatantly obvious 'useful' thing to happen to tech since the advent of machines - it takes a 'collapse of intellectual contextualization' to not realize that.
You're watching cars replace horses, they are not the same thing.
> Do you think it's more likely that 'the entire world is wrong and you are right' ... OR ... that you just personally have not figured out how to use the technology in way that makes sense and/or you have a bad sense of of what quality means in most contexts.
Yeah, see, I've seen this movie before. I've seen it with Lisp. "You just haven't used it enough to reach enlightenment! We all have! Keep going; you'll see it too!"
And then with Haskell. Exact same looking down on everyone who doesn't agree as unenlightened, as people who Just Don't Get It.
Both Lisp and Haskell are useful languages. Neither is the future of mainstream programming, not just because the mainstream is unenlightened, but because those languages don't fit the way most programmers think.
People are different. Different languages fit different people. If Lisp fits how you think, when you find it after wandering through procedural languages, it feels incredibly freeing. But if it doesn't fit...
So I get very nervous about "you just haven't figured out how to use it" claims. Maybe it's true. Or maybe it just doesn't fit a bunch of people, and browbeating them with claims that they're unskillful does not make it fit them any better.
> you have to spend so much time reviewing the code
Why? Unless you foolishly choose to target something like Javascript, or Python, or the like where the training corpus was full of poor quality code then the LLMs stay pretty true to how you'd write the code yourself. LLMs are only as good as what they were trained on, but luckily there was already a strong movement towards "all codebases should be the same" before LLMs, and as long as you hone in on those ecosystems then LLMs trained on those ecosystems can work quite well at producing quality code without the need for heavy oversight.
Why? I agree if you have chosen an ill-fitted technology stack where those aforementioned developers who didn't care about quality before LLMs tainted the training data then you do have to carefully read the code to watch for the LLM going crazy, because they will, but if you have chosen a suitable technology stack where the training data is overwhelmingly of high quality, then what need is there to read it? You already know that the quality will be on par with what you would have written yourself.
You might find it to be an interesting read, but the software will be fine if you don't.
The problem he's describing comes up any time you have a team of developers. That doesn't mean the usefulness of teams is a fantasy because you didn't write all the code yourself.
If you don't have a useful mental model of the architecture that's a communication / review / documentation problem, not a problem with the entire idea of codegen.
> I didn’t have a mental model for the thing that was in front of me. If there is a bug, or if a new feature needed to be added my mind was precisely where it was before I started prompting, and I couldn’t even begin to make changes until I had built a thorough understanding of the code.
Yes. Like working in a team. It can be hard to work on a team and to have to understand what your colleagues did, and how to fix or extend it. What if working in teams is a fantasy?
Millions of developers all day every day doing that for real, at some of the best companies in the world, but sure, you have the deep insights they have not figured out?
Anecdotally, my 81 year old father was able to point his phone at the boiler and ask Gemini what the error message meant. It correctly identified the boiler model, identified the error code being displayed on the screen, explained what it meant, provided him with a way to confirm the problem by checking the water pressure guage and then asked to be shown the underside. It again correctly identified and described to him the position and colour of the filling loop lever and how to adjust it to start and stop the water flow. He was able to do all this instead of spending hundreds on getting a plumber out at night because it was during a cold spell and he needed the heating on. That kind of thing is no fantasy and was amazing to witness.
edit: I had already looked up the error code the "old fashioned" way using Google to find the boiler manual, so I would have stepped in if needed, but it was literally flawless and I can't think of any time anything like that hasn't worked when I've used it for similar diagnostics.
Seems we are many to have had the exact same experience with the boiler and an LLM (Claude in my case). +1.
> That kind of thing is no fantasy and was amazing to witness.
How about the case where the "AI" misided the error code?
Did you test that? Result could be even more amazing...
> He was able to do all this instead of spending hundreds on getting a plumber out at night
... or consulting the manual.
I had to find the manual for my parent's oven a few months ago, just finding the model n° was almost impossible
Usefulness depends on intended job to be done. A hair dryer isn't very useful at drying clothes.
If someone spent a lifetime mastering woodworking with hand tools, and then was shown a couple very early rudimentary power tools (lacking safety features, crude features, etc) they would rightly conclude they weren't useful. The artisan can do better work faster with less risk of dismemberment without them.
Prior to LLMs, the world's demand for good software was bottlenecked by access to competent software engineers. The people who want software just want it, they don't care about the craft. They have a different job to be done than the engineer.
An example this reminds me of is a jobs to be done theory thought exercise:
Two different first time home owners need to store yard working tools in their backyard, and determine they need a shed. The first one cares most about minimizing the time it takes to get the shed. The second one has some special constraints to deal with AND also wants to start developing their amateur construction skills. They both need sheds, but they have different values, so:
- the first one buys a shed-kit made of plastic panels that can easily be assembled in 20 minutes.
- the second one buys a power saw, power drill, tool belt, saw horses, lumber, screws, metal roofing, etc and builds a custom shed from relative scratch over a few weekends.
Another example is getting take out vs cooking the meal yourself. There are many many examples.
LLMs are already useful to many. They are also not useful to many others. To assume they aren't useful to anyone just because they aren't useful to you is a sign of absent cognitive empathy. Not acknowledging that other people have other priorities and values they are equally valid to your own.
I agree. Although the author didn't delve into it he seems to hint that it may not prove useful enough to the companies that spend money on tokens to justify the huge investments in AI. For me yes, it's very useful. But what remains to be seen is is it useful to the entities and people willing and able to spend huge amounts of money in AI in exchange for increased productivity or profit... And if so is it useful to generate a cash return on the huge cost investments in datacenters that AI demands
The author of the article mainly talks about agentic programming, code generation, and reasoning. Ans very rightly identifies a big problem with agentic programming, in my experience. If developers can't maintain the software without AI, it's doubtful they can steer AI to maintain it either. Maybe this is not true and we can tell AI something like 'reduce the number of lines of code' until the essential software is exposed and pared down to a quantity and modularity that humans can then participate.
No doubt, most of the value creation is outside of creating software. But if Nvidia and Anthropic do succeed in making better hardware and better software, then the positive reinforcement loop does seem like it could take off. And coding is a big part of that.
Maybe we don't need to understand the code at all? Hard to fathom.
> Maybe we don't need to understand the code at all? Hard to fathom.
Doesn't seem too crazy. This is the life of every customer (and most managers) of a software dev. To fathom it, think of yourself as the unicorn customer that can define projects, specs, and methods with competence, and even make architectural calls, with the AI being the dev.
To me, this is the incredibly clear eventuality of our profession, within the next 5-10 years. You writing code will be the same as most devs who try to hand write ASM: you will very very rarely be able to beat the compiler at its own game. And, just as now, there's a very small chance you'll go read what compiler produces.
I'm a decent full stack web developer, who knows almost nothing about kernel architecture or machine code. Before LLMs I could make a functional app, despite not understanding several layers the app ran on top of.
Those layers are deterministic though...
Determinism is NOT a requirement, as proven by your existence of software devs. Again, shift the perspective to the people you work for, or to your fellow dev. They get along fine with you, who are also not deterministic. They check your output by setting requirements that you can prove are true in a way that satisfies them. You yourself guarantee correctness by tests, because you learned long ago that you yourself are not to be trusted, and are not capable of writing 100% correct code on first pass, except for the most trivial tasks. Your lack of determinism is built into your workflow, with tools like linters, code analysis, etc, to help deal with it all.
Not really. It is true that supply chain attack concerns has put increased pressure on compiler makers to make compilers deterministic, but historically they haven't been, generally speaking. It was long thought that determinism didn't matter. Where compilers do differ from LLMs is that compiler output correctness is binary, while LLM output correctness is probabilistic.
However, compilers have long suffered from inputs not being correct, so we have put a lot of effort into finding ways to validate that the input is correct relative to the output. Which turns out works just as well for LLMs, to the point that validating that the input is correct relative to the output ends up offering the same assurances for both compilers and LLMs alike. Compiler output correctness alone is not actually all that beneficial in practice because, again, it is only as useful as the correctness of the input and inputs are not trustworthy without "double entry accounting" to go along with it. Once you have "double entry accounting" in place then you already have what you need to turn LLM correctness into being binary as well.
> A hair dryer isn't very useful at drying clothes.
More useful for drying clothes than a blender would be though. Or an LLM for that matter. No real point here I just thought it was an interesting example you used because it's still something that does work if you're lacking better options
> Another example is getting take out vs cooking the meal yourself
Interesting analogy
Buying ingredients and Cooking for yourself is almost guaranteed to be cheaper and healthier for your body in the long run. It's pretty difficult to eat out cheaply and very difficult to eat out healthily.
I dunno, might turn out to be some parallels there. I suspect that much like eating too much takeout, LLM usage makes us less fit.
As someone who cooks most of my own meals (and almost never eats out) I agree. I also know quite a few people who don't know how to cook at all and the thought of cooking stresses them out.
So far it seems that we can't all be good/capable at everything.
Maybe you write all your own code and cook your own meals, but do you do your own taxes? would you defend yourself in court? would you fix your car yourself?
Some people may answer yes to all the above, but most people outsource most things. Many people can't afford to outsource to other people, but can afford to outsource to LLMs.
Again it comes down to the job to be done:
- does the person want to develop or practice a skill, or do they just want it done? What is their budget, and how important is it to them to minimize certain risks?
> does the person want to develop or practice a skill, or do they just want it done
I'm probably biased, but I think when it comes to professional work it should always, 100% of the time be the former. People who "just want it done" are the reason every piece of commercial software is becoming dogshit
I seriously don’t understand the naysayers around here. Have they just not used anything past gpt4? You can’t just outsource all of your thinking to them, but they’re obviously useful.
> At first this seems far-fetched, but consider what happened to Facebook’s Metaverse. For a brief window of time it actually seemed reasonable to believe that we would all be spending most of our waking hours with high-tech ski goggles strapped to our heads. That we would work, relax, and socialize with these bulky headsets tricking our brains into thinking they were in a different world.
Literally nobody thought this.
It's so interesting that people are so divided on this. He even said he generated multiple applications. Yet he has found a way to just about dismiss it somehow.
AI is not going to go away and it's not going to stop improving. That would go against the entire history of computing.
We have levels of improvements in the R&D pipeline in every area: hardware, software, model architecture and training. The models will get larger, architecture more sophisticated, computation dramatically more efficient. New materials, paradigms, more efficient nano-devices, scaling up manufacturing for better devices that are already out of the lab, etc. are in progress pointed towards multiple orders of magnitude efficiency gains. And by the way, that is not at all unusual -- we have been making large and small innovations in computing efficiency for decades.
There are still some things lacking in AI -- it still is jagged intelligence. Give it a few years, people will be nostalgic about the time when humans could still point to some victories here and there. That doesn't require any major breakthroughs -- just continuing to increase the size of the models and improving the training.
I guess, just to play devils advocate, what if code turns out to just be the minimally expressive communication medium required for a human to maintain agency over the function of a system?
If we consider (and I’m not saying this is true, just for arguments sake) that this agency is important as other humans enjoy talking to humans about a goal, rather than machines, and we think this will never change.
In this peculiar case, it actually doesn’t matter how “good” llms get - they will never cross a barrier fundamental to human nature?
There are already extreme edge case uses for AI that are incredibly, freakishly efficient compared to literally any human. Just for say, book or film recommendations... If you want obscure in the sense that no critical re-appraisal or any kind of cult following or curiosity towards it doesn't exist at all period, I'm talking about movies with literally 7 comments across the entire internet. Great films, loved by noone, usually too old to catch any meaningful wave by now... Nothing will ever compare. I've not been able to read nearly as many books as I've watched films but every single recommendation has been fantastic and tailored to my taste in such a way that only very few people have ever achieved, and even then those people are not capable of giving you a brand new list of books to read or movies to watch every single night.
In a broader sense, computational chemistry and drug discovery is already exploding- very, very hard to see that genie being put in the bottle. They've probably created such a large backlog of potentially useful outputs that it would take centuries for humans to sift through.
Useful AI is already here! Get over it. Your term "jagged intelligence" is very good, I will have to write that one down. Jagged indeed.
that recommendation is just a reddit comment in a thread similar to your prompt.
Sounds like author is currently at the point of learning to use AI to develop mental models (simply continue?)
I think one of the issues with LLMs that confuses people is that they're such a broad open-ended tool that people don't know what to do with them and so conclude they're not sure if they are useful, but in a meta way you could just keep asking the AI questions like "how do I make good use of LLMs in this case" and so on
They seem like an upgraded kind of search engine for text prompts, if nothing else
Personally, I've found that understanding what the AI generates well enough to make substantial changes to it hasn't been too difficult, though it'll obviously depend on how complex the project you're working on is. Effectively speaking, it's like using any sort of off-the-shelf solution and then modifying that solution; you don't need to understand it to use it for your project, but you'll need to do so to make significant changes or add extra functionality.
Practically speaking, is there really a difference between prompting an LLM to create a CMS for a website and installing something like WordPress or Ghost? The setup requires no understanding of the underlying software, but you'll have to build up a mental model from scratch if you want to extend or modify it later.
Of course, it's not going to please anyone that likes the process of programming or knows enough to tell that the output is mediocre at best. But it's very useful for people that don't know how something works and want a 'workable' solution that functionally does what they need. It's basically the next iteration of the WYSIWYG web design tool, the CMS, the site builder, etc. A professional software engineer would be horrified by the code they output, but a non-technical manager type wouldn't notice or care.
> Practically speaking, is there really a difference between prompting an LLM to create a CMS for a website and installing something like WordPress or Ghost?
There is some difference; when you install WordPress or Ghost, your problems are shared. There's a community that goes along with the software and a "shared understanding in the world" of how it works and where the rough edges and limitations are. A lot of the time, LLMs will actually be better at modifying WordPress or Ghost to do what you want than they will be at fixing the CMS you built last Tuesday.
When you custom build, you can get an exact fit for your needs but your misery is yours alone.
> What if <common sense>?
I find it baffling that some people had to find this out the hard way. You already knew that it's — if not more work — then at least more tedious to study existing code than write your own.
People have been choosing greenfield rewrites over grokking legacy code since forever.
AI has useful features but the majority of use cases are not useful but are thrust upon us as an excuse to extract more data.
Even if the worst happens in the AI market some uses will still remain. Far fewer, I suspect, than will be demand the computational capacity of all the DCs popping up everywhere.
The hypothetical is that everyone will eventually agree with the author that AI is not useful. I don't see what could possibly change everyone's minds at this point. People have too much direct experience with it. At best, it might remain controversial.
It's obviously useful - we have to figure out best practices.
It is most certainly not obviously useful. To anyone who actually cares about quality and not just turning out slop, you have to spend so much time reviewing the code the machine generated that you wind up not saving any time. There are lots of people who don't give a damn about quality, but those were terrible developers before LLMs and remain so now.
Even if you have a moral opposition to a single line of LLM-generated code marring your codebase (not even unit tests!) it can be incredibly useful to assist with reading code.
Take a complex, sophisticated codebase you wrote, that only you have deep understanding of, and throw Codex or Claude at it and ask it questions. How does the Foo work? What happens if the user tries to Frozzle a Bar? Give me a guided tour of how the Baz feature was implemented, top to bottom of the stack.
Personally, I was amazed at how accurate the results were, on a codebase where I knew the answers already. Try to pretend that's not a useful tool to use when you have to fix bugs or add features to a large codebase you aren't already familiar with.
> Try to pretend that's not a useful tool to use when you have to fix bugs or add features to a large codebase you aren't already familiar with.
how will I know the answers are accurate?
It's delusional for someone to contemplate that millions of developers out there are just churning out 'non usable stuff' and don't care about quality and that somehow you have some 'magic insight' they don't have.
And lacking in self awareness to publicize that.
Obviously some people are using it recklessly, it's the same with everything.
Do you think it's more likely that 'the entire world is wrong and you are right' ... OR ... that you just personally have not figured out how to use the technology in way that makes sense and/or you have a bad sense of of what quality means in most contexts.
AI produces easily 'better than human' code where it has a lot of guardrails, and is probably 10x better than humans at hunting down and fixing bugs.
Software is 90% 'maintenance' , and a well maintained code base provide ample basis for the AI to understand what design and formatting to use for a given solution.
It definitely falls down on it's own architecture, it's not that hard to work along with the AI.
It doesn't matter what we think of Altman or whoever, or whatever or AGI or 'the bubble' - that's all noise. It's literally the most blatantly obvious 'useful' thing to happen to tech since the advent of machines - it takes a 'collapse of intellectual contextualization' to not realize that.
You're watching cars replace horses, they are not the same thing.
> Do you think it's more likely that 'the entire world is wrong and you are right' ... OR ... that you just personally have not figured out how to use the technology in way that makes sense and/or you have a bad sense of of what quality means in most contexts.
Yeah, see, I've seen this movie before. I've seen it with Lisp. "You just haven't used it enough to reach enlightenment! We all have! Keep going; you'll see it too!"
And then with Haskell. Exact same looking down on everyone who doesn't agree as unenlightened, as people who Just Don't Get It.
Both Lisp and Haskell are useful languages. Neither is the future of mainstream programming, not just because the mainstream is unenlightened, but because those languages don't fit the way most programmers think.
People are different. Different languages fit different people. If Lisp fits how you think, when you find it after wandering through procedural languages, it feels incredibly freeing. But if it doesn't fit...
So I get very nervous about "you just haven't figured out how to use it" claims. Maybe it's true. Or maybe it just doesn't fit a bunch of people, and browbeating them with claims that they're unskillful does not make it fit them any better.
> you have to spend so much time reviewing the code
Why? Unless you foolishly choose to target something like Javascript, or Python, or the like where the training corpus was full of poor quality code then the LLMs stay pretty true to how you'd write the code yourself. LLMs are only as good as what they were trained on, but luckily there was already a strong movement towards "all codebases should be the same" before LLMs, and as long as you hone in on those ecosystems then LLMs trained on those ecosystems can work quite well at producing quality code without the need for heavy oversight.
For bespoke things you 100% have to read the code, and for very straight forward SaaS stacks you still have to read.
It depends on how much 'rails' there are.
More rails, then there is a lot less to examine.
Why? I agree if you have chosen an ill-fitted technology stack where those aforementioned developers who didn't care about quality before LLMs tainted the training data then you do have to carefully read the code to watch for the LLM going crazy, because they will, but if you have chosen a suitable technology stack where the training data is overwhelmingly of high quality, then what need is there to read it? You already know that the quality will be on par with what you would have written yourself.
You might find it to be an interesting read, but the software will be fine if you don't.
The problem he's describing comes up any time you have a team of developers. That doesn't mean the usefulness of teams is a fantasy because you didn't write all the code yourself.
If you don't have a useful mental model of the architecture that's a communication / review / documentation problem, not a problem with the entire idea of codegen.
> I didn’t have a mental model for the thing that was in front of me. If there is a bug, or if a new feature needed to be added my mind was precisely where it was before I started prompting, and I couldn’t even begin to make changes until I had built a thorough understanding of the code.
Yes. Like working in a team. It can be hard to work on a team and to have to understand what your colleagues did, and how to fix or extend it. What if working in teams is a fantasy?
To me, it is as practical and concrete as it can be. Keep coping
Definitely a fantasy, just one that people try to realize at the moment.
So-called "agentic" AI is definitely a fantasy.
It's like saying a calculator can do your taxes.
Doesn't work that way.
But grifters gonna grift
Millions of developers all day every day doing that for real, at some of the best companies in the world, but sure, you have the deep insights they have not figured out?
[dead]
We can already see (net) useful so-called AI is more than a fantasy.
It's a psychotic delusion.
Useful AI is.
Useful LLM is not.