I hope some of these backdoors and/or 0-days being kept under wraps by nation states or corporate espionage companies (Pegasus et al) are being found incidentally through all this.
It’s frustrating to have all these 0days and CVEs to patch for now, but in the long run perhaps we’ll leave with more robust software.
I may be ignorant, but what do you mean with this and why is it upvoted so much?
To me KISS is basically about how you’d write the code, the architecture, which is something that - at least currently - humans need to oversee when writing new code with LLMs.
But what does it have to do with AI discovering a ton of bugs in a lot of (open-source) software?
Depends on your incentives, most people and companies are measuring Lines of codes and features as the metric so ofc it will increase it, but AI can create small KISS applications. It might have a little difficulties in doing so sometimes because it sometimes likes to overengineer things but I have to wonder how much of it is because it is RLHF'd to solve questions at any costs in terms of lines of code rather than training it in solving it in KISS principles. Also I wonder how AI will perform if we explicitly treat KISS as the only principle and have checks running on it and have it discard all the code which wouldn't follow KISS principle if it doesn't follow the principle.
This is the real reason for the vulnpocalypse, not "technical debt".
NSA spent 20 years stockpiling vulns and pushing the industry towards half-assed solutions like address sanitization and pointer tagging when they could have put the industry on memory-safe languages instead.
They managed to get the IME/PSP/TrustZone into 99% of the CPUs out there by mandating it for government purchases; they could have done the same thing with memory-safe languages. But instead they chose to stockpile vulns. Hope they burn in hell for this.
Why do you think the NSA could have put the industry on memory-safe languages?
Even if they wanted to, I highly doubt they could push the industry to do that. The industry does what it wants to do, and rewriting tons of software was far from its priorities. There were no LLMs back then to help with such a massive migration.
There's also the fact that rewriting software tends to introduce a lot of new bugs.
The U.S. government could have required all new software for any government-procured system to be written memory-safe languages, thus making it ubiquitous. This could have been phased in slowly with a few years of on-ramp time to allow for industry to acquire the relevant skills.
The DoD tried that with Ada and largely failed. That was with defense contractors whose primary customer mandated it. That was despite Ada being proven to work in its first projects.
But even in memory-safe languages, compiler bugs are still an issue. The theory is perfect but the implementation is flawed. In such case once the compiler bugs are found, the software would have to be rewritten to compile. For example an 11 year old memory bug was discovered recently in the rust compiler: https://www.penligent.ai/hackinglabs/rust-borrow-checker-bug....
Memory safe languages are not a silver bullet either.
The U.S. government had pushed Ada for years and it never worked out. The government was forced to accept C++ because industry was rebelling against ADA, even though ADA was memory safe. Well, Ada did have the ability to be memory unsafe. It was much better by default.
The DoD have even created their own language: ADA. Massive contracts power, but different markets to incentivize.
The distance between the usual consumer/b2b tech startup and the kind of company that sells software to the air force is even bigger than the distance between Google and the kind of company who hires Accenture.
As someone who has been programming for about 25 years now and has always been frustrated by the tendency of software devs, and in particular web devs, to not address the root issues but just scaffold on top of them and try to hide the ugly bits under the carpet, I feel a bit vindicated.
Everybody in the industry knows that we've been building on quicksand. Moore's law meant that we could just sacrifice performance and add a dozen layer on top of technology from the 90s (if not older) and keep the thing chugging along.
Look at the PHP situation:
- HTTP is an old and quirky protocol with a lot of surface open for attackers that are rarely useful nowadays and might be better left off to higher layers. But of course, we can't break compat, so everybody still implements all the naughty parts, just in case.
- PHP was a joke language from the start, we were making fun of it in the early 2000s, the fact that over 20 years later it still powers a huge chunk of the web is enough to demonstrate that we're not serious people. Cue the PHP fans telling me that "it's good now". Maybe, I don't really care, the effort that went into making it good should have gone into ditching PHP for a better language 20 years ago. The bandaid mentality is what got us in this situation.
- And of course to add insult to injury, PHP, this security-critical language, is still implemented in garbo C using things like setjmp/longjmp. Just to make sure to absolutely maximize the damage if a vuln is discovered, I suppose. Brilliant.
- Oh and all of this is still running overwhelmingly on top of x86 which is its own bag of legacy garbage. "No but you don't get it, modern CPUs are really more like RISC CPUs, the RISC ISA is translated internally" yeah exactly that's my point let's stop doing things like that.
We're a clown industry. We only made it this far because, usually, buggy code doesn't kill people and we can iterate and patch things quickly. We never had to do things correctly, we could flourish with abysmal levels of quality acceptance. This may not be true anymore.
Of course the reason things are done this way is generally to maintain backward compatibility and make transition easier. At least now that we have LLMs this excuse will hopefully vanish: porting even huge codebases to a new language/framework, something that was a tremendous undertaking mere months ago, is now vastly easier and faster. We can leave the 1990s behind us.
It's coordination problems and large scale distributed action, not some moral character of how serious the people are. Effort is not fungible. To make php good, you only have to act at one point, in a localized way, at the php foundation. To ditch php, you have to act in a coordinated way across any resource strapped companies and individuals. Just see how hard it was to ditch Python 2 or how ipv4 is still not ditched.
Trying to understand underlying reasons and incentive structures is much more productive and useful than simply venting about the worse moral character and seriousness of people in one industry vs the next. Curiously, the grass seems always greener - when you ask people in other industries, you'll often hear endless complaints about the state of things there too.
I'll generally agree, and I include myself in this industry, but we've simply have been bound by compatibility. I don't think anyone chose intentionally to keep the crap around; it was simply never worth it to pay the "break compatibility" cost, until now.
I get your drift but... This sounds a bit like advocating "why not ditch all progamming languages and build some super language that makes no exploits possible". While we are at it, ditch the Von Neumann architecture. Or saying "why not ditch all natural languages in favor of Esperanto".
> PHP was a joke language from the start.... Cue the PHP fans telling me that "it's good now".
First, I am not a PHP fan, but I will absolutely stick up for the "its good now".
Yes, absolutely no debate the old PHP 4.x/5.x was shocking. However to give PHP their due, they absolutely did make serious efforts in 7.x/8.x. They deserve due recognition for that.
Unfortunately lots of people – I'm guessing you're one of them – still have battle-scars from 4.x/5.x and they think (incorrectly) that 7.x/8.x is "more of the same".
Second "joke language", well, cast your mind back to what else was available at the time... ColdFusion, CGI, ASPX ... it was not exactly a beautiful world back then.
Sure, to borrow your terminology, the Java fans will be along soon to tell us all how awesome Java is for the web. But many of us still bear battle-scars of the horrors of Java stacktraces, messing around with incomprehensible XML config files, Java's penchant for eating RAM for breakfast, and generally trying to figure out why the hell Tomcat is not working (again !).
Wait a second, in very recent years the major Java projects I touched were still messing around with incomprehensible XML config files. Has that actually changed in some part of the industry?
> Has that actually changed in some part of the industry?
I suspect the only thing that's changed is that Oracle now charge you for the pleasure. ;)
More seriously though, its been a long time since I've touched Java, but I suspect the only thing that's changed is that its all hidden in magical kubernetes containers these days.
Absolutely. The industry could have moved towards safer languages, better coding methodology and formal verification years ago (see for example MISRA C as at least a partial movement in this direction). It didn't, because it was cheaper not to.
> the effort that went into making it good should have gone into ditching PHP
What effort? PHP foundation has one employee!
> The bandaid mentality is what got us in this situation.
> We're a clown industry.
We are clown industry becauee nobody would sponsor solid research, infrastructure and fixing all those bugs! PHP is junk literslly written over nights by some volunteers!
It's especially fun when not implementing the naughty bits is its own source of vulns. Your code is simpler and less prone to RCE, but your proxy mishandling or whatever allows for credential smuggling attacks, identity spoofing, request smuggling, cache poisoning, and all the normal issues at the application layer.
>to not address the root issues but just scaffold on top of them and try to hide the ugly bits under the carpet
It's totally the devs fault and not the environments devs have to work in. Deadlines forced down on you until you have to make compromises, 24/7 having to compete with people who actually don't care because you have to meet months end payments somehow. Whole businesses blinded by a tech gold rush.
>We're a clown industry.
I do agree here, but that includes every non dev person.
You don't see the same problems in say, structural engineering as you do in software engineering. I suggest this is due to needing to go through the formal training required of a structural engineer, and membership bodies like chartered institutions, with standards and codes of ethics. We might want to reflect on that
competing with others doing the same thing in a zero sum game. We all just have to simultaneously coordinate to agree not to set unrealistic expectations or deadlines! Sounds easy, right?
You forgot NPM. Whereby we download code from unknown authors on GitHub and run in with full access to our filesystem on our machines. No vulnerability, it's been designed like this, we even have designed scripts to autoupdate vulnerabilites, that's the proof that vulns are not common.
Is it forcing us to "pay off" technical debt or is it starting us on a compulsory endless treadmill of feature-neutral fix-work? Collectively, we just won't know for a while.
It totally feels like the former if you watch it find and fix vulnerabilities or other debt-tasks one after another.
But only so many of these "technical debt" issue are little off-by-one errors that get remedied by a surgically narrow patch.
Many enough require some kind of restructuring or redevelopment of their own and it's going to take time to figure out if AI faces the same dilemma as human developers: where each non-trivial change carries the risk of some new issue, ad infinitum.
If that happens to be the case, and AI isn't able to magically produce perfect software in finite time, then what's really happened is that the threat of hostile AI exploiting our technical debt will force us to always be burning and churning code just to remain a moving target.
The original technical debt, which cannot be paid off, was the decision to use a Von Neumann architectures instead of a Harvard architectures for CPUs.
Basically, have an explicit distinction between data and code at the architectural level, to enforce "Write xor Execute" in hardware. In this way, you would prevent e.g. a buffer overflow letting an attacker injecting shellcode that gets executed later by somehow pointing at it. AFAIK the issue would still remain with some programming languages though.
I was down voted a lot months ago for saying that a lot of open source software had a false sense of being secure with the assumption that someone else would have audited the code. And this AI auditing is going to make the promise of open source software getting audited by others actually real. Nice to see others waking up to this reality.
There was always, in my mind at least, a tension with Linus's Law ("given enough eyeballs, all bugs are shallow).
It's true but the problem is assuming just because something is open source it has received sufficient eyeballs. AI now gives us near unlimited eyeballs so, if we survive the bumpy transition, open source should indeed become very secure compared to closed source.
I don't agree that it was an assumption as part of open source, but that it was a possibility with open source. However, there are lots of assumptions made about computer software in general - part of the problem is having cognitive models of software that aren't necessarily reflected in how the hardware runs code.
Also, closed source software can similarly be analysed with AI, but there's no guarantee that the company involved will deal responsibly with disclosures.
Having worked on both sides of the table, the reality is that it is far too easy to game certs like ISO27001. The risk calculation by companies taking that route is changing right out from under them thanks to Ai tools.
The article takes PHP2Shell as the worst offender of the present wave of vulnerabilities but HTTP2Shell/Brotli2Shell was arguably worse. The consequence over at Brotli may very well be the same, but the timeline shows, if anything, that at least part of reality is actually still very much "humans just sitting on vulnerabilities for months or even years". From the timeline: 8 years in the code, 8 months unreleased (but fix in Git) and at least 4 independent researchers finding the problem and pointing it out.
ironically, AI is also making us create a new whole new level of technical debt - where _no one_ has actual understanding of the code being written and we're just trusting the machine
Indeed, and it's just going to end up as a zero-sum game where the winner is the one who can most adeptly manipulate the machine to cause harm to the other.
Is a nuclear power plant dangerous? Or an airplane? Both are examples of an advanced form of what other entities in their category do.
Of course we should proceed with even more powerful tools. We also need even stronger governance of said tools. We don't want an AI-version of the Chernobyl incident but we do want the AI-version of the benefits that nuclear power gives.
I like to think of it as a new abstraction layer on top of something we've been comfortable with for years. The developers of the past had to trust that human-readable code would compile to the machine-level bytecode they were used to write, and that even though it wasn't as efficient as they could make it, one day the benefits would outweigh the negatives. Today most developers don't understand compilers well enough to even begin questioning if the code doesn't work because the compiler has a bug or because their code has a bug. They have to trust the abstraction layer beneath their current domain, and they have to find ways to prove that their work is correct in that domain (e.g., writing a sufficient number of unit- and integration tests, compiling with different compilers to different targets, etc.).
We, too, have to adjust to this change, this new abstraction layer, because whether we like it or not, the code an LLM will produce in no more than five years will arguably be much, much better than the code any 10x developer could produce even if given months of time and an infinite supply of caffeine.
We "just" need to find a way to confirm that our work in this new abstraction layer - primarily prompt engineering - results in the code functionality that we had in mind, not necessarily the code implementation that we had in mind.
I've spend the past month-or-two on developing a platform scroller game for smartphones using nothing but AI. It has created everything given my instructions, to my liking, from the way the game is played to graphical assets to sound effects. It looks and feels like any other typical studio-developed game you could find in the store, the difference is that I didn't have a studio of professionals working fulltime for months or years with various levels of project management and what not. It's just me. Me and my agentic coding subscription.
Vibe coding originally meant giving an LLM an instruction, having it regurgitate something potentially wholesome, testing it and feeding back the errors for it to fix it. What I've done in the past couple of months is so, so much more than that. I'm a puppet master and my coding agent is my puppet. I don't need to understand the code when I can describe the problem I want solved and it will do it 100x faster than I ever could.
I didn't use code to describe this level of abstration. I used English prose - and a heck of a lot of software engineering theory, mind you. If you know the solution requires a quad-tree and binary search because in this case, that is the best option, then that's an instruction, not a block of code.
Is it a challenge to Agile methodologies? I mean is most software today designed at all? If we want a pm to be able to write half baked feature ideas and expect the devs to sort it out, rather than just paste the ticket into Claude, don't we make this chaos inevitable. Maybe there is a missing formal engineering layer missing between feature request and prompt. Llms seem to do much better when they are told exactly what to do, vs when they are given room to...express themselves
We could rephrase it like this: “AI has removed all the excuses we had not to remove technical debt”
Yes, AI developed code might have vulnerabilities or bugs, but so already has your JSP based web server that has Oracle and God knows what else among dependencies.
we're seeing easier PR approvals when AI is involved. Since changing the code in the future is a lot easier.
Instead of a highly structured engineering process it's more like a gardening process of figuring out what works, what doesn't, what the Business actually wants vs what their requirements were
So more debt because there's more code but it's easier and simpler to change that debt. And in 3 years the product is EOL so shrugs
It also inflated almost all of that debt away, though. So in a way people who built "just ship it" buggy things in the last few years, made the right bet.
The amount of e-mails I get of the form "Important information regarding your xxxxx account" (which essentially is just code for "we've been hacked and your data has been stolen") is quite concerning, but not surprising. Security by obscurity was always big in most companies, as long as the bar to carrying out sophisticated attacks was putting a costly IT professional to the task you could be relatively relaxed as no one would bother to hack your stupid little e-shop, but now carrying out such an attack costs a few cents to a few dollars at most, that changes the game entirely and everyone is a target.
This summer I had my Docker registry poisoned with malware artefacts, which again is something that no sane attacker would do before as it required downloading my images, carefully modifying them and then overwriting them, ensuring they work as before. Now it's just a simple prompt away. I am definitely ramping up my defenses, everything that doesn't need to be Internet connected goes behind double VPN and port knocking schemes, essentially making my infrastructure invisible from the outside, all unknown packet types get hard dropped, end-to-end encryption of all internal traffic, web endpoints behind CDN proxies and WAF providers (as much as I hate doing that), double and tripe security mechanisms everywhere, maximum restriction of privileges, ultrafast automated patching and AI based continuous monitoring, audits and interventions. You should also get used to that, this will surely still get much worse as attacker tooling gets better.
Earlier this year, the cybersecurity firm Calif offered to independently audit PHP, sponsored by OpenAI. Within two weeks, a researcher at Calif, aided by AI tools, found a way to use the out-of-memory bug to run arbitrary code on the host server, allowing a malicious actor to install malware or read critical data. Now, it was no longer a minor bug—it was a critical vulnerability, and the Calif team alerted the PHP foundation to start working on a patch immediately.
So, the headline should read,
"AI is helping us eliminate 40 years of technical debt."
Since AI found the bug, and probably also generated the patch at this point.
An abundance of solutions is a problem for people who sell solutions.
I'm personally not sharing the connotations of negativity towards software development and the term technical debt. I keep reading implications that most of this has always been in some way preventable, but it really is not that simple. How did nobody find these types of exploits 40 years ago? Why are we suddenly shocked and blame the field for this? I think the majority of people severely underestimate the complexity and seriousness of software. Once that perception stops and the economic incentives are right, we can start to look closer.
Just consider how laughable some working conditions in software are and then come to realize how architecture is shaped by the organization communication patterns and not reality. You will see it in a different light then.
What's hillarious is people vibe coding massive AI apps thinking its not creating tech debt and security holes. If anything, AI is discovering existing holes, and creating them faster than ever before in history.
All those "paying down" technical debt with AI, I would advise reading the code _very_ carefully. Sometimes the "solution" which passes some test or adheres to some described functionality is incredibly narrow / brittle, and/or has failure cases that are non-obvious.
Yes I'm using $latest_model on high, its still not there. Unless your threshold for "fixed" is "the 1M line lean proof runs, we don't know how it works", it still requires a lot of attention, hand-holding, and scrutiny, especially when fixing issues / working around failures (where it tends to shortcut or invent workarounds).
And that's the biggest win with AI. With old-fashioned development, it was considered acceptable to defer addressing valid feedback ad infinitum, hiding behind whatever lousy excuse you could come up with. No more.
In my opinion, the whole paradigm how we use computers needs to be replaced. It is like the 80s, but now with internet and a thread larger then anything in computer history before. I remember boot sector viruses and stuff.
My approach is https://tix.im/ - not launched yet. But from expedience: modeling everything as workflows of composeable nix flakes (expert systems from programs, skills, agents, workflows and their snippets, DBs blobs, service descriptions, ...) and then run everything in highly isolated environments. It is not only so much more efficient, reduces errors, increases quality, reduces unnecessary cognitive load, ...
I start to move more and more daily tasks into the systems. Even UI programs become interactive workflow steps, which allow me to attach from anywhere back to a task in an environment optimized for the task.
Dependency updates for example, a workflow. A subgraph from a security scanner expert checks each change in the dependency tree.
I don't mean to be that guy, but your website feels inscrutable to me. I know that a lot of that might be because it's written by an LLM, but the page seems like it's full of private language and is the product of a deep context conversation with your agent that I don't have enough context to understand.
I'm not going to lie, I'm having a great time thinking at the endless discussions around unsafe languages and the relentless smugness of many practitioners claiming everything was fine as long as the responsible and the skilled (them) were in charge; and let's not forget the formulaic "with great power comes great responsibility" closers.
Well now, Great Responsibles, it's clear you too were out of your collective depth and were all paying for your smug hubris. Perhaps take a bite out of that humble pie you've obviously never tasted?
I think we'll look back at this moment as peak C/C++ before they begin a long, slow decline into irrelevance.
I've been bullish on Rust for years because memory safety is simply too important and Rust can essentially prove an implementation correct and solve a certain class of very important bugs. It doesn't solve every problem and there is a complexity cost.
I've also said that writing multithreaded code is simply too complex. In an era of out-of-order execution, different levels of caching, asymmetric cores, hyperthreading (although this is largely dead now?), branch prediction and having to support multiple architectures it's simply too hard for most people to write correct multithreaded code. Period. If you find yourself as a non-library developer creating a mutex or spawning a thread, somebody has screwed up.
This is also why I think the single-threaded async/await model with post-request teardown of PHP is the correct choice for most people.
We're going to go through a dark period of a lot of 0days. As the article notes, this is paying 40 years of technical debt. It will result in a more secure ecosystem however. I believe this will ultimately also hamper black markets, governments and 3 letter agencies who trade in those 0days.
As painful as this all is, it will ultimately be a good thing, one of the few to come out of AI (IMHO).
Yep, the takeaway from this should be that we move to leaner applications that can be actually be formally verified. But what I imagine will actually happen is that we'll patch over these vulns with AI that produces 10x the amount of code, just for the next generation of AI to find vulns in that so we can patch that with more code, repeat ad infinitum.
I hope some of these backdoors and/or 0-days being kept under wraps by nation states or corporate espionage companies (Pegasus et al) are being found incidentally through all this.
It’s frustrating to have all these 0days and CVEs to patch for now, but in the long run perhaps we’ll leave with more robust software.
Hopefully we'll also see us back to KISS principles too. But who am I kidding
I may be ignorant, but what do you mean with this and why is it upvoted so much?
To me KISS is basically about how you’d write the code, the architecture, which is something that - at least currently - humans need to oversee when writing new code with LLMs.
But what does it have to do with AI discovering a ton of bugs in a lot of (open-source) software?
A lot of these bugs are in rarely exercised code paths that probably shouldn't have been written into the software in the first place.
that people will finally start solving issues by simplifying the software rather than generating even more edge cases.
What I have heard from AI it is anything but KISS... More features and more code as fast as machinelly possible.
Depends on your incentives, most people and companies are measuring Lines of codes and features as the metric so ofc it will increase it, but AI can create small KISS applications. It might have a little difficulties in doing so sometimes because it sometimes likes to overengineer things but I have to wonder how much of it is because it is RLHF'd to solve questions at any costs in terms of lines of code rather than training it in solving it in KISS principles. Also I wonder how AI will perform if we explicitly treat KISS as the only principle and have checks running on it and have it discard all the code which wouldn't follow KISS principle if it doesn't follow the principle.
Dunno about your company, mine measures in lines of code removed.
Could the global economy survive what that would do to the B2B SaaS industry?
Plain http 1.0 is KISS. Does not really work that well.
Systems should be made as simple as possible, and no simpler.
That doesn't mean every simple system solves the problems we intend for it to solve. It means that all else being equal, we should prefer parsimony.
This is the real reason for the vulnpocalypse, not "technical debt".
NSA spent 20 years stockpiling vulns and pushing the industry towards half-assed solutions like address sanitization and pointer tagging when they could have put the industry on memory-safe languages instead.
They managed to get the IME/PSP/TrustZone into 99% of the CPUs out there by mandating it for government purchases; they could have done the same thing with memory-safe languages. But instead they chose to stockpile vulns. Hope they burn in hell for this.
Why do you think the NSA could have put the industry on memory-safe languages?
Even if they wanted to, I highly doubt they could push the industry to do that. The industry does what it wants to do, and rewriting tons of software was far from its priorities. There were no LLMs back then to help with such a massive migration.
There's also the fact that rewriting software tends to introduce a lot of new bugs.
The U.S. government could have required all new software for any government-procured system to be written memory-safe languages, thus making it ubiquitous. This could have been phased in slowly with a few years of on-ramp time to allow for industry to acquire the relevant skills.
The DoD tried that with Ada and largely failed. That was with defense contractors whose primary customer mandated it. That was despite Ada being proven to work in its first projects.
But even in memory-safe languages, compiler bugs are still an issue. The theory is perfect but the implementation is flawed. In such case once the compiler bugs are found, the software would have to be rewritten to compile. For example an 11 year old memory bug was discovered recently in the rust compiler: https://www.penligent.ai/hackinglabs/rust-borrow-checker-bug....
Memory safe languages are not a silver bullet either.
Definitely not a silver bullet (consider also that memory management isn'the only source for vulns), but still much better than C/C++.
The U.S. government had pushed Ada for years and it never worked out. The government was forced to accept C++ because industry was rebelling against ADA, even though ADA was memory safe. Well, Ada did have the ability to be memory unsafe. It was much better by default.
The DoD have even created their own language: ADA. Massive contracts power, but different markets to incentivize.
The distance between the usual consumer/b2b tech startup and the kind of company that sells software to the air force is even bigger than the distance between Google and the kind of company who hires Accenture.
>IME/PSP/TrustZone into 99% of the CPUs
Yeah, about that. Its the biggest vulnerability in the industry. Nobody wants to confront this fact, though.
Too much profit to be made in subversion and human rights abuses.
As someone who has been programming for about 25 years now and has always been frustrated by the tendency of software devs, and in particular web devs, to not address the root issues but just scaffold on top of them and try to hide the ugly bits under the carpet, I feel a bit vindicated.
Everybody in the industry knows that we've been building on quicksand. Moore's law meant that we could just sacrifice performance and add a dozen layer on top of technology from the 90s (if not older) and keep the thing chugging along.
Look at the PHP situation:
- HTTP is an old and quirky protocol with a lot of surface open for attackers that are rarely useful nowadays and might be better left off to higher layers. But of course, we can't break compat, so everybody still implements all the naughty parts, just in case.
- PHP was a joke language from the start, we were making fun of it in the early 2000s, the fact that over 20 years later it still powers a huge chunk of the web is enough to demonstrate that we're not serious people. Cue the PHP fans telling me that "it's good now". Maybe, I don't really care, the effort that went into making it good should have gone into ditching PHP for a better language 20 years ago. The bandaid mentality is what got us in this situation.
- And of course to add insult to injury, PHP, this security-critical language, is still implemented in garbo C using things like setjmp/longjmp. Just to make sure to absolutely maximize the damage if a vuln is discovered, I suppose. Brilliant.
- Oh and all of this is still running overwhelmingly on top of x86 which is its own bag of legacy garbage. "No but you don't get it, modern CPUs are really more like RISC CPUs, the RISC ISA is translated internally" yeah exactly that's my point let's stop doing things like that.
We're a clown industry. We only made it this far because, usually, buggy code doesn't kill people and we can iterate and patch things quickly. We never had to do things correctly, we could flourish with abysmal levels of quality acceptance. This may not be true anymore.
Of course the reason things are done this way is generally to maintain backward compatibility and make transition easier. At least now that we have LLMs this excuse will hopefully vanish: porting even huge codebases to a new language/framework, something that was a tremendous undertaking mere months ago, is now vastly easier and faster. We can leave the 1990s behind us.
>We're a clown industry. We only made it this far because, usually, buggy code doesn't kill people
Another industry is does even better: the medical/health industry. It's both a clown industry AND kills people.
It's coordination problems and large scale distributed action, not some moral character of how serious the people are. Effort is not fungible. To make php good, you only have to act at one point, in a localized way, at the php foundation. To ditch php, you have to act in a coordinated way across any resource strapped companies and individuals. Just see how hard it was to ditch Python 2 or how ipv4 is still not ditched.
Trying to understand underlying reasons and incentive structures is much more productive and useful than simply venting about the worse moral character and seriousness of people in one industry vs the next. Curiously, the grass seems always greener - when you ask people in other industries, you'll often hear endless complaints about the state of things there too.
> We're a clown industry.
I'll generally agree, and I include myself in this industry, but we've simply have been bound by compatibility. I don't think anyone chose intentionally to keep the crap around; it was simply never worth it to pay the "break compatibility" cost, until now.
I get your drift but... This sounds a bit like advocating "why not ditch all progamming languages and build some super language that makes no exploits possible". While we are at it, ditch the Von Neumann architecture. Or saying "why not ditch all natural languages in favor of Esperanto".
> PHP was a joke language from the start.... Cue the PHP fans telling me that "it's good now".
First, I am not a PHP fan, but I will absolutely stick up for the "its good now".
Yes, absolutely no debate the old PHP 4.x/5.x was shocking. However to give PHP their due, they absolutely did make serious efforts in 7.x/8.x. They deserve due recognition for that.
Unfortunately lots of people – I'm guessing you're one of them – still have battle-scars from 4.x/5.x and they think (incorrectly) that 7.x/8.x is "more of the same".
Second "joke language", well, cast your mind back to what else was available at the time... ColdFusion, CGI, ASPX ... it was not exactly a beautiful world back then.
Sure, to borrow your terminology, the Java fans will be along soon to tell us all how awesome Java is for the web. But many of us still bear battle-scars of the horrors of Java stacktraces, messing around with incomprehensible XML config files, Java's penchant for eating RAM for breakfast, and generally trying to figure out why the hell Tomcat is not working (again !).
Wait a second, in very recent years the major Java projects I touched were still messing around with incomprehensible XML config files. Has that actually changed in some part of the industry?
You can use annotations to get rid of a lot of XML in server side Java. See Spring Boot for example.
> Has that actually changed in some part of the industry?
I suspect the only thing that's changed is that Oracle now charge you for the pleasure. ;)
More seriously though, its been a long time since I've touched Java, but I suspect the only thing that's changed is that its all hidden in magical kubernetes containers these days.
I remember when that exact line of argument was made about '3/4 was silly, but they're putting a real effort into 5'
> we're not serious people
Absolutely. The industry could have moved towards safer languages, better coding methodology and formal verification years ago (see for example MISRA C as at least a partial movement in this direction). It didn't, because it was cheaper not to.
> the effort that went into making it good should have gone into ditching PHP
What effort? PHP foundation has one employee!
> The bandaid mentality is what got us in this situation.
> We're a clown industry.
We are clown industry becauee nobody would sponsor solid research, infrastructure and fixing all those bugs! PHP is junk literslly written over nights by some volunteers!
> HTTP, naughty bits
It's especially fun when not implementing the naughty bits is its own source of vulns. Your code is simpler and less prone to RCE, but your proxy mishandling or whatever allows for credential smuggling attacks, identity spoofing, request smuggling, cache poisoning, and all the normal issues at the application layer.
>to not address the root issues but just scaffold on top of them and try to hide the ugly bits under the carpet
It's totally the devs fault and not the environments devs have to work in. Deadlines forced down on you until you have to make compromises, 24/7 having to compete with people who actually don't care because you have to meet months end payments somehow. Whole businesses blinded by a tech gold rush.
>We're a clown industry.
I do agree here, but that includes every non dev person.
Time for me to drag this old chestnut back up.
You don't see the same problems in say, structural engineering as you do in software engineering. I suggest this is due to needing to go through the formal training required of a structural engineer, and membership bodies like chartered institutions, with standards and codes of ethics. We might want to reflect on that
The people who set the deadlines are trying to stay competitive so there are contracts with clients so your salary can come from somewhere.
competing with others doing the same thing in a zero sum game. We all just have to simultaneously coordinate to agree not to set unrealistic expectations or deadlines! Sounds easy, right?
You forgot NPM. Whereby we download code from unknown authors on GitHub and run in with full access to our filesystem on our machines. No vulnerability, it's been designed like this, we even have designed scripts to autoupdate vulnerabilites, that's the proof that vulns are not common.
You could just as easily headline this as:
AI is enabling us to fix longstanding software security issues that nobody had the time, skill, or desire to find
All a matter of perspective, I suppose.
Is it forcing us to "pay off" technical debt or is it starting us on a compulsory endless treadmill of feature-neutral fix-work? Collectively, we just won't know for a while.
It totally feels like the former if you watch it find and fix vulnerabilities or other debt-tasks one after another.
But only so many of these "technical debt" issue are little off-by-one errors that get remedied by a surgically narrow patch.
Many enough require some kind of restructuring or redevelopment of their own and it's going to take time to figure out if AI faces the same dilemma as human developers: where each non-trivial change carries the risk of some new issue, ad infinitum.
If that happens to be the case, and AI isn't able to magically produce perfect software in finite time, then what's really happened is that the threat of hostile AI exploiting our technical debt will force us to always be burning and churning code just to remain a moving target.
It's another "we'll see" thing for now, I think.
The original technical debt, which cannot be paid off, was the decision to use a Von Neumann architectures instead of a Harvard architectures for CPUs.
Explain? (Genuine question because I wanna know)
Basically, have an explicit distinction between data and code at the architectural level, to enforce "Write xor Execute" in hardware. In this way, you would prevent e.g. a buffer overflow letting an attacker injecting shellcode that gets executed later by somehow pointing at it. AFAIK the issue would still remain with some programming languages though.
and we don't believe a sterilized virtual machine creates a similar situation?
any respective machine doesn't require full Turing completeness?
any ad-hoc machine allows for mutations and flexibility, which I'm sure is an added strength.
I was down voted a lot months ago for saying that a lot of open source software had a false sense of being secure with the assumption that someone else would have audited the code. And this AI auditing is going to make the promise of open source software getting audited by others actually real. Nice to see others waking up to this reality.
There was always, in my mind at least, a tension with Linus's Law ("given enough eyeballs, all bugs are shallow).
It's true but the problem is assuming just because something is open source it has received sufficient eyeballs. AI now gives us near unlimited eyeballs so, if we survive the bumpy transition, open source should indeed become very secure compared to closed source.
Perhaps it will also challenge the assumption that open source is something you can run a billion dollar business on without re-investing a penny.
I don't agree that it was an assumption as part of open source, but that it was a possibility with open source. However, there are lots of assumptions made about computer software in general - part of the problem is having cognitive models of software that aren't necessarily reflected in how the hardware runs code.
Also, closed source software can similarly be analysed with AI, but there's no guarantee that the company involved will deal responsibly with disclosures.
a lot of open source software had a false sense of being secure with the assumption that someone else would have audited the code
Do you have examples of people saying this? I don't think I've come across idea being espoused...
> ... a lot of open source software had a false sense of being secure with the assumption that someone else would have audited the code ...
I agree on this part, but what's your point here? Are you suggesting closed source is better?
Having worked on both sides of the table, the reality is that it is far too easy to game certs like ISO27001. The risk calculation by companies taking that route is changing right out from under them thanks to Ai tools.
The article takes PHP2Shell as the worst offender of the present wave of vulnerabilities but HTTP2Shell/Brotli2Shell was arguably worse. The consequence over at Brotli may very well be the same, but the timeline shows, if anything, that at least part of reality is actually still very much "humans just sitting on vulnerabilities for months or even years". From the timeline: 8 years in the code, 8 months unreleased (but fix in Git) and at least 4 independent researchers finding the problem and pointing it out.
[0] https://www.bugsink.com/blog/http2shell-timeline/
ironically, AI is also making us create a new whole new level of technical debt - where _no one_ has actual understanding of the code being written and we're just trusting the machine
Indeed, and it's just going to end up as a zero-sum game where the winner is the one who can most adeptly manipulate the machine to cause harm to the other.
This is exactly what has happened the past 50+ years, too, your toolbox was just different in the 90's than it is today.
So if it has happened, should we now proceed with even more powerful and dangerous tools, with a much tighter integration with society?
Is a nuclear power plant dangerous? Or an airplane? Both are examples of an advanced form of what other entities in their category do.
Of course we should proceed with even more powerful tools. We also need even stronger governance of said tools. We don't want an AI-version of the Chernobyl incident but we do want the AI-version of the benefits that nuclear power gives.
I like to think of it as a new abstraction layer on top of something we've been comfortable with for years. The developers of the past had to trust that human-readable code would compile to the machine-level bytecode they were used to write, and that even though it wasn't as efficient as they could make it, one day the benefits would outweigh the negatives. Today most developers don't understand compilers well enough to even begin questioning if the code doesn't work because the compiler has a bug or because their code has a bug. They have to trust the abstraction layer beneath their current domain, and they have to find ways to prove that their work is correct in that domain (e.g., writing a sufficient number of unit- and integration tests, compiling with different compilers to different targets, etc.).
We, too, have to adjust to this change, this new abstraction layer, because whether we like it or not, the code an LLM will produce in no more than five years will arguably be much, much better than the code any 10x developer could produce even if given months of time and an infinite supply of caffeine.
We "just" need to find a way to confirm that our work in this new abstraction layer - primarily prompt engineering - results in the code functionality that we had in mind, not necessarily the code implementation that we had in mind.
I'm still unsure if we won't figure out that the right abstraction level to describe what code means is, well, code.
I've spend the past month-or-two on developing a platform scroller game for smartphones using nothing but AI. It has created everything given my instructions, to my liking, from the way the game is played to graphical assets to sound effects. It looks and feels like any other typical studio-developed game you could find in the store, the difference is that I didn't have a studio of professionals working fulltime for months or years with various levels of project management and what not. It's just me. Me and my agentic coding subscription.
Vibe coding originally meant giving an LLM an instruction, having it regurgitate something potentially wholesome, testing it and feeding back the errors for it to fix it. What I've done in the past couple of months is so, so much more than that. I'm a puppet master and my coding agent is my puppet. I don't need to understand the code when I can describe the problem I want solved and it will do it 100x faster than I ever could.
I didn't use code to describe this level of abstration. I used English prose - and a heck of a lot of software engineering theory, mind you. If you know the solution requires a quad-tree and binary search because in this case, that is the best option, then that's an instruction, not a block of code.
Is it a challenge to Agile methodologies? I mean is most software today designed at all? If we want a pm to be able to write half baked feature ideas and expect the devs to sort it out, rather than just paste the ticket into Claude, don't we make this chaos inevitable. Maybe there is a missing formal engineering layer missing between feature request and prompt. Llms seem to do much better when they are told exactly what to do, vs when they are given room to...express themselves
We could rephrase it like this: “AI has removed all the excuses we had not to remove technical debt”
Yes, AI developed code might have vulnerabilities or bugs, but so already has your JSP based web server that has Oracle and God knows what else among dependencies.
It's also introducing a new kind of technical debt much faster.
we're seeing easier PR approvals when AI is involved. Since changing the code in the future is a lot easier.
Instead of a highly structured engineering process it's more like a gardening process of figuring out what works, what doesn't, what the Business actually wants vs what their requirements were
So more debt because there's more code but it's easier and simpler to change that debt. And in 3 years the product is EOL so shrugs
I think I doubled the amount of tempermanent fixes I pushed in my whole career just in the past two weeks.
It also inflated almost all of that debt away, though. So in a way people who built "just ship it" buggy things in the last few years, made the right bet.
Of course no vulnerability has ever been found by a human or by fuzzing.
AI does find things, but it is a small percentage the overall vulnerabilities.
IEEE is of course and industry owned organization that shills for BigAI, NSA and any other entity with money or influence.
personally have started going the other way: AI-assisted microkernels, and only the programming that is actually needed for the problem at hand
my webserver never needed a million lines of code just to pass bytes to other people
The amount of e-mails I get of the form "Important information regarding your xxxxx account" (which essentially is just code for "we've been hacked and your data has been stolen") is quite concerning, but not surprising. Security by obscurity was always big in most companies, as long as the bar to carrying out sophisticated attacks was putting a costly IT professional to the task you could be relatively relaxed as no one would bother to hack your stupid little e-shop, but now carrying out such an attack costs a few cents to a few dollars at most, that changes the game entirely and everyone is a target.
This summer I had my Docker registry poisoned with malware artefacts, which again is something that no sane attacker would do before as it required downloading my images, carefully modifying them and then overwriting them, ensuring they work as before. Now it's just a simple prompt away. I am definitely ramping up my defenses, everything that doesn't need to be Internet connected goes behind double VPN and port knocking schemes, essentially making my infrastructure invisible from the outside, all unknown packet types get hard dropped, end-to-end encryption of all internal traffic, web endpoints behind CDN proxies and WAF providers (as much as I hate doing that), double and tripe security mechanisms everywhere, maximum restriction of privileges, ultrafast automated patching and AI based continuous monitoring, audits and interventions. You should also get used to that, this will surely still get much worse as attacker tooling gets better.
Earlier this year, the cybersecurity firm Calif offered to independently audit PHP, sponsored by OpenAI. Within two weeks, a researcher at Calif, aided by AI tools, found a way to use the out-of-memory bug to run arbitrary code on the host server, allowing a malicious actor to install malware or read critical data. Now, it was no longer a minor bug—it was a critical vulnerability, and the Calif team alerted the PHP foundation to start working on a patch immediately.
So, the headline should read,
"AI is helping us eliminate 40 years of technical debt."
Since AI found the bug, and probably also generated the patch at this point.
An abundance of solutions is a problem for people who sell solutions.
I'm personally not sharing the connotations of negativity towards software development and the term technical debt. I keep reading implications that most of this has always been in some way preventable, but it really is not that simple. How did nobody find these types of exploits 40 years ago? Why are we suddenly shocked and blame the field for this? I think the majority of people severely underestimate the complexity and seriousness of software. Once that perception stops and the economic incentives are right, we can start to look closer.
Just consider how laughable some working conditions in software are and then come to realize how architecture is shaped by the organization communication patterns and not reality. You will see it in a different light then.
What's hillarious is people vibe coding massive AI apps thinking its not creating tech debt and security holes. If anything, AI is discovering existing holes, and creating them faster than ever before in history.
All those "paying down" technical debt with AI, I would advise reading the code _very_ carefully. Sometimes the "solution" which passes some test or adheres to some described functionality is incredibly narrow / brittle, and/or has failure cases that are non-obvious.
Yes I'm using $latest_model on high, its still not there. Unless your threshold for "fixed" is "the 1M line lean proof runs, we don't know how it works", it still requires a lot of attention, hand-holding, and scrutiny, especially when fixing issues / working around failures (where it tends to shortcut or invent workarounds).
And that's the biggest win with AI. With old-fashioned development, it was considered acceptable to defer addressing valid feedback ad infinitum, hiding behind whatever lousy excuse you could come up with. No more.
In my opinion, the whole paradigm how we use computers needs to be replaced. It is like the 80s, but now with internet and a thread larger then anything in computer history before. I remember boot sector viruses and stuff.
My approach is https://tix.im/ - not launched yet. But from expedience: modeling everything as workflows of composeable nix flakes (expert systems from programs, skills, agents, workflows and their snippets, DBs blobs, service descriptions, ...) and then run everything in highly isolated environments. It is not only so much more efficient, reduces errors, increases quality, reduces unnecessary cognitive load, ...
I start to move more and more daily tasks into the systems. Even UI programs become interactive workflow steps, which allow me to attach from anywhere back to a task in an environment optimized for the task.
Dependency updates for example, a workflow. A subgraph from a security scanner expert checks each change in the dependency tree.
I don't mean to be that guy, but your website feels inscrutable to me. I know that a lot of that might be because it's written by an LLM, but the page seems like it's full of private language and is the product of a deep context conversation with your agent that I don't have enough context to understand.
>In my opinion, the whole paradigm how we use computers needs to be replaced.
>But from expedience: modeling everything as workflows
No thanks. I like private computing. I like all the fun stuff you can do with Computers. I'll never want agent based computers.
I'm not going to lie, I'm having a great time thinking at the endless discussions around unsafe languages and the relentless smugness of many practitioners claiming everything was fine as long as the responsible and the skilled (them) were in charge; and let's not forget the formulaic "with great power comes great responsibility" closers.
Well now, Great Responsibles, it's clear you too were out of your collective depth and were all paying for your smug hubris. Perhaps take a bite out of that humble pie you've obviously never tasted?
And you know how are those technical debts created in the first place? More than 40 years of negligence in IT infrastructure.
What does that mean exactly?
luckily it will help us paying it off too
We've patched more bugs and more vulnerabilities in the last 3 months than entire years at my last job.
I fear it's only going to get worse.
Worse?
Better you mean? Or are you saying you prefer having unfixed bugs?
Worse as in having to stay up late patching more bugs.
I'm sure the NSA has an eye on the groups that are asking AI about security issues in various popular softwares ...
Better sooner than later, to be honest.
I think we'll look back at this moment as peak C/C++ before they begin a long, slow decline into irrelevance.
I've been bullish on Rust for years because memory safety is simply too important and Rust can essentially prove an implementation correct and solve a certain class of very important bugs. It doesn't solve every problem and there is a complexity cost.
I've also said that writing multithreaded code is simply too complex. In an era of out-of-order execution, different levels of caching, asymmetric cores, hyperthreading (although this is largely dead now?), branch prediction and having to support multiple architectures it's simply too hard for most people to write correct multithreaded code. Period. If you find yourself as a non-library developer creating a mutex or spawning a thread, somebody has screwed up.
This is also why I think the single-threaded async/await model with post-request teardown of PHP is the correct choice for most people.
We're going to go through a dark period of a lot of 0days. As the article notes, this is paying 40 years of technical debt. It will result in a more secure ecosystem however. I believe this will ultimately also hamper black markets, governments and 3 letter agencies who trade in those 0days.
As painful as this all is, it will ultimately be a good thing, one of the few to come out of AI (IMHO).
That's a weird way to say creating 40 years of technical debt per year.
Yep, the takeaway from this should be that we move to leaner applications that can be actually be formally verified. But what I imagine will actually happen is that we'll patch over these vulns with AI that produces 10x the amount of code, just for the next generation of AI to find vulns in that so we can patch that with more code, repeat ad infinitum.
The most pessimistic interpretation is that AI is superhumanly good at exposing technical debt and not that good at avoiding it.
We'd be replacing decades of technical debt with debt we'd have no hope of paying.
While we're at it, I'd appreciate easy secure authentication of a device without having to pull in a third party authentication provider. Thanks.
The real slop was the human slop we made along the way.
Yeah but AI also helps in lowering the debt ...