AI is forcing us to pay off 40 years of technical debt
Posted by pseudolus 13 hours ago
Comments
Comment by closetheloopdev 2 minutes ago
If AI's code is sloppy but bug-free, is it better than the alternative?
Comment by Arbortheus 13 hours ago
It’s frustrating to have all these 0days and CVEs to patch for now, but in the long run perhaps we’ll leave with more robust software.
Comment by Loquebantur 11 hours ago
The only way to do that is to make your code provable. That again is only realistically doable using AI.
In the process, people will have to turn the pretense of "software engineering" into reality, making the engineering part explicit.
That only works when humans actually understand what they're doing.
Comment by Quimoniz 10 hours ago
> -- Donald Knuth (if I recall that exact quote correctly)
Just because you have proven the correctness of your code, that does not mean that it is infallible. Especially -- as a lot of HN users point out -- there's intricate complexity that is likely to cause erroneous behavior if several interfaces/complexities get combined. And those might be caused, for example by a 'user' setting the driver to 'db.backend.sqlite' instead of 'db.backend.sqlite3'; and then you have suddenly an unexpected configuration.
Comment by casey2 3 minutes ago
Comment by Loquebantur 7 hours ago
That's why it doesn't suffice for only LLMs to know the relevant stuff.
You can't have a zoo run by animals.
Comment by Revanche1367 7 hours ago
Comment by hinkley 4 hours ago
Comment by jopsen 2 hours ago
There will also be compiler bugs, etc.
The point is fewer bugs.
Comment by hinkley 4 hours ago
Anyone who has ever wielded a wrench should know that implicitly.
Comment by svachalek 6 hours ago
Comment by hinkley 4 hours ago
Inconclusive tests to increase automation and coverage accomplish less than nothing. The opportunity cost, the loss of trust, the conflation of the manual testing surface area, are all worse than doing nothing.
The point of testing is to establish a level of trust. That level doesn’t need to be 100%. It can’t be 100% regardless. So you do what you can, and you solve the remaining problem a different way. But you need to know it’s there to address it, and the smoke and mirrors fight that.
Comment by skydhash 5 hours ago
All languages are built on top of an ISA, whether directly (assembler), or by mapping it to an abstract machine. The you have the dreaded “eval” function. Then because a lot of software lives on top of an OS, there’s all the virtualization that is going on (timesharing, virtual memory, filesystems,…). Memory safety is nice, but it doesn’t help you when the attacker send valid, but malicious data from a compromised source, or by exploiting a logic error.
And for kernel code, where everything is accessible to everyone, it’s much a matter of discipline, especially with so much hardware interrupts and syscalls from userspace going around. Most kernel code has very short lifecycle, it’s the data that is long lived, and it’s a shapeless blob in memory that you have to trust the other party kept in good shape.
Comment by MisterMunchkin 5 hours ago
That’s the whole point, you can find all of these things which would take far too much manual effort.
Comment by telebub 5 hours ago
Comment by hinkley 4 hours ago
Is that what you intended to type?
Software is not required to support the requirements of every potential user. Only the ones they mean to support. That’s why Porsche and BMW ignore people complaining about how expensive they are. You aren’t their customer and they don’t want you.
Comment by telebub 4 hours ago
Comment by hinkley 1 hour ago
Comment by timschmidt 5 hours ago
Comment by solyin 3 hours ago
Comment by moomin 6 hours ago
Personally I think proving Kubernetes, for instance, would be an exercise in pure frustration. There’s just too much to do.
Comment by hinkley 4 hours ago
I say that as someone who sees Knuth as problematic in other respects starting in the 2010’s. Physics governs so much of computational complexity now that I think the idealized form he presents has transitioned from merely holding us back to being crippling. Only game designers, hyperscalers, and an esoteric few of the rest of us seem to understand the vast gulf between theory and practice that has developed in the five orders of magnitude that compute power has experienced since his first edition.
Many, many things that are considered constant in his world are instead logarithmic to the size of the data being managed, or in the case of very large data sets, order n^1/2 because of speed of light and thermodynamics. That vastly reorders the toolbox, because second shelf algorithms now rival or beat the top shelf ones and some options even farther down now become interesting for novel situations.
Comment by Quimoniz 1 hour ago
Regarding your remark "[...] because second shelf algorithms now rival or beat the top shelf ones [...]". That is honestly something I would never have believed back in the 2000s, Windows XP era:
I thought we needed more than the classic mathematical operations and if/else for an operating system that drives the modern world like it had been envisioned by sci-fi in the past 100 years.
That turned out to be wrong: Your modern android with a couple of simple apps is everything a man from 1900 might ever conceivably want from technology.
And it is all if/else/while and mathematical operations.
It turned out you just needed enough of these, to get everything working nicely.
Watching in awe. Watching in awe.
Comment by 27183 6 hours ago
Comment by devin 6 hours ago
Comment by 27183 1 hour ago
I do think that most software out there is unsafe. That doesn't make it useless, but if the effort required to exploit vulnerabilities is decreasing we will probably need to stop building things that way.
Comment by lilbigdoot 8 hours ago
I think that LLMs actually cannot do this right now while people can?
Also there is a lot of defensive programming that has been left to application and runtime layers because of dated security models in our OS and permissions that are way too coarse. It's a lot easier to write a correct program when you don't have to check every path and do that perfectly when the permission system can ensure that no files that haven't been whitelisted cannot be read
Comment by adastra22 6 hours ago
Comment by foopod 6 hours ago
But if the reverse were true it would be impossible to prove. AI is trained on all of humanities solutions - LLMs stand on the shoulders of giants, not the other way around.
Comment by adastra22 2 hours ago
Comment by gfosco 6 hours ago
Comment by Loquebantur 7 hours ago
"Provable" essentially means, you have explicit operators acting on explicit states in an exhaustively defined way that preserves "truth", explicitly defined invariants.
A finite thing you can argue over deterministically.
Obviously, "you" there is an LLM, the machine your code is run on has to be defined in that very same exacting manner and so on.
That still isn't enough, you need abstraction, a hierarchy of categories that preserve your intent and accurately describe the problem space.
"Programming" then is solving an inverse problem over that hierarchy, filling in the blanks sensibly on your way down through the model hierarchy, all the way to atomic instructions your (proven) hardware understands.
Those layers of proper abstraction is what's largely missing currently. Or rather, is merely vaguely approximated.
All of which is to say: yes, the current state is a mess. Resolving that is far from easy.
But you really have no choice, that mess is transparent to large-enough LLMs. Security by obscurity only goes so far.
Comment by AnthonyMouse 7 hours ago
It's transparent to large-enough LLMs given enough resources. Which is the thing that was always true. The difference now is that the level of resources is lower, so we're finding a bunch of things it wasn't previously economical to find.
But the cost still isn't zero. On top of that, the same LLMs are now generating enormous volumes of new code, which both nobody has verified to that level of exaction and nobody has even considered how it interacts with existing and other new code.
Comment by lilbigdoot 7 hours ago
This I can agree with, and my perspective would be that we have theoretical solutions with no ergonomics, and rely on unproven (as in formally proven/validated) mechanisms. WASM + capability passing is enough, but the WASM layer is still built on an OS or runtime that is not proven and you'll have a heck of a time trying to build a product on it
Comment by combinebeatsai 11 minutes ago
Oh yeah? slides baton Pick up that can, Clanker.
Jokes aside, I do want to see the first news of AI going out of its way to acquire and make use of actual currency and hiring a meat proxy. I don't even care what for, can be making paperclips.
Comment by boston_clone 7 hours ago
I think narrowing the scope of this statement would better support your argument, as it is easily disproved in its current form.
Comment by freecodeio 7 hours ago
Comment by solyin 3 hours ago
Comment by drivebyhooting 6 hours ago
Comment by criley2 8 hours ago
Folks are creating some interesting stuff in this space
Comment by dbingham 9 hours ago
Namely:
> The only way to do that is to make your code provable.
Proofs are a mathematical concept. They only work in the abstract world of mathematics where everything is perfectly logical.
Engineering exists in the real world. The physical realm where our mathematics are only imperfect models of what actually exists. Removing bugs through physical engineering requires persistent trial and error until you have found all the places where our mathematical models don't adequately represent the real world.
It's the same in software engineering, which involves cobbling together a myriad of systems not all designed to talk to each other and massive, ever changing codebases where perfect knowledge is impossible.
So yes, we should be trying to engineer bug free code. But that's really more a matter of changing our resource allocation, spending more time and money on QA and doing away with mottos like "Move fast and break things". It's a cultural shift and will only come when the bosses -- the venture capitalists who ultimately control our industry -- are held accountable when things break, and given strong incentives to ensure they don't break.
Comment by AnthonyMouse 6 hours ago
The problem here is that the correct amount of resources is context-dependent.
Suppose you're a small business and you're putting up a website to display the menu for your restaurant and directions to get there. Do you want it to get compromised? Of course not. Does it matter very much if it does? Eh. Should you spend $50M in formal verification to make sure it doesn't? Definitely not.
Whereas if you're developing the software that keeps planes from falling out of the sky, that's an entirely different context.
The problem comes when someone grabs the awful WordPress junk that was suitable for displaying static menus and wants to use it to process credit cards. But even that isn't really the fault of the awful WordPress junk, it's the fault of the credit card companies for designing their financial technology in such a ridiculous way that Merchant A has to be given a secret that allow purchases to be made against the same card at Merchant B, instead of using the decades-old cryptography that would prevent the customer from ever needing to give anyone a secret that enables that wide a scope of compromise.
It comes down to how much work is justified to ensure that the thing not have problems, which is not at all the same across different domains.
Comment by Loquebantur 6 hours ago
You're right about approximations: we gather data and improve theoretical models in an attempt to exhaust that complexity reality poses.
You're wrong about exhaustive knowledge about codebases being impossible. LLMs enable exactly that.
The trick then is to transfer that knowledge in a usable way to what we know about the codebase.
I agree with the necessary cultural shift.
Comment by saalweachter 6 hours ago
It doesn't actually give you the truth of a statement, it just convinces a mathematician of its truth.
That doesn't actually change when computers are brought into things. You can convince a mathematician that an automatic theorem prover constructs correct proofs with another proof, but it's "I think this looks right" all the way down.
Comment by Loquebantur 4 hours ago
Mathematicians are trained to act as the compiler, verifying the code, doing the type checks, and to a degree the state machine "running" those programs.
Human mathematicians use introspection to tell you whether a proof/program does what it is supposed to do. They keep those programs small enough to be run on human brains by modularization, abstracting away complexity behind defined interfaces. They mostly rely on other mathematicians to have checked those sub-modules properly.
By defining that state machine, compiler and the tree of "all" known proofs as programs explicitly on a computer, you get Lean and the according libraries of verified proofs.
A proof-program there is verified when it compiles and passes the deterministic structural type-checks built into the language while fulfilling the requirements stated for its interface.
It all reduces to a few thousand lines of trusted code, the Lean kernel, being correct. It's mostly what humans do too, but actually considerably more thorough.
Comment by bathtub365 8 hours ago
Comment by joshheitzman 8 hours ago
But there wasn't enough demand and it was discontinued.
Comment by WalterBright 8 hours ago
Comment by IAmBroom 6 hours ago
"I decided for myself that X wasn't worth me pursuing as a path for my career." Good for you.
I made the same decision for professional photography and cardiac surgery, but both of those careers still hold value to society.
Comment by WalterBright 5 hours ago
Comment by gmueckl 8 hours ago
Comment by txdv 10 hours ago
We have way more resources now to achieve way higher standards.
Comment by RHSeeger 8 hours ago
Kind of like automated tests in software development. Except the people that pay for the engineering stuff are more likely to pay for the extra tests, because of liability.
Comment by Den_VR 1 hour ago
Comment by Yokohiii 8 hours ago
Comment by ryandrake 8 hours ago
Open Source was supposed to prove this, but as it turns out, there weren't enough human eyeballs, and those eyeballs weren't looking for bugs. LLMs eyeballs see the entire codebase at once.
Comment by drivebyhooting 6 hours ago
Comment by kulahan 5 hours ago
Comment by doug_durham 8 hours ago
Comment by alerighi 8 hours ago
The opposite. The amount of bugs that I've seen AI add to software is tremendous, some bugs are things that even a 18 years old that starts working as a developer right after school wouldn't do.
Also they are unable to fix them, if you ask them to fix a bug they probably introduce 3 other more, and so on, and you arrive quickly at the point that the code is a mess of spaghetti that needs to be rewritten completely (by a human, this time).
On the opposite side AI is good at finding bugs, so I use it more as an advanced linter that tells me if I'm doing something wrong.
Comment by dpark 8 hours ago
If you ask an AI to generate a novel math proof, it will inevitably produce garbage. If you give it the tools to do this (e.g. Lean), evidence is that it can in fact generate meaningful proofs.
To the extent that we cannot prove software correct, this is probably largely a tooling problem.
Comment by icantevenhold 7 hours ago
Comment by dpark 6 hours ago
If you told me five years ago that AI would eventually be generating novel math proofs, I probably would’ve said that it’s not happening in my lifetime. And yet here we are.
I don’t know what it looks like to get to fully provable code, but I don’t see that it’s fundamentally impossible in my lifetime. Especially depending on scope. Will an entire operating system and everything on it be fully proven correct in my lifetime? Maybe not. Could the authentication stack for a product be entirely proven correct? Sure. I can’t believe that.
Comment by skydhash 5 hours ago
A single PC can solve a very complex simulation model in seconds, something that would require years using humans to do. Same as solving a very complicated maze. So if coupling a text generator with a proof verifier is now capable of coming up with a proof, I’m not particularly surprised. But given the cost, and the actual usefulness of the proof, I’m not particularly excited either.
But code is not a closed system, even if we do prove a lot of the internal mechanism, we still can’t know the input (noise), and whether the axiomatic foundation will hold. That’s the reason people don’t bother. The theory of computation is just too loose in its constraints. You just write enough tests to catch egregious errors.
Comment by velox_neb 3 hours ago
Doesn't matter. Just like you can prove something is true for all numbers, you can prove something is true for all possible inputs.
Comment by LordDragonfang 3 hours ago
Code is absolutely a closed system.
"Software" might not be, and definitely not software systems -- but code can have each part of it atomically proven to be correct, assuming you test it with a sufficiently complex simulation model (which you've already granted is within the possibilities for LLMs).
Prove all the code in the software system correct, then prove their interactions are correct, and you've achieved your goal.
(I won't claim this is inevitable with the current tools, but it's at least within the realm of possibility)
Comment by kulahan 5 hours ago
Comment by LordDragonfang 8 hours ago
Comment by btilly 5 hours ago
The quote captured the fact that a human reviewed proof could have mistakes in it. This is very different from a Lean verified proof.
Comment by adrianN 7 hours ago
Comment by 10111001 8 hours ago
> The only way to do that is to make your code provable. That again is only realistically doable using AI.
If you're code is "provable" doesn't that only mean you've proved the code meets its specification? The specification can still have bugs.
Comment by Loquebantur 7 hours ago
Engineering builds models that approximate the real world under practical considerations. You e.g. have to weigh costs vs risks and apply good judgement.
In essence, you try to achieve the best possible with the resources you've got.
Comment by IAmBroom 6 hours ago
Comment by jwpapi 2 hours ago
Comment by c_engineer56 3 hours ago
Comment by Betelbuddy 5 hours ago
Extreme naivety alarm... Explain how code provable allows for bug free software. Would love to understand the intellectual jump there.
Comment by zeroonetwothree 9 hours ago
Comment by dataflow 8 hours ago
But also I would bet those undergo more rigorous testing than a lot of software.
Comment by WalterBright 8 hours ago
Airplanes have much smaller safety margins, meaning a lot more calculation is done, and there's a high emphasis on double checking everything.
Spacecraft have tiny safety margins, which amps up the design rigor to 11.
Comment by jameshart 7 hours ago
The difference is, people who build civilian bridges and buildings are not actually generally expected to build structures that can withstand such attacks because preventing those attacks is considered the government’s job.
So no, they typically don’t undergo ‘more rigorous testing’ than software against security threats. The fact that governments make no effort to protect networked systems from being attacked places the burden of withstanding those attacks fully on the private civilian operators.
Comment by pegasus 5 hours ago
Comment by lelanthran 6 hours ago
I'm skeptical; every line of code is a liability. Humans regularly make the trade-off between adding lines of code vs doing without the incremental feature addition. LLMs don't; everything gets added.
The recent brotli attack should have been a wakeup call - few people use it, no one actually needs it, but the exploit that it carried was present on every system. Everything from IIS to Caddy and in-between, and yet no one was actually using the thing.
It's a liability to add code, whether you're duplicating existing code in the codebase or adding a feature 1% will use.
Comment by MetaWhirledPeas 5 hours ago
LLMs can run with whatever directive you give them. If the directive is lots of code that's what you get. But when everyone gets repeatedly burned by vulnerabilities the directive becomes building bullet-proof code. With the right prompting an LLM can be a lot more diligent about that than we ever could.
Comment by Brian_K_White 5 hours ago
It might be true but it's also a pointless statement that doesn't impact the problem, because it won't happen. And "it won't happen" isn't some unsupported prediction that no one can say. The aggregate behaviour of humans is a known thing, and it's not productive to the actual problem to pretend otherwise. It's only productive an individual looking for an easy excuse to avoid dealing with the problem.
Comment by deepsun 5 hours ago
Comment by jsbg 5 hours ago
By default, yes. But you don't have to let them, and enforcing some code quality was shown to use significantly fewer tokens (can't find the link now).
Comment by syshyk 13 hours ago
Comment by Ekaros 12 hours ago
Comment by simpaticoder 8 hours ago
Comment by RHSeeger 8 hours ago
I could have worked through all that on my own, but having the AI as a rubber duck to talk to - plus the edge cases it was able to point out - was really helpful.
Comment by keybrd-intrrpt 1 hour ago
I wonder if a refactor is even necessary.
Concepts like KISS, DRY, etc. are largely human-centric. They're meant to make code easier to write, understand, and maintain - for people.
Duplicating code was traditionally "bad" because it increased maintenance costs, risked divergence, and added cognitive overhead for human developers.
But with coding agents, some of those tradeoffs start to change. An agent can find and update duplicate implementations in seconds. It doesn't need to maintain the same mental map of a codebase that a human does.
And if we have good specifications and comprehensive tests, behavioral divergence should be detectable and correctable. In fact, duplication can offer an advantage over shared abstractions: isolation. Changes to one implementation don't risk breaking unrelated consumers of shared code, and each implementation can evolve independently.
Maybe we won't need as many high-level abstractions, or eventually even some of the architectural patterns we consider best practices today.
If the cost of writing and maintaining code approaches zero, perhaps we should optimize less for human readability and more for runtime performance, memory efficiency, isolation, and correctness.
We might even regain some CPU and memory efficiency by eliminating layers of indirection, virtual calls, and abstractions that primarily existed to make code manageable for humans.
Perhaps the optimal architecture for AI-maintained software is fundamentally different from the optimal architecture for human-maintained software.
A lot of our software engineering best practices are based on the economics/ergonomics of human maintenance. What happens when those fundamentally change?
Comment by rafaelRiv 1 hour ago
Until proven wrong by coding agents, this is still the case even for agents. Not sure why you are assuming this doesn't apply for the machine
Comment by marcus_holmes 17 minutes ago
I cannot read a 50K LOC code base at once. I mean, obviously, I can physically do it, but I don't have the patience, etc. I need mental shortcuts like abstractions layers to hold even parts of the code base in my head. LLMs don't. They can read that in seconds and hold the whole thing in context.
Comment by ern 1 hour ago
The models don't know that they're not human when they estimate. It's worth telling them that all estimates need to be made in the context of an agentic AI doing the work.
Comment by iamveen 6 hours ago
Now, I wonder if it actually agreed, or if it sensed that I preferred the robust solution and was trained to make me happy.
Comment by RHSeeger 6 hours ago
Comment by eithed 11 hours ago
Same with any other acronyms - if you don't set up guardrails on when given principle should apply you'll get code that you can only complain about.
Comment by bunderbunder 9 hours ago
Oftentimes there are two broad ways to fix a gap in the logic. You can eliminate the source of it, which often involves refactoring but tends to result in less code and cleaner, more comprehensible logic. Or you can leave the gap there, but attempt to render it safe by adding extra logic to detect and handle the error condition. This reduces changes to existing code, but makes the overall system more complicated.
You can’t just indiscriminately favor one approach. Which one of those is best depends on the situation. The deciding factor is often something like a time constraint that you can’t know just by reading the code. You might need to talk to your team before deciding.
But LLMs don’t do that. They always choose option B: slap a bandage on it. Keep on adding more epicycles. In the 2000s we learned that, over time, always choosing this approach inexorably leads to codebases that are brittle, bloated and expensive to maintain.
So the problem, from my perspective, isn’t that we shouldn’t be coding defensively. It’s that LLMs encourage us to do it in a way that we’ve already learned is unsustainable.
Comment by eithed 9 hours ago
I don't think this is anything new, nor LLMs are to blame here. Although - LLMs want to make you happy, or maybe they don't think about what user actually needs, just how to accomplish the goal that was stated; they do stuff without asking questions (unless you constrain them or plan first). I've actively put in a hook for my agent to not make decisions on their own and ask me when it detects it's about to make a decision.
Comment by bunderbunder 8 hours ago
I think where a lot of people run into problems with the KISS principle is they conflate simple and easy. More often than not, simple is hard. Which is maybe a soft reason why agentic coding, which is all about trying to make every task easy, leads so consistently to complexity and bloat.
Comment by ShadowOfThePit 7 hours ago
If you write your prompt as a request, and not carefully as an idea or suggestion, the bot can blindly agree to 'help'. And if you are stubborn about it, you can lose hours on something that cannot work, or make the code infinitely worse.
However, it's also the path of getting better at programming. To learn to take a step back, to ask yourself if it's really sensible. Just that now it also involves questioning the bot's help, as well as writing your prompts to prevent it.
Comment by zeroonetwothree 8 hours ago
Comment by layer8 11 hours ago
Not coding for situations that “do not happen” is exactly what creates vulnerabilities. But I also don’t think that such code is the main source of complexity in LLM-generated code.
Comment by eithed 11 hours ago
Comment by Orphis 8 hours ago
Having it in the lower layer is correct, a proper optimizing compiler should be able to detect there's a single code path to the function and that the checks have already been done, skipping the second check.
Comment by alright2565 2 hours ago
[1]: https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-va...
Comment by eithed 7 hours ago
Comment by layer8 10 hours ago
That’s why I think code needs to document all its assumptions, and in a way that is checked by the compiler to the extent possible. If you document that the values have restricted range, then the AI should be taking it into account.
Comment by eithed 10 hours ago
My main point still stands = LLMs code defensively catering for unknowns that are not loaded into context.
Comment by lilbigdoot 9 hours ago
Comment by ShadowOfThePit 10 hours ago
Yet, I removed 2/3s of the code, because it triple-checked that every file really existed and was valid, that the input form was less than a million bytes in size, that there were propper HTTP status codes for everything, that every possible error from reading data was eagerly checked and caught, and so on.
Like come on, three people will ever look at it. So keep it small and tidy so I can understand it better!
Comment by aruametello 12 hours ago
i am going to quote that a lot...
Comment by waffletower 8 hours ago
Comment by Imustaskforhelp 12 hours ago
Comment by sam_lowry_ 12 hours ago
Comment by layer8 11 hours ago
We don’t have effective objective measures of what would constitute KISS.
Comment by renehsz 10 hours ago
Comment by keybrd-intrrpt 1 hour ago
Impossible for you, me, and other humans maybe. But not hard to understand for LLMs
> I want to see an LLM reduce a complex multi-million line codebase to 1/10th of its size or less without breaking functionality. That would be impressive!
They can and do, if you ask them to.
If you have tests to validate functionality, it can easily reduce and refactory.
But, why are we caring about LOC again?
Comment by stingraycharles 12 hours ago
To me KISS is basically about how you’d write the code, the architecture, which is something that - at least currently - humans need to oversee when writing new code with LLMs.
But what does it have to do with AI discovering a ton of bugs in a lot of (open-source) software?
Comment by boomlinde 12 hours ago
Maybe, when they build a pomodoro timer, people will consider the possibility that the entirety of a browser engine as well as 30 000 javascript dependencies from random unvetted sources introduces unnecessary complexity and a practically infinitely fractal attack surface.
Comment by sznio 12 hours ago
Comment by Tuna-Fish 12 hours ago
Comment by embedding-shape 8 hours ago
Someone not following KISS, might install Windows Server + tons of bloated applications, in order to serve a .asp server, which serves JSON that never changes, or something similarly stupid.
Meanwhile, the KISS developer would put static .json files on a CDN or self-hosted Varnish or whatever, and call it a day.
The first opened up a whole can of potential venues for exploitation, while the second might only have opened up one potential venue for exploitation.
Comment by titzer 8 hours ago
Comment by isoprophlex 13 hours ago
Comment by throw938990 12 hours ago
Comment by mhluongo 12 hours ago
That doesn't mean every simple system solves the problems we intend for it to solve. It means that all else being equal, we should prefer parsimony.
Comment by bunderbunder 12 hours ago
Comment by ChuckMcM 5 hours ago
It is also unfortunate than in more than half the cases where AI provides a 'fix' that fix breaks something else, so my guess is this is going to be 90% human labor to correct, at least initially.
Comment by vulk 8 hours ago
Comment by hnedeotes 12 hours ago
Comment by Zigurd 11 hours ago
A complete plan to reduce vulnerability from untoward influences is more feasible now with agent assisted engineering. You have to be carefully selective about what matters and what doesn't, or the cost and time balloon out of control.
Comment by hnedeotes 11 hours ago
Comment by pnt12 7 hours ago
I don't see China antagonistic to the west - they want autonomy, and don't share many of the values, but they're open to a wealthy West spending lots of money in Chinese goods.
Russia, at least its government, is a bit more, especially recently. But anyway, why reinvent the wheel when a battle tested FOSS is there for the taking?
Comment by ben_w 12 hours ago
Comment by hnedeotes 11 hours ago
Comment by stickfigure 8 hours ago
One does not simply "write a browser"...
Comment by tremon 11 hours ago
HTML is entirely the wrong comparison as it's a document format and not a programming language, and "as long as you can write a browser" is not a simple condition. Do we have any browser (even in development) that isn't written with current toolchains? Or a headless javascript runtime perhaps?
Comment by hnedeotes 11 hours ago
But I also don't understand why (without compatibility layers) a browser should be as complex, or remotely as complex, as currently, specially with all the knowledge in hindsight.
I'm not saying this is easy, doable, or whatever, but that was what I meant.
Comment by aborsy 8 hours ago
Comment by phendrenad2 8 hours ago
Comment by hn_submit 3 hours ago
I predicted long ago that when computers become totally secure intelligence and LEA will lobby for mandatory backdoors, possibly through secret courts.
Comment by walrus01 1 hour ago
Comment by saidnooneever 5 hours ago
in the beginning, ppl didnt know a lot, and default passwords and stack overflows were 0days.
then awareness came and a lot of that is actually fixed. Writing a modern exploit more often now requires many stages and a shit ton of skills and knowledge.
that will increase.
nothing will be 'fixed' in that utopian sense.
Comment by InsidiousWombat 12 hours ago
Comment by lilbigdoot 8 hours ago
Comment by nullsanity 12 hours ago
Comment by octoberfranklin 13 hours ago
NSA spent 20 years stockpiling vulns and pushing the industry towards half-assed solutions like address sanitization and pointer tagging when they could have put the industry on memory-safe languages instead.
They managed to get the IME/PSP/TrustZone into 99% of the CPUs out there by mandating it for government purchases; they could have done the same thing with memory-safe languages. But instead they chose to stockpile vulns. Hope they burn in hell for this.
Comment by FartyMcFarter 13 hours ago
Even if they wanted to, I highly doubt they could push the industry to do that. The industry does what it wants to do, and rewriting tons of software was far from its priorities. There were no LLMs back then to help with such a massive migration.
There's also the fact that rewriting software tends to introduce a lot of new bugs.
Comment by lambdaone 13 hours ago
Comment by elcritch 12 hours ago
Comment by bluGill 12 hours ago
Comment by terminalcommand 12 hours ago
Memory safe languages are not a silver bullet either.
Comment by dns_snek 6 hours ago
Comment by terminalcommand 5 hours ago
Basically, the borrow checker allowed writing memory unsafe programs without using the keyword unsafe.
Compilers for C, C++ etc. are well established but Rust is a relatively new language. There are many bugs to come just like this.
Using a memory safe language is not a silver bullet for writing memory safe code. The implementation of the memory safe language introduces bugs as well.
Comment by dns_snek 2 hours ago
I'm saying that these compiler bugs are extremely low on our list of priorities. If we want to eliminate all vulnerabilities in all of software we have about a century worth of logic bugs that are likely more critical and easily exploitable than compiler bugs.
Comment by GTP 12 hours ago
Comment by elzbardico 12 hours ago
The distance between the usual consumer/b2b tech startup and the kind of company that sells software to the air force is even bigger than the distance between Google and the kind of company who hires Accenture.
Comment by aa-jv 12 hours ago
Yeah, about that. Its the biggest vulnerability in the industry. Nobody wants to confront this fact, though.
Too much profit to be made in subversion and human rights abuses.
Comment by leonidasrup 11 hours ago
" In 2013, The New York Times reported that documents in their possession but never released to the public "appear to confirm" that the backdoor was real, and had been deliberately inserted by the NSA as part of its Bullrun decryption program. In December 2013, a Reuters news article alleged that in 2004, before NIST standardized Dual_EC_DRBG, NSA paid RSA Security $10 million in a secret deal to use Dual_EC_DRBG as the default in the RSA BSAFE cryptography library, which resulted in RSA Security becoming the most important distributor of the insecure algorithm. "
Comment by aa-jv 10 hours ago
People seem to be fine with it as long as their own team seems to be profiting from it - but for those of us who are not subject citizens of the NSA's government, it is a truly heinous state of affairs.
Comment by swatcoder 12 hours ago
It totally feels like the former if you watch it find and fix vulnerabilities or other debt-tasks one after another.
But only so many of these "technical debt" issue are little off-by-one errors that get remedied by a surgically narrow patch.
Many enough require some kind of restructuring or redevelopment of their own and it's going to take time to figure out if AI faces the same dilemma as human developers: where each non-trivial change carries the risk of some new issue, ad infinitum.
If that happens to be the case, and AI isn't able to magically produce perfect software in finite time, then what's really happened is that the threat of hostile AI exploiting our technical debt will force us to always be burning and churning code just to remain a moving target.
It's another "we'll see" thing for now, I think.
Comment by vjvjvjvjghv 9 hours ago
Yes, let's fix the foundation. It will make adding features easier once the debt is paid off. And we will probably learn how to do better in the future. It reminds me of settings compiler warning to be treated as errors. Once you do that, you have a ton of work to do. But if you are smart (not all devs are), you learn why the warnings are there and avoid these problems in the future.
Comment by intrasight 12 hours ago
Comment by victorbjorklund 12 hours ago
Comment by GTP 12 hours ago
Comment by sznio 12 hours ago
and it's not sufficient anyways. even with NX you could have the attacker control a function pointer you call. in any sufficiently complex program you'll have enough widgets to build anything you want just out of data.
code and data are the same thing. whatever inputs your system receives is "code" that your application interprets, just like your application is the code that the CPU interprets. turtles all the way down.
Comment by intrasight 11 hours ago
Comment by rdevilla 11 hours ago
Comment by ihavesomeexampl 11 hours ago
Comment by AdamH12113 4 hours ago
Comment by intrasight 11 hours ago
Comment by esterna 10 hours ago
Comment by lelanthran 5 hours ago
Says you. I wrote a program for a Hardvard architecture chip (PIC) that read bytecode from the data memory and executed it. Updates were made by replacing the data block with a new data block.
In practice, there is no way to separate data and instructions. After all, JS downloaded inside a webpage is literally data, and it gets executed.
Comment by tosapple 12 hours ago
any respective machine doesn't require full Turing completeness?
any ad-hoc machine allows for mutations and flexibility, which I'm sure is an added strength.
2nd edit,
Also, didn't we have a weaker model of this with segmented non-flat memory models on i386-
Comment by AlienRobot 11 hours ago
Comment by SSLy 10 hours ago
Comment by AlienRobot 6 hours ago
That doesn't even make sense, to be honest. Interpreted languages like Python aren't running machine code. They are taking data and executing machine code based on the data. Is Python safe?
Comment by AdamH12113 4 hours ago
In practice, it's usually possible to write to the embedded flash memory via software, but it's much more difficult and disruptive than a simple CPU memory access.
Comment by Merad 5 hours ago
I don't know any details about Python's architecture, but it's totally possible to write an interpreter for machine code; that's basically what an emulator is. The trade off is that pure interpreters tend to be very slow.
Comment by chrisjj 3 hours ago
Uh? It would work via a file - as AOT.
Comment by SSLy 5 hours ago
Comment by rdevilla 12 hours ago
Comment by layer8 11 hours ago
Comment by intrasight 11 hours ago
Comment by joe_the_user 2 hours ago
see: https://www.computer.org/csdl/magazine/an/2022/03/09779481/1...
Comment by 01HNNWZ0MV43FF 10 hours ago
Really, we never should have left the Garden of Eden. Things were great there, other than the maternal mortality rate.
Comment by fnord77 5 hours ago
Comment by hashmap 10 hours ago
Comment by worldsayshi 10 hours ago
Comment by asa123 6 hours ago
Comment by pessimizer 4 hours ago
My problem with even talking about this is that there's absolutely no evidence for it. The idea that coming up with a technology to find holes in the hulls of boats starts with a flurry of holes being found and a bunch of patching to be done does not even hint at the theory that there are an infinite number of holes in all boats, and now that we can find them we will have to spend the rest of our lives patching them.
Code was bad, and we knew it was bad. We would have found most of these bugs if instead of coming up with LLMs, a bunch of countries got together and decided that they were going to put $10B into hiring programmers to do nothing but review all the code in the world for bugs. With AI, we managed to do it for only a trillion dollars.
Comment by keybored 11 hours ago
But there are potential upsides like the already mentioned possible nation state exploits (sleeper exploits).
Comment by naasking 7 hours ago
Comment by Balgair 4 hours ago
I remember reading Godel Escher Bach back in the day, and there is the parable of the record player between the tortoise and the rabbit. And the tortoise keeps building more and more secure record players that the rabbit keeps finding ways to break.
GEB then goes on to teach you that, no, in fact, you cannot have a perfectly secure system. Not just practically or economically, but even in theory, all systems are fundamentally breakable. GEB, after a lot of chapters (what a doorstop!) ties that parable into Godel's Incompleteness Theorem, and essentially proves that no, you can't make unbreakable things, as a fundamental point of logic.
So, when I look at all this, I see that yes, there is an economics point of view here, that fixing code is just expensive, like making a less and less breakable record player. And that AIs push that breaking and economics point much further along than what we have had.
But then I also see a lot of comments here that seem (to me) to be arguing that there is an unbreakable code out there.
Am I reading this right? Is there such a thing as 'perfect' code? Or is my outdated reading of GEB still valid or not?
[0]Note, I'm not a CS person, more a biophysics person.
Comment by wrs 4 hours ago
Godel’s theorem says that unprovable true statements exist. Fortunately there is a vast set of provable true statements that are quite useful in real life.
Similarly, the halting problem is that you can’t write a program that always determines whether absolutely any program you give it will halt. But in real life compilers can still detect infinite loops in your code unless you try really hard to fool them.
In practical terms a software system can never be “perfect”, because first you have to define “perfect” (i.e., specify correct behavior in every possible state), which is practically impossible, and even if you manage that, the hardware has to implement the software abstraction perfectly, and that never happens. (See prefetch attacks, cache timing attacks, supply glitch attacks, rowhammer, etc.)
The security issues come about when someone finds where you failed to specify desired behavior, or (at any level) your system failed to implement it in some circumstance. No strange loop required.
Comment by goodmythical 2 hours ago
Having never written compiled code, this intriques me. I often write one offs that I actually want to loop infinitely until I tell them to stop.
How does the compiler know whether it's just really long or infinite? Like, if I write a progressive approximation of pi or prime search algorithm, both are never going to halt because we're certain there are infinite digits of pi and prime numbers. The program could technically halt, but not as a function of the program but rather the hardware.
Or, being silly, if I write a "flip bits in target directory until all bits are flipped" and target /dev/null as a user of the software, it wouldn't halt, so the compiler would detect that?
Is there some --don't-mind-the-recursion flag for developing a program like ycruncher? I get the feeling that there's a list of "this typically causes infinities" that it checks against and if your use isn't on there well then... So it's just the mouse/mouse trap thing but in a compiler?
Or is there some unique CS going on that the compiler employs to determine certain classes of loops?
Comment by pillmillipedes 40 minutes ago
for(;;) // intentional
for(unsigned i=10;i>=0;i--)
for(int i=0;j<10;i++)
for(unsigned i=0;i!=100;i+=8)
// unintentional
in C infinite loops with no side effects are UB, so detecting them is useful for diagnostics.guaranteed termination also is needed for some optimizations. but since side-effect-less infinite loops are UB in C, a compiler doesn't have to prove that such a loop terminates, it can just assume it does. other languages may differ, but I don't think any of them prove convergence of algorithms.
a don't-mind-the-recursion flag can be `-fno-finite-loops` on clang, or a read or write to a `volatile` variable in standard C.
Comment by tarsierscripter 37 minutes ago
One straightforward approach is to use Hoare logic [1], specifically the method of loop invariants and loop conditions. If you have a loop where the conditions are expressed only in terms of invariants then the conditions will never change truth values during iteration of the loop, thus you have an infinite loop.
Now, some loops are written deliberately like this, i.e. "while (true)", and so there you need to look at whether the body of the loop contains any side effects (reading input, printing, etc). Then you have what is called a "productive infinite loop", such as an event loop or a REPL. Of course you can't prove this sort of loop will ever halt, since the user may never elect to issue the quit command, but these sorts of loops often tend to spend most of their time with the process sleeping until the operating system wakes them up to handle an event.
Comment by lalaithion 55 minutes ago
Comment by rdevilla 51 minutes ago
Comment by zeroonetwothree 4 hours ago
Comment by tennysont 4 hours ago
Clearly our math system gives basic math, so we should conclude that it is either 'incomplete' or broken. Before the incompleteness theorem, it was exciting to imagine that it might be complete, but we had no idea. Now it seems likely to be incomplete. The odds of it being broken haven't really changed.
If it is broken, 2+2 still equals 4 (we didn't really learn that fact from abstract math), but we'll need to fix up set theory. In a world where math isn't formally verified (and even lean itself isn't formally verified), the Incompleteness Theorem is permission to be awed by how huge the space of all logical deduction is, not an argument that logic is a fiction.
To come at it another way, do you believe that there is an 64-bit integer which, any time it is inserted into register 8 on any computer, will immediate convert the raw mass of the earth into a black hole? No, because even if you suspect there is a hole in the foundation of math, you still believe in brute-force case-analysis, and you know that every 64-bit integer has been inserted into register 8 on a computer before.
Comment by hilariously 4 hours ago
Comment by y1n0 57 minutes ago
Comment by Tryk 2 hours ago
Gödel's Incompleteness, very roughly, says that not every true statement (in a formal system) can be proven. In particular, it says that for a logic equipped with arithmetic the statement holds; how you get to security I do not follow.
Comment by rdevilla 41 minutes ago
Comment by rdevilla 4 hours ago
Comment by xbmcuser 12 hours ago
Comment by asdff 20 minutes ago
Comment by adunsulag 11 hours ago
That said, if we were a closed source project, I don't know that our own tooling would have discovered all the issues using LLMs as we are seeing more and more creative ways of people exposing exploits that I never would have thought to put into the LLM prompts.
Comment by MisterMunchkin 5 hours ago
If all you had was an API endpoint, you wouldn’t realistically have any way of figuring out how the internals work and finding obscure exploits.
Comment by keeda 6 hours ago
A 1000 of us regular devs scanning a piece of code is not as effective as a single review by, say, Tavis Ormandy. So "enough eyeballs" is only "enough" when one of those set happens to belong to a Tavis O. or the likes.
Not to mention, this assumes a 1000 of us even look at the code, which is statistically never true. Approximately 100% of us just install the software without looking at a single line of code. Hence the code auditability aspect of open-source software never really applied. And hence, "supply chain attacks."
Unfortunately, this misguided "law" was taken as gospel by many, and as you found out, still is.
Comment by MisterMunchkin 5 hours ago
Instead they court martial anyone who talks about the designs. Because obviously, telling everyone every aspect of your security isn’t actually more secure. How would you know how to do a 24-step process to exploit something if you have no idea what code is running on their server?
Imagine trying to build a cheat for CS:GO if you had no access to the game and could only play it via a remote connection. Good luck.
Comment by wvenable 2 hours ago
Comment by corford 12 hours ago
It's true but the problem is assuming, just because something is open source, it has received sufficient eyeballs. AI now gives us near unlimited eyeballs so, if we survive the bumpy transition, open source should indeed become very secure compared to closed source.
Comment by Terr_ 1 hour ago
Except we won't get the expected benefit, because all those eyeballs are attached to a very small number of different "brains" (LLM models.)
The original saying depends on an implicit assumption that eyeballs are proportional to minds, and that among those varied independent events different people will catch what others miss.
In other words, you shouldn't assume 5.5x review thoroughness simply because the new guy Bobby the Beholder has 11 eyeballs to work with.
Comment by Jyaif 8 hours ago
I don't understand how you get to that conclusion.
Do you think that an open source project will receive more AI eyeballs than a closed source project?
Comment by tcoff91 6 hours ago
Linux kernel will get an insane amount of tokens burned on analyzing it. Most other open source, not so much.
Comment by ptmvp 6 hours ago
Assuming offensive and defensive capabilities are similar, and incentives scale roughly with project size but start from a non-zero value for attackers, I'd expect larger open projects to be well defended by plenty of motivated people, and smaller ones less so (depending on maintainers). Closed source will mostly just depend on maintainers.
Also assuming closed source is less exposed to AI auditing overall, I'd guess large open projects end up more secure than large closed projects, but small open projects possibly less secure than small closed projects, since attackers might have more incentive to target them than defenders do to audit them.
Comment by MisterMunchkin 5 hours ago
Comment by pessimizer 4 hours ago
I'd lean more towards systems that only install diamond-hardened software at all - once you have one project in a class of software that has been analyzed to death, that's the one everybody is going to use.
The vast majority of the development of all projects is going to be (already is) LLM-assisted, so they're going to get analyzed every time they're touched. The odds that huge bugs are going to be lying everywhere to be exploited are bad imo.
I'm starting to rewrite an open source project just because I don't want it to be buggy anymore. I barely want to change anything about it (especially not the licensing.) All I'm doing is porting it to rust from c++ and refactoring it like crazy. I just want to make it easy to audit and extend, and hard to crash, that's it. Once I'm done, I'm already have another (buggy) project in mind to do the exact same thing to. Great UIs and usability, just thwarted by the time that the developers had to invest in them. You could think of them as almost purely security-oriented rewrites.
Comment by weitendorf 5 hours ago
Using open source source software lets you share responsibility and oversight with a large number of people. But it brings with it a much larger attack surface than you'd have if you only implemented the part of that software that you need, and a much more complicated threat model for supply chain risk.
If a large number of people collaborate on open source software for something, especially if it's security or infrastructure related, that makes the project itself a much higher value target than any individual company's home grown mail server or http proxy client would be. Participation makes your vulnerabilities more public, since interacting with the public open source project, making blog posts or hiring ads mentioning it, hosting it (and thus serving traffic in a way that signals or directly tells clients what implementation it's using), bundling it with your site or binaries, etc. tells prospective hackers 1. how to find potential exploit targets for a general open source vulnerability 2. where to look for or introduce vulnerabilities to pwn a particular target.
AI auditing isn't really a silver bullet because it costs a non-trivial amount of money for each commit or PR, and scales with the number of contributors to a project and the project's size. Normally you want more of those with a software project, but in this case it makes AI auditing more expensive unless it's shared across the project and basically part of its CICD (and I don't think that's really been fully adopted yet). But then that introduces other risks like adversarial payloads where the attacker verifies that the AI security checks in place for a project don't trip when they actually send the malicious code for review, because they can run the same checks offline and tweak the payload until it passes.
Personally I think we're headed toward an outcome where only very challenging, well-resourced, and critical open source software is worth using for developers. For smaller things it makes more sense to just use AI to implement your own version of it, or fork/take inspiration from a project and mold it to fit your needs.
Comment by mrkeen 10 hours ago
Comment by jimnotgym 12 hours ago
Comment by vb-8448 12 hours ago
I agree on this part, but what's your point here? Are you suggesting closed source is better?
Comment by bigfishrunning 11 hours ago
Comment by ndsipa_pomu 12 hours ago
Also, closed source software can similarly be analysed with AI, but there's no guarantee that the company involved will deal responsibly with disclosures.
Comment by dfxm12 12 hours ago
Do you have examples of people saying this? I don't think I've come across idea being espoused...
Comment by knorker 11 hours ago
Comment by jansan 12 hours ago
Comment by duskdozer 11 hours ago
Comment by StilesCrisis 9 hours ago
Comment by nullsanity 12 hours ago
Comment by glennfiedler 5 hours ago
Yes, we're going to see the bugs exposed by AI and it's going to suck a bit for a few months.
But when this is all over, software will be more secure than it has ever been before.
Comment by hexapus 4 hours ago
Comment by joe_the_user 3 hours ago
I mean, LLMs themselves come from exactly this phenomena. They are a product of finding a technology that scales and going with that no matter what. And parts of this will hit a wall and have to retrench, current players may implode but the genie isn't going back into its bottle so disruption within disruption will continue.
Comment by asdff 22 minutes ago
Comment by PeterisP 4 hours ago
Comment by rwz 4 hours ago
I don't see sufficient evidence that this is not going to be a perpetual race between people/AIs fixing existing software and more advanced and powerful next generation AIs finding new vulnerabilities and exploits.
Comment by NooneAtAll5 4 hours ago
Comment by srveale 4 hours ago
Comment by softfalcon 4 hours ago
Maybe? I’m not 100% on this one.
Not trying to be a downer, but it could become a mixed bag is all I’m saying.
Comment by Bassiestroep 4 hours ago
Comment by chrisjj 4 hours ago
I know how to ride a bike.
Does your chatbot?
Comment by vanschelven 13 hours ago
Comment by themanualstates 10 hours ago
Comment by fooyc 4 hours ago
Comment by herval 13 hours ago
Comment by keeda 6 hours ago
The world didn't end. It was a complete black box, but we followed the runbooks and kept things running, and only learnt enough of the code as needed to fix any issues or add any enhancements that popped up. Diving in, doing some software archaeology, and spot-changing relevant code became a skill we had to develop.
At scale and over time, I would guess that the percentage of code running on machines that is actually understood by anybody approaches 0. Yet the world does not end.
Comment by herval 6 hours ago
I'm working on a repo these days that's larger than _entire companies_ I worked with in the past. A 10-person team maintaining more code than 100s at a previous job. The _scale_ makes this wildly different from what we're used to, no matter how much "legacy code" you had to maintain before.
Comment by joshheitzman 3 hours ago
Being able to work effectively in large amounts of unfamiliar code was an expectation for mid-level engineers (L62 if I recall correctly). Its not a new skill, but it is increasing in value again.
Comment by scrollaway 5 hours ago
These legacy codebases could be incredibly massive and documenting how to run them alone could be a multi month reverse engineering project.
Now ai can do all that part. Dealing with a legacy or unknown codebase is so much easier…
Comment by joe_the_user 5 hours ago
Comment by Izkata 2 hours ago
Comment by Izmaki 12 hours ago
We, too, have to adjust to this change, this new abstraction layer, because whether we like it or not, the code an LLM will produce in no more than five years will arguably be much, much better than the code any 10x developer could produce even if given months of time and an infinite supply of caffeine.
We "just" need to find a way to confirm that our work in this new abstraction layer - primarily prompt engineering - results in the code functionality that we had in mind, not necessarily the code implementation that we had in mind.
Comment by contravariant 12 hours ago
Comment by Izmaki 12 hours ago
Vibe coding originally meant giving an LLM an instruction, having it regurgitate something potentially wholesome, testing it and feeding back the errors for it to fix it. What I've done in the past couple of months is so, so much more than that. I'm a puppet master and my coding agent is my puppet. I don't need to understand the code when I can describe the problem I want solved and it will do it 100x faster than I ever could.
I didn't use code to describe this level of abstration. I used English prose - and a heck of a lot of software engineering theory, mind you. If you know the solution requires a quad-tree and binary search because in this case, that is the best option, then that's an instruction, not a block of code.
Comment by megaman821 8 hours ago
Comment by Izmaki 1 hour ago
My first lead developer was an old school developer who preferred to look up guidelines and solutions in physical books he had on his desk. Nobody should waste time reinventing the wheel.
To answer your first question, it’s faster for me because I can formulate my request as a developer as if my AI agent was also a developer (in a sense it is) faster than I can figure out how to implement it on my own, especially in a new language and ecosystem. I didn’t just prompt Claude Fable at Max thinking capacity “yo dawg make me a game for iPhone that will make me rich just pick whatever style and stuff” and let it run for a week or two. I brainstormed the idea of the game with my AI agent (Opus 5.5 mostly at High), asked for suggestions, adjusted its proposal to my liking, asked it to flesh out an implementation plan for each milestone of the development journey, worked on different topics as we progressed (introducing complex graphics was sometimes challenging which required a bit of guidance for it to get the assets right), had it create a level editor for me because explaining every single thing was taking too long when I could just drag the sprite a little on my screen and tell it to compile my changes, we worked on ambience audio and sound effects at multiple levels (background, middle, foreground), and the list goes on.
A LOT of work has gone into this project, but without an AI I would not have gotten any further in the time spent than barely figuring out how to get even a simple round ball bouncing on a flat surface in Unity. With AI I’m looking at a feature complete game ready to ship to the App Store after the last bit details have been settled (advertisement laws and whether to monetise one way or the other or not at all, mostly).
AI is an incredibly powerful technology and when you wield it correctly you can go from 1x or 10x engineering to 100x or even 1000x at minimal risk (because you mitigated it along the way).
Comment by contravariant 1 hour ago
But when you do care, it becomes important.
And, I suppose, computer code in general is terrible about writing things at the right level of abstraction, or at least human generated code is and I think LLMs do worse, somehow. So there is something to be said for prose as a specification language in most cases.
But I still think we're going to end up with some kind of code to reason about programs. Same way mathematicians have been using mathematical notation to reason about maths. But maybe it will be different code. Who knows, literary programming might catch on.
Comment by jimnotgym 12 hours ago
Comment by Gareth321 11 hours ago
Comment by tremon 11 hours ago
Comment by Izmaki 11 hours ago
I, personally, never downvote people I disagree with. Who am I if I don't use another perspective as an opportunity to learn and grow? I can tell that many people agree, too, because my "comment score" is still mostly neutral at the time of writing.
Comment by vyper91 5 hours ago
But software powers so much of our modern world, seeing companies decide "well maybe we don't understand the code but trust the governance and harness will allow AI to always fix it" is essentially that above hypothesis in action today.
Comment by bemtevi 13 hours ago
Comment by mminer237 11 hours ago
Comment by Izmaki 12 hours ago
Comment by bemtevi 12 hours ago
Comment by Izmaki 12 hours ago
Of course we should proceed with even more powerful tools. We also need even stronger governance of said tools. We don't want an AI-version of the Chernobyl incident but we do want the AI-version of the benefits that nuclear power gives.
Comment by lirolero 12 hours ago
Comment by hashmap 10 hours ago
Comment by gib444 11 hours ago
It's like an over-eager junior engineer that will implement whatever pattern they read about last week. It just doesn't feel trained on high-quality enterprise, battle-hardened codebases.
Even Opus 5.5 on High thinking still littered a root directory of a JS project with source files without thinking to put it in a 'src' directory. It had to be told to use a testing framework. Had to be told to also test with Firefox (the project already had established Chrome /and/ Firefox targets).
I have to continuously burn tokens and effort correcting it.
I imagine it performs a bit better with a huge corpus of architecture and style guides. But I did expect to get some that for free with the "leading" model.
Comment by zaphar 6 hours ago
Comment by spathi_fwiffo 5 hours ago
Same with that situation. One shot code from prompt is still not great. Maybe if you had a prompt with highly detailed requirements for structure and code formatting it would be ok.
I generally hate the look of AI gen code, so I have some pretty detailed code formatting rules for even small edits. And the AIs still ignore them some of the time.
I don't know what I'm getting at here; other than. It is acting pretty human like at this point in the code delivery phase. Human as in asking someone to do your work for you and you not always getting back what you want; as opposed to being good enough to replace your actual work (assuming you are good at the task you are asking from the Agent -- does work pretty fast though. which makes it a trade off worth considering; but then your time is in picking through the code to confirm it is correct).
Comment by kentm 7 hours ago
Comment by iLoveOncall 5 hours ago
For every bug or security issue it fixes, it creates 10 more.
Comment by stevenhuang 5 hours ago
Comment by iLoveOncall 4 hours ago
You're not using LLMs at all or not auditing anything if you don't see them making major mistakes on basic things every day multiple times a day.
Comment by simias 13 hours ago
Everybody in the industry knows that we've been building on quicksand. Moore's law meant that we could just sacrifice performance and add a dozen layer on top of technology from the 90s (if not older) and keep the thing chugging along.
Look at the PHP situation:
- HTTP is an old and quirky protocol with a lot of surface open for attackers that are rarely useful nowadays and might be better left off to higher layers. But of course, we can't break compat, so everybody still implements all the naughty parts, just in case.
- PHP was a joke language from the start, we were making fun of it in the early 2000s, the fact that over 20 years later it still powers a huge chunk of the web is enough to demonstrate that we're not serious people. Cue the PHP fans telling me that "it's good now". Maybe, I don't really care, the effort that went into making it good should have gone into ditching PHP for a better language 20 years ago. The bandaid mentality is what got us in this situation.
- And of course to add insult to injury, PHP, this security-critical language, is still implemented in garbo C using things like setjmp/longjmp. Just to make sure to absolutely maximize the damage if a vuln is discovered, I suppose. Brilliant.
- Oh and all of this is still running overwhelmingly on top of x86 which is its own bag of legacy garbage. "No but you don't get it, modern CPUs are really more like RISC CPUs, the RISC ISA is translated internally" yeah exactly that's my point let's stop doing things like that.
We're a clown industry. We only made it this far because, usually, buggy code doesn't kill people and we can iterate and patch things quickly. We never had to do things correctly, we could flourish with abysmal levels of quality acceptance. This may not be true anymore.
Of course the reason things are done this way is generally to maintain backward compatibility and make transition easier. At least now that we have LLMs this excuse will hopefully vanish: porting even huge codebases to a new language/framework, something that was a tremendous undertaking mere months ago, is now vastly easier and faster. We can leave the 1990s behind us.
Comment by yodsanklai 11 hours ago
I think you have things backwards. Devs aren't clowns who make it this far. But in industries where buggy code doesn't kill people, devs are expected to ship fast and iterate, because they can and more importantly, it serves the business.
Personally, I'm all about good software engineering principles, unfortunately I don't think you can last long in most companies if you're too opinionated about this and can't compromise (and it's getting even worse with AI, reading the code is no longer a requirement, go explain your teammate that there's an abstraction leak in their code...).
Comment by saltwatercowboy 11 hours ago
But explaining that openly, as is your duty, means surfacing something which could be read as 'bad work' by a non-technical manager unable to grasp the complexity. So in low-trust teams it very quickly becomes a game of brinkmanship, or a prisoner's dilemma for technical diligence
Comment by leoqa 10 hours ago
My manager can’t understand the RFC, and knows that clean up projects aren’t palatable to his senior eng leadership so he keeps asking me to simplify or derive some business justification. I’ve proposed superficial ones like improving security posture etc but he wants a hard number that can be measured (yet another senior leadership mandate).
I’m kind of exasperated and honestly think it’s his job to figure this out.
Comment by JTbane 10 hours ago
Comment by asdff 16 minutes ago
Comment by pyrolistical 9 hours ago
Comment by chasd00 5 hours ago
Comment by HPsquared 9 hours ago
Comment by dumbfounder 9 hours ago
Comment by florbo 8 hours ago
Comment by sensanaty 9 hours ago
I now work in a highly regulated field (payments/banking) and it's a bit better, but from prior experience if there's no regulator to beat people with, they will simply not give a shit, the only thing they want is to shove out whatever feature as fast as humanly possible.
As an example GDPR was a godsend because I could finally convince people to get rid of the 11 trillion stupid analytics and tracking bullshit we had "Because GDPR", whereas no number of arguments, no matter how well researched and presented, would ever convince anyone before
Comment by ericyd 8 hours ago
Comment by dionian 10 hours ago
Comment by preg_match 9 hours ago
Business people should understand, at a high technical level, how their products work and what they’re actually asking to implement.
I’ve worked with many great business analysts and product people and consistently the best ones are the ones with the “engineer brain”. They might not know how to read even a single line of code, but they understand software products from their pieces. Those people consistently propose high-quality and feasible solutions. Other business people do not.
As developers, we are expected to constantly think on our feet and adapt. We learn new products and methods on practically a weekly basis. But for whatever reason, the business folk are not even expected to understand the product they help build. There’s such a large disconnect there between dev and business.
Comment by IAmBroom 9 hours ago
If higher-ups have no technical insight, round up and give them two numbers: $$ if a bad thing happens, and $$$$$ if the worst thing happens.
Comment by skilning 9 hours ago
The director of my department kept asking, "But do we HAVE to do this now?", so my final answer was this:
"Do we _have_ to? No, but it's like a bridge. Bridges have an expected lifespan when they're built, and they're supposed to be torn down and rebuilt after that. It's not a HARD limit, though. The bridge doesn't immediately collapse the day after its lifespan, but there are fault accumulating. Metal fatigue, erosion of support material, and other things. Commuters can't tell the bridge is past its lifespan, and they'll continue crossing it every day without issue. And then one day the issues will accumulate to the point that the bridge fails and people DIE. That's the situation we're in with this stack. It will continue to function just fine until the day it doesn't, and on that day I won't be able to tell you how quickly we'll be able to recover the application stack, if at all."
That FINALLY got through to him.
Comment by skydhash 8 hours ago
If someone says that something is a risk and we should address it, I don't expect a director to ask for a specific date the risk to become a disaster. Instead, he should just add it to some todo list and slot some time to analyze it and and budget resources. AKA, doing some "management" work. You shouldn't have to explain his job. You didn't need to learn semiconductors physical properties to know that storage can fail and you need to do backup.
Comment by robotresearcher 9 hours ago
Comment by Sharlin 9 hours ago
Comment by meindnoch 9 hours ago
Comment by Woodi 9 hours ago
Comment by Loquebantur 10 hours ago
The logical extrapolation of AI capabilities stress-testing human software is all bugs being discoverable. Hence, you have to aim for actually bug-free software.
The only way to do that is to make your code provable. That again is only realistically doable using AI.
In the process, people will have to turn the current mere pretense of software engineering into reality, making the engineering part explicit.
That in turn only works when humans actually understand what they're doing and can communicate it to outsiders.
Comment by naasking 4 hours ago
Comment by acedTrex 9 hours ago
Comment by kalcode 11 hours ago
But shipping faster was pushed at almost every company until, if it's not revenue generating, stop putting resources into it. Not all are like this but it's not all or nothing, it's always a sliding scale how much you can put into making the software correct.
Comment by glimshe 11 hours ago
There are things you buy out there that don't need Swiss quality because most people will simply not need the quality or won't have the money to pay for it.
I think the biggest problem isn't that a random webpage is sloppy, but when a critical system that could kill people is built the same way as the random webpage.
Comment by everforward 10 hours ago
Near the point where things become minorly safety critical, people will blur the line. The point where safety kicks in moves a little further out, and we start blurring the lines again.
There’s a reason the FAA is anal-retentive about things with only very minor safety implications, like the in-flight entertainment system or turning on airplane mode. It helps prevent complacency where it actually really does matter.
You either do it the right way 100% of the time, or I’m not taking bets that you’ll do it right on a particular occasion.
> eventually I realized that if I didn't get things out of the door, my company would start losing money and I'd lose my job.
This is like 90% because most tech is different enough that civil torts don’t exist for them yet, so they don’t have to care.
Eg if Google deletes all your photos they kind of just shrug and you’re out of luck. If I deposited photos at a bank who burned them without notice or approval, they’re on the hook for that.
The law will eventually catch up and acknowledge that bits can have value. Then everyone will have to care again.
Comment by IAmBroom 9 hours ago
We track EVERY requirement to ensure it is met. Maybe it's a safety issue; maybe it's landscaping approvals (I work in Rail). Regardless, it's bigger $ if not caught up front, and if you start grading requirements on perceived risk, you start the slippery slope to not caring. Example: mark all landscaping requirements as "Not Critical". Then scheduled clearing of the right-of-way is missed. Then pedestrians are at risk with blocked line-of-sight, and trees might drop across the rail.
Comment by c-linkage 10 hours ago
Comment by kalcode 10 hours ago
Comment by pronghorn 9 hours ago
Comment by vbezhenar 9 hours ago
What that PE certification will check? That guy knows how to write unit tests? Well, that's not much. Who will write certification? Where's that genius, why he's hiding from the rest of us?
Maybe building is figured out. I don't know. I still see bridges failing and buildings crushed, so apparently it's not perfect. But I do know that programming is not figured out. Absolutely not. We can apply tremendous effort, pay billions and produce some simple firmware which can be uploaded to a martian rover. But that doesn't scale to the rest of industry.
Comment by kalcode 9 hours ago
Then when company forces a product to skip safety checks, test units. And the PE proves the did their but the company pushed forward, the company gets wide open to fines and lawsuits.
Make a company responsible and hit their bottom line, other will think twice before they say "just ship it, or I'll hire someone cheaper to do it". And no they don't say that, that's how it's done.
When your paycheck or livelihood is on the line, you tend to slowly sacrifice the principles.
I think your being short sighted here. And a aggressive for no reason.
Comment by jimbokun 9 hours ago
The cost of "losing customer data" is too abstract for a company trying to survive financially. Corners will inevitably be cut.
Unless the costs to be suffered by the company for exposing private or sensitive information are made very clear through legislation.
Comment by bongofizzbuzz 11 hours ago
Comment by swatcoder 11 hours ago
Look at the solo projects from professional devs with nobody overseeing them. Look at what so many of your colleagues submit despite the frustration of your thoughtful team lead and manager. Look at what your so many of your team leads asks for even when they've been granted full discretion. Some select few are great, but most are disturbing if you have any sense of quality or preparedness yourself.
Even for decades before AI, there was lazy naive misguided slop everywhere. During the social coding era of the last 15 years this even amplified itself, streamers and influencers and conference presenters -- professional developers themselves -- would share horrid practices to wide applause. Moving fast and breaking things was what developers were excited about. Rigor was dead. Correctness was a problem for later. Code was presumed best written by a rando on github who'd accumulated a lot of stars rather than by a known responsible hand. Security was a copy and paste job from Stack Overflow, not something to be comprehended and considered.
If only it were a top down problem from outsiders!
But the truth is that the call has been coming from inside the house all along, and the outsiders giving direction have had no problem finding celebrated industry "experts" and "thought leaders" affirming and encouraging all the clowning they could imagine.
Developers could always move fast and do things right but most just didn't have any interest in learning how, and many of those who might have just gave up on bothering.
Comment by yodsanklai 10 hours ago
Comment by brabel 8 hours ago
In fact, I have myself spent years on solo projects with the goal of making perfect software. You know, nothing like getting a few users to quickly destroy that idea from your naïve brain. You can’t write perfect code because that is nearly impossible for most of us, no matter how hard and how long you try. Think of it like this: can you beat the greats in your favorite sport or hobby? Probably not, you may be good at a small team level, but at a large city level you are almost certainly mediocre. This goes for your coding skills too. How many great developers out there creating great software we all know and use and love? I know a couple of names, that’s it. Most of us are writing shit enterprise applications everyone who is forced to use hates and that has more bugs than PHP in the early days. And when we try our own thing it’s even more horrible. Show me your repos and let me se if you’re the genius you think you are. Oh of course, you don’t have time to spend on hobby projects since you have a life… great, enjoy that, but don’t pretend you’re able to paint the Mona Lisa just because you spend all day with a brush in your hand striking on a canvas that no one can really see and if they did, it would most likely be a repulsive, incomplete, bug hidden piece of shit.
Comment by xbar 11 hours ago
2026 is saying "here is the consequence."
Comment by chrispyfried 11 hours ago
Comment by dboreham 10 hours ago
Comment by jimbokun 9 hours ago
Explain those super human hacking AIs like the ones that broke containment and hacked outside sites are coming for your infrastructure too.
All the business people have heard about the Huggingface incident by now (or are living under a rock).
Comment by jjkaczor 11 hours ago
Just take a look at some of the opensource academic, infrastructure or "passion-projects", where there is no management pressures to "move fast & break-things" and ship-ship-ship. They tend to be very stable, very predictable - and very high quality.
Comment by SoftTalker 11 hours ago
I've seen software that academics write. It's no better than average. It's often cut-and-paste code, or these days AI slop. Software is a means to an end in academia, rarely a product in itself.
Comment by dotancohen 10 hours ago
Comment by baldursdev 10 hours ago
Comment by Ericson2314 10 hours ago
That might be true on the margin, but highly doubt it was ever true once you escape a very shitty local optimum.
Comment by vlovich123 10 hours ago
https://ijcionline.com/paper/13/13224ijci12.pdf
Wow, it’s almost like it’s really hard to get everything perfectly right and this is endemic to software, particularly when the language accepts incorrect code.
The only two things we know mitigate this are proofs (eg rust borrow checker, lean, etc) and DST which is basically the equivalent of accelerated time testing which is done in other industries too. Most people don’t even bother with property testing
Comment by tylermk04 11 hours ago
simias referred to the industry and you took it personally.
Why is it whenever someone mentions the shortcomings of software industry, there is always a comment saying "no no no, developers were just following orders!"
Comment by dymk 11 hours ago
Sounds like they’re talking about devs
Comment by lilbigdoot 8 hours ago
I don't disagree with the point that a lot of software that isn't pressured to be shipped quickly still ends up following bad practices, but I don't think we can ignore that the quality issues in software aren't also found all over in many industries because of profit motives?
Comment by jasondigitized 10 hours ago
Comment by _ph_ 11 hours ago
But also every developer has a responsibility. At least not to promote any technology which is part of the problem.
Comment by oblio 11 hours ago
Yeah, but money.
Comment by bonoboTP 12 hours ago
Trying to understand underlying reasons and incentive structures is much more productive and useful than simply venting about the worse moral character and seriousness of people in one industry vs the next. Curiously, the grass seems always greener - when you ask people in other industries, you'll often hear endless complaints about the state of things there too.
Comment by Eji1700 8 hours ago
This isn't unique to coding, but coding has been built on it for decades, with the "safety valve" of "who has the time to check all that?". So defcon speakers and intelligence agencies burn time investigating for weak links that might take months to find, and the rest of the world churns on.
Now you have companies throwing thousands or millions in compute at problems just checking EVERY possible problem and yeeeeup, looks like there's a lot of "good enough, ship it" which is now a major problem for all those companies using that tech.
It's still going to be interesting to see how this develops because, yes, there's a lot of code being written by people who haven't even read it (although the amount of enterprise production level code I've seen that's bad stack overflow copy paste makes me less worried about this), and there's maybe better tools and standards to catch these things (why not spin up a test suite if it's an afternoon of AI poking rather hiring a dev and getting them up to speed?).
Comment by anthk 10 hours ago
Comment by TedDoesntTalk 10 hours ago
I’ll keep checking
Comment by dotancohen 10 hours ago
Comment by TedDoesntTalk 9 hours ago
Comment by dotancohen 9 hours ago
I tend to write integration tests for the paths users take through the application (website), and fill in the holes with unit tests. I'll also write unit tests for methods that change often or are critical, such as business logic.
Comment by actionfromafar 9 hours ago
Comment by lifeisstillgood 11 hours ago
Comment by traceroute66 12 hours ago
First, I am not a PHP fan, but I will absolutely stick up for the "its good now".
Yes, absolutely no debate the old PHP 4.x/5.x was shocking. However to give PHP their due, they absolutely did make serious efforts in 7.x/8.x. They deserve due recognition for that.
Unfortunately lots of people – I'm guessing you're one of them – still have battle-scars from 4.x/5.x and they think (incorrectly) that 7.x/8.x is "more of the same".
Second "joke language", well, cast your mind back to what else was available at the time... ColdFusion, CGI, ASPX ... it was not exactly a beautiful world back then.
Sure, to borrow your terminology, the Java fans will be along soon to tell us all how awesome Java is for the web. But many of us still bear battle-scars of the horrors of Java stacktraces, messing around with incomprehensible XML config files, Java's penchant for eating RAM for breakfast, and generally trying to figure out why the hell Tomcat is not working (again !).
Comment by chasd00 5 hours ago
well, there's no Java2Shell at least.
Comment by traceroute66 4 hours ago
Ahem, cough....
Log4Shell (CVE-2021-44228)
Comment by chasd00 1 hour ago
Comment by mitchitized 11 hours ago
Pffft. VRML is the whole game, baby! Once everyone runs out and gets an SGI oxygen, Nobody is going to use freaking web browsers ever again. I mean, HTML, what a joke.
Comment by tinco 11 hours ago
Comment by jmull 11 hours ago
Classic ASP was the more direct competitor to PHP, while ASP.net came along later, as part of a next generation of web server languages/frameworks/platforms.
Comment by thorin 11 hours ago
The other option I was actually offered was in machine learning and categorisation, but I avoid it as it was obviously a dead end lol.
Comment by hansvm 12 hours ago
Comment by mrkeen 12 hours ago
Unless you count the XML dependency management with Maven, which isn't too bad.
Comment by drdec 12 hours ago
Comment by traceroute66 12 hours ago
I suspect the only thing that's changed is that Oracle now charge you for the pleasure. ;)
More seriously though, its been a long time since I've touched Java, but I suspect the only thing that's changed is that its all hidden in magical kubernetes containers these days.
Comment by a57721 11 hours ago
Comment by traceroute66 10 hours ago
Congratulations Java Fan on missing BOTH the smily-face at the end of the first phrase AND the "More seriously though" at the start of the second.
I'm glad someone still loves Java though. ;)
P.S. The kubernetes reference was – since that one appears to have gone over the top of your head too – that in todays world Kubernetes is extraordinarily popular and therefore it is likely most people now hide the complexities of Java deployment into a container that is magically generated and deployed through CI/CD. Hence I was inferring that life is probably "easier" today with Java because you just deploy the magical container.
Comment by a57721 9 hours ago
Comment by DocTomoe 12 hours ago
Comment by fabioborellini 12 hours ago
Although that was more connected to the performance improvements through Zend Engine than language sanity or robustness, at the time.
Comment by TedDoesntTalk 9 hours ago
You’re missing the point. Those are all dead. The implication is that PHP should have had the same fate.
Comment by actionfromafar 9 hours ago
Comment by sneak 11 hours ago
Comment by knome 9 hours ago
That's a C language error, not a PHP language error. I don't see that CVE as particularly different from when they showed python's dictionaries could be attacked using dictionary hash-collision (something that also affected PHP). It's worse in being remotely executable, rather than merely a denial of service, but it's ultimately a facet of the C implementation rather than something wrong with the language itself. CVEs, even remote ones, will happen. Even java had its logging execution attack.
I've made fun of plenty of things in PHP over time, but the language hit the mark it was aiming for initially. It was easy. A 12 year old could sit down and write an ugly little forum and it would work. Because it was easy for amateurs, a lot of amateurs wrote in it, and there's a ton of clumsy code out there. Because it was easy to setup and run with minimal setup, popular packages were very popular. Wordpress vulnerabilities matter because it was very easy for programming chop shops to hack out tons of websites for all the millions of little businesses out there that didn't often follow up on patching etc once that was done. PHP was bad largely for the same reason BASIC was bad. It was easy.
PHP being easy gives it a huge blast radius for anything that goes wrong with its implementation or popular frameworks and libraries.
Comment by robotresearcher 9 hours ago
(I say this as someone who enjoys C programming)
Comment by dboreham 10 hours ago
Comment by maccard 10 hours ago
What industry do you think is different?
I've had a decent amount of construction work done on my house, and if I had a pound for every shortcut that the guys doing the work took, it would pay for the work they did. And these are the _good_ ones. The bad ones don't even bother following the legal requirements. Somewhat recently [0] it was found that less than half of healthcare workers followed recommendations on _hand washing_.
[0] https://www.cambridge.org/core/journals/infection-control-an...
Comment by physicles 10 hours ago
Comment by maccard 7 hours ago
There are plenty of software projects that are managed to an incredibly high standard professionally - OpenBSD, PostgreSQL, Curl all come to my mind pretty much immediately. Linux is probably the closest thing to "skyscraper" development and has been going for 35 years at this point.
Comment by GuB-42 11 hours ago
If you want a web page that works with a database (aka. CRUD), presents it in HTML form, and with a bit of business logic, like 90% of websites, then PHP does it simply. None of these astronautic patterns like you see in Spring, the dependency hell that comes with node, or whatever the cool kids are using right now. There is a reason PHP became so successful despite the hate it gets.
PHP is not great for security, it was terrible when it started, but now it is decent. And sure, a vulnerability was found, it is now fixed, every time a vulnerability is found and fixed, PHP becomes better. New technology may have an advantage due to better design, safer languages, etc... but you shouldn't dismiss the qualities that 20 years of bugfixes get you.
As for x86, yep, it is legacy, but it is not completely stupid either. The "RISC core" is not right, it is not like x86 become ARM or RISC V code internally. It is translated into long microcode-style instructions that wouldn't make sense as an ISA, where a compact representation is important for memory/cache reasons. RISC code is also translated just like x86 code, and the advantage of CISC vs RISC has gone back and forth. Now it seems we are going back towards RISC again, but before that, it was CISC, and before that, RISC again. The quirk about x86 that complicates things seems to be the variable length instructions. But the real problem is not really about which ISA is the best (it is a very small part of modern CPUs), it is more about licensing and backwards compatibility.
Comment by kevstev 10 hours ago
In the end, PHP wasn't great, but around 2000 it was a great intro to building things on the web.
Comment by icedchai 6 hours ago
Comment by icedchai 10 hours ago
Comment by chasd00 5 hours ago
Verbosity of a language/framework is meaningless now. Pick a language or platform that makes it as hard as possible to write insecure code, no matter the tedium, and let a coding agent bang away on the keyboard for you.
Comment by gbuk2013 10 hours ago
Comment by jimbokun 9 hours ago
PHP is hard to beat for development velocity. Edit a file, refresh a page in your browser, see the results.
Comment by evilduck 8 hours ago
Comment by win311fwg 7 hours ago
Now that people are back to writing their applications in languages that aren't as slow as Ruby they've, in many cases, taken the server back out of their application and have returned to the CGI model again; what is often marketed as "serverless".
Comment by dennis_jeeves2 12 hours ago
Another industry does even better: the medical/health industry. It's both a clown industry AND kills people AND they have made it way beyond what software can only dream.
Comment by splatter9859 10 hours ago
The big fun factor is the place in the Venn diagram where they both intersect. I used to work for a medical device manufacturer who's biggest and most popular product routinely had stack traces at inopportune times.
Comment by andai 12 hours ago
(I'm not kidding. If you take each element of their designs and invert it, you end up with a very solid design.)
On that note, I misspoke somewhat: I assert that PHP and JavaScript were not "badly designed", because that would imply that someone sat down, carefully considered how the different parts of the system interact, second order consequences etc. — and then somehow came to all the incorrect decisions. Rather, it's quite clear a serious design process has simply not occurred to begin with.
Once I recognized this phenomenon, I began to see it in all areas of life. Things are much worse than they could be, simply because nobody has designed them to be good. They just ended up the way they are now, by accident of history.
--
Polemic aside, I am fond of JS and PHP in many respects. I just think we should clean them up! "This mess has always been here, so it's too late to clean it now" would be laughed at in any other context.
We already broke compat with "use strict". Now we just need a "use sane" :)
Comment by ordu 8 hours ago
I don't think it holds with
> If you take each element of their designs and invert it, you end up with a very solid design.
The rule of thumb is "the inverse of stupidity is not intelligence". So, if we have an example where you can get a good design by inverting the bad one, we can't then state that the bad one was a result of a chance. The probability is very low.
Comment by jvanderbot 11 hours ago
Comment by BoppreH 11 hours ago
- Variable naming conventions (_field in Python).
- Lexical scopes.
- Visibility annotations (public, private, protected).
- Coroutines, threads.
- Packages/modules.
- Sandboxes.
- Processes.
- OS capabilities (pledge/unveil, SELinux, etc).
- Local user accounts.
- File permissions.
- Containers/chroot.
- User mode vs kernel (Ring 0 vs ring 3).
- Virtual machines.
- Cryptographic measures (secureboot, attestation, memory encryption, etc).
On the other hand we have few mechanisms to share data: the filesystem, clipboard, sockets, maybe dbus. Shared memory if you're feeling brave.
If a script wants to know what file is currently open on my editor or what email attachments I recently uploaded, the editor or email client has to explicitly implement this feature via one of those limited mechanisms. Almost nobody does this, so we're stuck with black box applications that are hostile to customization.
Comment by cjfd 10 hours ago
Comment by BoppreH 10 hours ago
The data is already in memory and the source program knows its "shape"; there should be an easy mechanism to provide access to that by default, like /proc or gdb but for arbitrary data structures and with better UI.
Comment by cjfd 9 hours ago
Comment by seethishat 11 hours ago
So that leaves non-technical managers telling non-technical nerds to just install WordPress. Then, AI comes along and wipes them out. And everyone is like... how did this happen?
It's pretty obvious.
Comment by SoftTalker 10 hours ago
Comment by jimbokun 9 hours ago
This is not completely a bad thing. Using skyscraper techniques for dog houses is not sustainable.
The key is to identify when you are working on a skyscraper and when you're working on a dog house and act accordingly. To my mind, the boundary between the two is whether or not you are handling sensitive personally identifiable or financial data.
Comment by dennis_jeeves2 10 hours ago
>So that leaves non-technical managers telling non-technical nerds to just install WordPress.
Good summary of how the world operates generally everywhere. It's a clown world.
Comment by NostraDavid 13 hours ago
I'll generally agree, and I include myself in this industry, but we've simply have been bound by compatibility. I don't think anyone chose intentionally to keep the crap around; it was simply never worth it to pay the "break compatibility" cost, until now.
Comment by IX-103 12 hours ago
The interesting thing is that memory bandwidth has become a big issue. CISC instructions take up less space than the equivalent RISC instructions, so you can effectively get more instructions per cache line. Effectively the CISC to RISC conversion acts as a type of decompression. Because if that, stopping the CISC messiness without moving to a compressed instruction stream would actually hurt performance.
Comment by renox 11 hours ago
In theory CISC instructions take less space than the equivalent RISC instructions, but x86 has lots of legacy features which makes it far less efficient than a CISC could be.
Comment by lambdaone 12 hours ago
Absolutely. The industry could have moved towards safer languages, better coding methodology and formal verification years ago (see for example MISRA C as at least a partial movement in this direction). It didn't, because it was cheaper not to.
Comment by duskdozer 11 hours ago
This goes double when you add in the same non-technical-sourced pressures that caused developers to take shortcuts are still there, and now even worse with their unrealistic expectations for LLMs.
Comment by Ajedi32 10 hours ago
LLMs might make this somewhat easier, but you're still going to have to deal with backwards compatibility and legacy systems. Completely eliminating the use of architectural cornerstones like x86 or HTTP or even PHP across the entire industry isn't going to happen anytime soon, let alone overnight, especially if the only argument for doing so is "we could design this better if we implemented it from scratch".
Comment by giancarlostoro 11 hours ago
There's only one area I sorta give a pass, and that's mainframes. It takes an insane amount of physical servers to replace a mainframe. Why IBM didn't invest more into making mainframes a drastically better platform for devs is beyond me. I don't know that any other server platform has ever achieved the level of uptime a mainframe has.
Still, it is no excuse to leave them with ultra legacy code that remains unmaintained for decades.
Comment by KellyCriterion 10 hours ago
Comment by giancarlostoro 4 hours ago
Comment by brodouevencode 11 hours ago
Exactly. There was even a whole domain of expertise (SRE) that exploded because of the lack of engineering rigor. That's not to say if everyone did all the right things all the time there would be NO need for SREs, but there would be fewer of us - something that I've tried to communicate to everyone I've worked for in the past ~14 years of being one.
But we had to deliver. We had to increase shareholder value.
Comment by bigbuppo 11 hours ago
Comment by wkjagt 11 hours ago
I am no fan of PHP but it wasn't really a joke language. Its goal I think was to add some dynamic snippets to your homepage for which I guess it served its purpose pretty good. PHP even originally stood for Personal Home Page iirc. The problem seems to be that the rest of the language was built on top of that. I haven't used PHP in over a decade, so I don't know if it's still the case, but you used to start a PHP file with a PHP opening tag, which is such a weird "feature", but I guess kinda makes sense in a world where everything outside of PHP tags used to be HTML.I remember seeing PHP files that were impossible to work with: thousands of lines with HTML mixed in with PHP classes etc.
Comment by vonunov 9 hours ago
This was the tradeoff (or should I say trade-up) for going from
#!/usr/bin/perl -w
use strict;
my $username = <>;
chomp($username);
# blah blah blah blah blah
print "<html>\n<head>\n[...]<body>\n";
print "<p>Hello, " . $username . "!</p>";
print "</body>\n</html>";
to <html>
<head>[...]
<body>
<p>Hello, <?php echo $username; ?>!</p>
</body>
</html>
(Of course, as you point out, we promptly recreated the former situation in PHP -- but maybe there was some rationale to it? I wouldn't know; I did my best to avoid that kind of overwrought PHP project)Comment by 091273153 3 hours ago
Comment by MisterTea 5 hours ago
Who's we?
The issue is simple: YOU work for clowns who prioritize financial success above all else, including quality of output. You usually don't have the ability to say "No" to a bad decision made by a senior person. Deadlines, managers dumping more work, feature creep, etc. Then, when you can't deliver they taunt you for their incompetence. So you eat shit all day long churning out crap. That or management hires the bottom of the barrel because its cheap and maybe one or two seniors who eat turd sandwiches all day, give up and ship shit. Enjoy workplace hell. Fire your incompetent boss.
Comment by dsign 12 hours ago
> HTTP is an old and quirky protocol with a lot of surface open for attackers that are rarely useful nowadays
HTTP 1.x is that, perhaps. But I was in the trenches when HTTP/2 came out, brand new and running on binary. It was so new that there were very few implementations around, and my team had to code one. The people who worked on the spec had a lot of learning to apply from HTTP 1.1. That resulted in many nice bits in HTTP/2, but HTTP/2 also had serious issues at a spec level. Which is my way of saying that "modern" doesn't necessarily mean "bug free".
Comment by samiv 11 hours ago
Comment by jstummbillig 11 hours ago
Not sure why you would. It seems like the people who made these compromises were roughly right and the problem only escalated when it also came with a solution.
Comment by rkozik1989 12 hours ago
Comment by bigbuppo 11 hours ago
Comment by StilesCrisis 9 hours ago
Comment by Ra8 9 hours ago
The risk is of course, an earthquake where it could destroy all houses built before year X.
But we do the same in all industries, path of least resistance.
Comment by hansvm 12 hours ago
It's especially fun when not implementing the naughty bits is its own source of vulns. Your code is simpler and less prone to RCE, but your proxy mishandling or whatever allows for credential smuggling attacks, identity spoofing, request smuggling, cache poisoning, and all the normal issues at the application layer.
Comment by nuancebydefault 12 hours ago
Comment by _ph_ 12 hours ago
Comment by FranOntanaya 9 hours ago
Comment by thesuitonym 11 hours ago
Don't forget the other half of the web development stack: Javascript. The absolute worst scripting language ever desi--well no, it wasn't designed, it just kind of happened accidentally. At least PHP solved a problem of the day, Javascript was nonsense from the start, and somehow managed to become critical infrastructure.
Comment by crabmusket 10 hours ago
While yes, PHP begun as a collection of macros and was never intended to be a programming language, JS was made by someone with intention. To quote from wiki,
> [Eich] originally joined intending to put Scheme "in the browser", but his Netscape managers insisted that the language's syntax resemble that of Java. As a result, Eich devised a language that had much of the functionality of Scheme, the object-orientation of Self, and the syntax of Java.
I've seen quotes before from Eich himself I believe that he was combining these influences at the time - Scheme's closures, Self's prototypes. It didn't "just happen".
Comment by newsoftheday 9 hours ago
Comment by frevib 11 hours ago
Still, I pray you’re right.
Comment by kevinsync 9 hours ago
I bring this up because it's the first thing that popped into my head thinking about PHP's legacy. And, admittedly maybe it's not the best analogy, but when people shit on the caissons that bridged the internet because it seems crazy in hindsight to continue use those techniques, shit dreamed up by wacky people to do the impossible, or maybe said we should tear it down and build it a different way, the reality is that sometimes you just leave the bridge in place and improve where you can. It's still standing, you know?
I might be stretching here with all this, but despite you making some very defensible points, I just don't buy into the whole narrative. Just because retrospection has blessed us with the ability to intellectually point out that we built things on quicksand doesn't mean that there were real alternatives to build on at the time. The ease and speed that things like PHP provide(d) allowed enough things to be created to make the web as we know it coalesce into existence in the first place. Pedantic or more expensive or arguably safer choices easily could have slowed progress to a crawl, OR you just build the bridge and see if it's still standing 143 years later.
And in some cases, it is.
[0] https://www.pbs.org/kenburns/brooklyn-bridge
Comment by jlos 10 hours ago
There quality should be there, but it should only ever be “good enough”
Comment by Schlagbohrer 11 hours ago
Comment by pyrale 11 hours ago
Software devs absolutely love to create a mess, and are never challenged by business, product, management, or you name it, to increase speed to market, cut down cost estimates and, to say it crudely, deliver half-assed work.
Systemic issues have systemic consequences.
Comment by pjmlp 9 hours ago
The Morris worm was already a sign, but alas, as long as there is no punishment, no one cares, like buying at those 1 € shops.
Comment by Ternari 5 hours ago
Comment by mcv 11 hours ago
I've certainly taken time to address tech debt whenever I could. Which was not as often as I'd like.
Comment by edoceo 9 hours ago
Comment by txdv 10 hours ago
Comment by edoceo 10 hours ago
Comment by BigTTYGothGF 5 hours ago
Comment by joshblack1982 11 hours ago
1. It just werked. You could write incredibly shitty code and it, like HTML, would work as long as there wasn't a fatal error. And sure you could just say "git gud", but that's not going to happen. People will just use the best language that works for them, and in this case, it was PHP.
2. It required very few resources to run low to medium traffic websites. Because of the Apache mod_php module, PHP sites wouldn't use any resources if they weren't being served, unlike every other language at the time. So it meant places could host thousands of websites on the same server no proble.
The guy who created PHP, Rasmus Leerdorf, never wanted and never expected that PHP would become as popular as it has. He has mentioned that he thought that at any moment, someone was going to come around and create a language that would succeed PHP. But it never happened. People are just better at pointing and complaining instead of solving the issues that people want solved.
Comment by snarfy 10 hours ago
Comment by sevenzero 13 hours ago
It's totally the devs fault and not the environments devs have to work in. Deadlines forced down on you until you have to make compromises, 24/7 having to compete with people who actually don't care because you have to meet months end payments somehow. Whole businesses blinded by a tech gold rush.
>We're a clown industry.
I do agree here, but that includes every non dev person.
Comment by jimnotgym 12 hours ago
You don't see the same problems in say, structural engineering as you do in software engineering. I suggest this is due to needing to go through the formal training required of a structural engineer, and membership bodies like chartered institutions, with standards and codes of ethics. We might want to reflect on that
Comment by jandrewrogers 10 hours ago
No amount of bureaucracy and process theater gets around this fact. There is a vast and very expensive gap in software between "don't do stupid things" and "this is actually correct". The tooling to close this gap is severely deficient and not for lack of trying. In physical engineering we've had automated design checking for decades because it is a relatively trivial problem in those domains.
It would be great if compilers could generate proofs but after decades of research they can't even generate SIMD code in most contexts.
Comment by IX-103 12 hours ago
Comment by DrBazza 11 hours ago
I rarely call myself an 'engineer', preferring developer or programmer.
Comment by _ph_ 11 hours ago
In Europe, a lot of supermarkets were hit by a remote exploit which allowed attackers to switch off cooling of food. A blood bank in France was attacked the same way.
So how would building software for important or even safety-relvant systems not be considered "engineering"?
Especially, as faulty software might not only cause a product or device to fail in a dangerous manner, faulty software in this day and age often makes it possibly to remotely attack the faulty product.
Comment by drdec 12 hours ago
I think that has more to do with than some unverifiable sentiment like "software people are simply inherently less serious than other engineers".
Comment by pletnes 11 hours ago
Comment by mmcgaha 11 hours ago
Comment by jimnotgym 6 hours ago
Comment by ihavesomeexampl 11 hours ago
Comment by bonoboTP 12 hours ago
Comment by RugnirViking 12 hours ago
Comment by hybrid_study 12 hours ago
Comment by farhanhubble 11 hours ago
Comment by jasondigitized 10 hours ago
Comment by eastbound 12 hours ago
Comment by bigbuppo 11 hours ago
Comment by Woodi 9 hours ago
So "downloading" part is just looking similiar.
Comment by dsego 11 hours ago
Comment by Kaliboy 11 hours ago
I basically just worked extra unpaid hours with a colleague to do it since it was frustrating. Not even from a security standpoint as it is now, but just plain developer happiness.
But while we were cleaning the Swamps of Dagobah the other team was extending them.
Our deliverance came when the other team got woo-ed by React and started coding less in Ruby.
For my mental sanity I treat the React project as a black box.
Just heard they had to update the Redis connection configuration in 6 places, after we had switched to Sentinel based connections.
I seriously can't with my people sometimes.
Comment by jmull 11 hours ago
Things have moved on from PHP, but at the time, the alternatives had many more tradeoffs in a wide range of cases. Without PHP the internet would have unfolded much differently, much more slowly.
The "bandaid" mentality you decry exists of course, and is outright bad in some cases, but is just good business in many others.
Rewriting everything makes no sense. And, of course, in the vast majority of cases is never going to happen, so advocating for that is effectively advocating doing nothing at all for 90%+ of sites. A much more efficient and effective approach to using AI here would be to find and fix issues in the PHP runtime, and to assist in automating the deployment of the runtime. Time spent on this fixes thousands and millions of sites at a time, in a consistent, improvable way, rather than one at a time, in a piecemeal way.
> We're a clown industry
I think if we want to bring more engineering discipline to software development, we need sober analysis, not outraged rants.
Comment by raffael_de 10 hours ago
Comment by misterderpie 10 hours ago
Ironically, the opposite is the case with LLMs. Now those on your teams that wanted to stop the clownery will be overruled by the vast amount of bullshit code from their less skilled peers. Of course, those that care can use LLMs to faster clean up, but there's no way they are able to keep up.
Managers always favored outcome over correctness, and not giving two shits about the internals was, and will always be, incentivized.
Comment by knorker 10 hours ago
The CISC vs RISC debate hasn't been interesting in a quarter century at least. The only real difference is that the instruction decoding (which CISC and RISC both have) is likely going to need more transistors on x86 than other contemporaries.
But I'm not sure what you mean by "stop doing things like that". The ISA is "the API". Sometimes the best CPU implementation (for whatever definition of "best") will need to change the ISA-to-microcode translation.
It's not reasonable to assume that a decades old ISA will have a 1:1 instruction to microcode mapping. Hell, it's not even 1:N. Macro-op fusion makes the mapping N:M. So you could say that RISC also has variable length instructions, in a way.
And RISC-V has two sizes of instructions, with RVC. So it's not even "super transistor efficient because all instructions are 32bit".
Then there's the "reverse micro-op" that is instructions intended to cause traps for software to handle. E.g. Alpha's PALcode that handled (among other things) unaligned memory accesses.
Comment by throw938990 12 hours ago
What effort? PHP foundation has one employee!
> The bandaid mentality is what got us in this situation.
> We're a clown industry.
We are clown industry becauee nobody would sponsor solid research, infrastructure and fixing all those bugs! PHP is junk literslly written over nights by some volunteers!
Comment by jon-wood 11 hours ago
Comment by bigfishrunning 11 hours ago
And yet everyone knows this, and millions of people still continue to use it and develop new software with it, to avoid learning from our mistakes. The software industry is just not interested in fixing these problems; I've heard people upset that AI is pointing out flaws, instead of being upset that flaws exist. Deeply unserious.
Comment by fennecfoxy 11 hours ago
What you wrote reads a bit like ye olde typical "nyeeeh javascript! you're only a dev if you write everything in assembly! kids these days!"
Comment by ihavesomeexampl 11 hours ago
My guess is aviation or .. defense?
We don’t have billions to throw away on making sure our about-dialog is memory safe.
Comment by kmoser 11 hours ago
Comment by dennis_jeeves2 10 hours ago
The real clowns union paradoxically is the least clownish industry one can find.
Comment by jdw64 10 hours ago
Comment by keybored 10 hours ago
At least put some formal verification on top of that GenAI pile.
Comment by 1vuio0pswjnm7 10 hours ago
How can even call these people an "industry"
Comment by dev1ycan 10 hours ago
Comment by KellyCriterion 10 hours ago
Comment by sylware 11 hours ago
Comment by EGreg 11 hours ago
That’s what I started to do 10 months ago.
You want a safer language AND transpilers from PHP, JS, Python etc. into it? So you can find issues in your original code using source maps, or just speed up your PHP by 10x and your Python by 100x? Here you go:
Comment by BatchJob 11 hours ago
Comment by redwood 11 hours ago
Comment by yipinwong 11 hours ago
What the heck have you been doing instead of pointing out the obvious?
Comment by kakacik 11 hours ago
No we don't need to have documentation and keep updating it what you can't read code? No we don't need to rewrite system that was patched ad absurdum so codebase became fragile and its hard to navigate layers of abstractions. We need to deliver this project yesterday at all costs since somebody up there slept on requirements for 6 months and didn't do their basic job, and we will never pick up half-assed pieces we were forced to deliver. No we don't need to update those 12 years old libraries, too much work and testing. I could go on and on.
It seems like you consider engineering to be delivery of perfect things, or utter failure. Thats not how things are anywhere, sw dev is more unstructured but you also don't have 5 million ways to build a good wall using every possible substance in the universe, do you.
Comment by locknitpicker 9 hours ago
Your analysis is severely flawed and naive.
Software devs often patch and sidestep the root issues for the same reason why every single engineering field resorts to patching ans sidestepping root issues.
Why do you think there is such a thing as aviation duct tape? This is a field where people's lives are on the line.
There is a very puerile belief around opinionated junior devs which an inconvenience in the code justifies redesigning whole systems, even if they are still quite capable of performing their functions. Things need to be clean and ideally elegant, and any minor deviation is cause to tear everything down and start over.
In the meantime, experienced and competent engineers instead get the system back in working order and downgrade the criticality of existing issues until it's convenient to address them.
Go ask a project manager if all bugs should block a release, or if it is ok to release something with a known bug. Listen to them. Perhaps then you'll understand why patching works.
Comment by classified 11 hours ago
I'll believe it when I see it.
Otherwise, I completely agree.
Comment by hocusPokeUs 8 hours ago
Been saying to boos and rotten veggies "it's just electrical state in a machine; yall are deluding and debasing yourself for money." And "if you want to improve security stop using leaky abstractions to configure your machines".
Even better is that AI is replacing all knowledge work it just reinforces how absolutely bonkers the US "services economy" is and giving social stature to credentialed randos whose skills are whittled down to cooking frozen meals, grubhub, and Excel Sheet expert; unserious country peddling before the eyes of the poorest manual workers all these years that their degree in accounting earned them the right to wander through someone's home town across the world via resource intensive travel
Or better yet just being born rich earns them that!
This is an unserious country and social culture top to bottom to continue to assume modern people of nations rebuilt since Murica bombed them to the stone age don't see how Americans are drinking their own piss since the end of WW2; refusal to acknowledge externalities like generational churn (among all the others) makes it clear America is an unserious country.
HR and accounting and even a law degree do not make the individual worker bee irreplaceable themselves; whole lot of arrogant low skilled "knowledge workers", biological fuzzy VHS of often debunked pseudoscience studies now 30 years outdated... can pound sand.
Set aside the circumlocutions inherent to old thinkers like Marx and Adam Smith and capitalism is a social/communal discovery of services and prices; seems pretty fucking social and communal. But that's not "the Sacred Jedi Texts!"[1]
Top to bottom American society is a clown car stuffed full
[1] will take to my grave Luke screaming about Jedi Sacred texts was a metaphor for dogmatic people in general
Comment by psychoslave 12 hours ago
Comment by lifeisstillgood 11 hours ago
But we are out in the pacific now, and boats that sink make no profit, so we will evolve again, and again.
But it’s hard to blame Moana standing on a pacific island for not inventing a nuclear powered battleship.
(Trying not to be too blame-y on either side)
Comment by threatofrain 11 hours ago
Comment by lifeisstillgood 11 hours ago
That’s changed - we are in stormier weather and need better boats, but it would have been unlikely to have built battleships to sail the lagoon just in case…
Comment by bmurphy1976 7 hours ago
That is a choice I make every day when I get into an argument with Claude about the chosen design and how I find it insufficient. Does it slow me down? Sure, a little. But at least I know the foundation is bedrock instead of sand.
Comment by mikewarot 51 minutes ago
It's insane that we haven't yet migrated to capabilities based Operating Systems with default deny instead of default permit.
Maybe this will be the push to get us over the hump?
Comment by getfacl 13 hours ago
Comment by wickedsight 11 hours ago
These certs only exist for compliance, not for security. Nobody cares they don't add any extra value, because they allow every business to say "my supplier was ISO certified, so I did my due diligence".
And nobody really ever gets prosecuted for lying during their audit.
Comment by 1970-01-01 11 hours ago
https://consilien.com/news/what-is-iso-27001#how-do-you-actu...
Comment by getfacl 9 hours ago
I wouldn't agree with that. What I would say is that to pass the audit you must appear to be good at security. More to the point, you must appear to be doing many things that most companies don't actually want to do.
Comment by olowe 9 hours ago
Comment by wickedsight 9 hours ago
I have. What I wrote was based on personal experience working for an ISO 27001 certified company and being very involved in the (successful) audit and recertification.
> You will not pass the independent audit if you're bad at security.
Incorrect. You will pass if you have all required processes documented and have documentation that show you follow those processes. Whether you actually do follow the processes is irrelevant, the auditor just needs to think you do.
Also, you can be incredibly bad at security and still be ISO 27001 certified. For example:
Process: Describes that employees use an encrypted password vault for secret management. Keys in the vault are rotated each year and when employees leave.
Reality: Keepass database is sent through email, the password is "CompanyNameVault2023!", which is shared in the same email. Keys are never rotated because it's annoying.
This will 100% get you ISO certified, in my experience at least.
Comment by 1970-01-01 6 hours ago
Comment by getfacl 9 hours ago
Comment by doublepg23 6 hours ago
Comment by lesuorac 10 hours ago
Comment by inom155 8 hours ago
One thing that I find confusing is you can always prompt a model to find more bugs, and invariably it seems to always be able to
Comment by j45 8 hours ago
Comment by gavmor 7 hours ago
Bostrom touches on this somewhat pessimistically:
> The apparent safety of a tool-AI, meanwhile, may be illusory. In order for tools to be versatile enough to substitute for superintelligent agents, they may need to deploy extremely powerful internal search and planning processes. Agent-like behaviors may arise from such processes as an unplanned consequence.
Comment by Silagi 8 hours ago
Comment by Pedro_Ribeiro 8 hours ago
Comment by Silagi 8 hours ago
Comment by doublerabbit 8 hours ago
Double side sword. If you were the creator, the owner of the source tree, easy win. Why not. You know what it will change and the states changed from it.
If you're someone with no understanding to the code has been tasked with "use ai to fix bugs" that is where the trouble starts. When you don't hold the full knowledge of the source tree then you should be shy in using LLM's to fix bugs.
How do you know that bug fix doesn't cause issues further up-stream?
Comment by Lucasoato 13 hours ago
Yes, AI developed code might have vulnerabilities or bugs, but so already has your JSP based web server that has Oracle and God knows what else among dependencies.
Comment by stuaxo 13 hours ago
Comment by JsonDemWitOster 13 hours ago
Comment by fusslo 12 hours ago
Instead of a highly structured engineering process it's more like a gardening process of figuring out what works, what doesn't, what the Business actually wants vs what their requirements were
So more debt because there's more code but it's easier and simpler to change that debt. And in 3 years the product is EOL so shrugs
Comment by jryle70 12 hours ago
Comment by rglover 10 hours ago
Developers would do well to remove the word fast from their vocabulary. It has done tremendous damage to human systems (and the mental well-being of the people dependent on those systems). The whole reason the "technical debt" exists is because there's an addiction to speed. The sad part is that when you poke at that with most people, they couldn't give you a clear reason as to why they need to move fast; just that they have a sense that it's "how everyone does it."
Comment by thakoppno 11 hours ago
Comment by menomatter 3 hours ago
Comment by joshuaS98 11 hours ago
Comment by pmarreck 1 hour ago
I hope the effects of AI result in a massive reduction of PHP and Python prevalence.
Comment by kristianpaul 52 minutes ago
Comment by correlator 6 hours ago
Comment by sleepyams 9 hours ago
There is a concept of the hourglass model [0] of internet protocols, where IP sits at the waist. I believe that similar things are probably true in any "stack" based structure, such as computing, where the protocol at the waist has the most staying power (which at the current moment could be e.g. x86, or something) because it's essentially the universal translation layer that enables all the language protocols built above it to be agnostic to all the transport protocols below it. So, I see it as an open question as to what the best way to affect change in a situation like this is. I'm interested in if anyone else has any thoughts along these lines.
Comment by arlattimore 4 hours ago
Comment by program_whiz 13 hours ago
All those "paying down" technical debt with AI, I would advise reading the code _very_ carefully. Sometimes the "solution" which passes some test or adheres to some described functionality is incredibly narrow / brittle, and/or has failure cases that are non-obvious.
Yes I'm using $latest_model on high, its still not there. Unless your threshold for "fixed" is "the 1M line lean proof runs, we don't know how it works", it still requires a lot of attention, hand-holding, and scrutiny, especially when fixing issues / working around failures (where it tends to shortcut or invent workarounds).
Comment by fg137 11 hours ago
Turns out nobody wants to use them, they caused many subtle bugs in existing features and is a distraction for many things we do.
The developer left the team, and I am left with purging all of this.
Comment by program_whiz 10 hours ago
Comment by tjohnell 12 hours ago
Also, those very average developers are also the ones doing the code review.
Comment by program_whiz 10 hours ago
Anthropic is literally creating their own git alternative because they are slopping so much code. You think its 100% secure? Even if it has bugs at 10% the rate, there is 10x more of it to go around.
Do you think Anthropic's git alternative tool is more secure and better written than git? Hah, I bet it was just some vibe-coded thing they did over a couple of days, when it starts bogging down, dropping commits, or getting unstable, they just have to prompt harder! I wonder why they won't open source or release to the public, its probably too dangerous to release!
Comment by radu_floricica 9 hours ago
Comment by alex_duf 8 hours ago
Comment by Kevin_Flynn 13 hours ago
So, the headline should read,
"AI is helping us eliminate 40 years of technical debt."
Since AI found the bug, and probably also generated the patch at this point.
An abundance of solutions is a problem for people who sell solutions.
Comment by trgn 9 hours ago
AI removed 80k lines of code and nothing broke. much of this code was written by AI few months earlier.
Comment by carra 5 hours ago
Comment by animal_spirits 5 hours ago
Comment by misiek08 10 hours ago
Thanks to The Lord everytime someone discovers the vulnerability he reports it right to the owners/maintainers and there are no bad actors out there keeping it as secret and using for years.
Comment by samuelknight 11 hours ago
Anthropic is arguing for formal verification [0]. This is an excellent idea on the edge of feasibility, but it's hard to see how this can work for the majority of software that has to adapt rapidly and are so open ended that it's difficult to even conceptualize all the input modes. It's an easy prediction that we will see verification on large trunks of fundamental software in the next few years, and a sub-verification confidence scheme for the software on top of it.
Comment by hiker 10 hours ago
Comment by poelzi 13 hours ago
My approach is https://tix.im/ - not launched yet. But from expedience: modeling everything as workflows of composeable nix flakes (expert systems from programs, skills, agents, workflows and their snippets, DBs blobs, service descriptions, ...) and then run everything in highly isolated environments. It is not only so much more efficient, reduces errors, increases quality, reduces unnecessary cognitive load, ...
I start to move more and more daily tasks into the systems. Even UI programs become interactive workflow steps, which allow me to attach from anywhere back to a task in an environment optimized for the task.
Dependency updates for example, a workflow. A subgraph from a security scanner expert checks each change in the dependency tree.
Comment by hatefulmoron 12 hours ago
Comment by poelzi 10 hours ago
tix.im combines conecepts of expert systems with LLMs and other AI systems. Everything is composable via nix flakes. Any change is modeled via workflows that run in isolation. Each step type, is composed from the nix flakes defined and installed in the environment. Not only the projects nix flake (devshell), but also the tix project enabled experts. vx-maintainer is my software development expert - does only know about conecept of software development. Each workflow implemented is steps and workflows. For example, each change lands via land-feature workflow, it has ~40 steps. Workflows have complex parameters that define the flow inside the workflow DAG. In this case, rebase is mechanical, if it fails, medium smart rebase fixer is spawned, then mechanical step checks if all commits arrive in the rebase. Then it builds a collections, spawn a review swarm, open-code-review, etc... everything is tracked in an own issue tracking system and workflows can be restarted with minimal overhead.
Then I have an optimized queue-manager, an own agent prompt with an goal that can never be fulfilled. He manages that items from the tracker get done and land, extra workflows mechanically later check the landing.
rust expert, knows about rust specifics, has snippets, agents, skills etc. security expert has workflows with red-teaming, cyber models that get registered at the internal LLM gateway for usage, ...
I start to model everything in my daily live as an expert system now, with workflows that can be used by other expert flakes.
Workflow steps can be shared and also run wayland and x11 apps, rmux multiuser sessions etc. So, my goal is to move most of my tasks into some specialized environment.
And this is just a small part of the architecture - it is fully decentralized and user owned.
Comment by sevenzero 13 hours ago
>But from expedience: modeling everything as workflows
No thanks. I like private computing. I like all the fun stuff you can do with Computers. I'll never want agent based computers.
Comment by poelzi 10 hours ago
Comment by sevenzero 10 hours ago
Comment by Jtarii 11 hours ago
Comment by jareklupinski 13 hours ago
my webserver never needed a million lines of code just to pass bytes to other people
Comment by adregan 10 hours ago
Comment by jareklupinski 8 hours ago
nothing i'm proud enough to open to the world yet, but my personal public server will be my first deployment, and will definitely "Show HN" then too :)
Comment by 0xfedcafe 12 hours ago
[1] https://www.theverge.com/ai-artificial-intelligence/949601/a...
[2] https://openai.com/index/coordinated-disclosure-of-critical-... - "To help more defenders use these capabilities, we’ve committed $1 billion in subsidized Daybreak access through Daybreak for Frontline Defenders, focused on organizations with limited security resources protecting essential services."
[3] https://www.anthropic.com/glasswing - "Anthropic is committing up to $100M in usage credits for Mythos Preview across these efforts, as well as $4M in direct donations to open-source security organizations."
Comment by spcoder 10 hours ago
i hope we get to a place where we keep the same number of developers employed as today, but leverage ai to make all software better across the board. hopefully the people in power realize that we've been understaffed this whole time.
Comment by cj 10 hours ago
Not technical debt, just a preference for doing things a different way (which admittedly might be a better way, but just because there's a better way doesn't mean the current way is bad)
Comment by dymk 5 hours ago
Comment by mawadev 13 hours ago
Just consider how laughable some working conditions in software are and then come to realize how architecture is shaped by the organization communication patterns and not reality. You will see it in a different light then.
Comment by fractaline 8 hours ago
Comment by rglover 9 hours ago
I've never considered this problem from this perspective, but damn, I think you're dead on.
Comment by rothos 2 hours ago
Comment by vjvjvjvjghv 9 hours ago
Comment by c-linkage 9 hours ago
Comment by ederamen 2 hours ago
Comment by ctrlaltdileep 12 hours ago
Comment by 1970-01-01 11 hours ago
Comment by cushychicken 12 hours ago
AI is enabling us to fix longstanding software security issues that nobody had the time, skill, or desire to find
All a matter of perspective, I suppose.
Comment by brodouevencode 11 hours ago
They've always had the first two to varying degrees - it's the desire that killed most motivation.
Comment by jmull 10 hours ago
Should be something like:
> nobody is paying to find and fix
AI lowers the cost, but certainly doesn't eliminate it. I think we can hope significantly more technical debt will be paid off, but don't expect too much.
Comment by ferguess_k 10 hours ago
Comment by tony-vlcek 6 hours ago
Comment by igsomething 6 hours ago
Comment by pyrphoros 8 hours ago
Comment by tlhunter 12 hours ago
Aren't there still vulnerable services right now being hacked?
Comment by balderdash 8 hours ago
Comment by sva_ 13 hours ago
Comment by 0c3ca83 10 hours ago
What do you think the chance of nonzero budget and time being allocated for this is, given that almost no company has seen real consequences to the business?
Comment by bmiedlar 9 hours ago
Comment by WalterBright 7 hours ago
Comment by hoppp 9 hours ago
Yeah, hmm a lot of people don't actually auto update their PHP so.. it's not like the vulnerability disappeared from the wild.
Comment by m000 12 hours ago
Comment by 0x4e 11 hours ago
Comment by pletnes 11 hours ago
Comment by nwmcsween 7 hours ago
Comment by manmal 7 hours ago
Comment by goda90 7 hours ago
Comment by manmal 6 hours ago
Comment by raincole 7 hours ago
Comment by nwmcsween 6 hours ago
Comment by bearjaws 13 hours ago
I fear it's only going to get worse.
Comment by EQYV 13 hours ago
Comment by xbar 11 hours ago
Comment by IshKebab 13 hours ago
Comment by pixl97 12 hours ago
Comment by artvepa80 10 hours ago
Comment by stevefan1999 13 hours ago
Comment by CyberDildonics 12 hours ago
Comment by xbar 11 hours ago
Comment by CyberDildonics 2 hours ago
Comment by amelius 13 hours ago
Comment by eecc 12 hours ago
Well now, Great Responsibles, it's clear you too were out of your collective depth and were all paying for your smug hubris. Perhaps take a bite out of that humble pie you've obviously never tasted?
Comment by ThePhysicist 12 hours ago
This summer I had my Docker registry poisoned with malware artefacts, which again is something that no sane attacker would do before as it required downloading my images, carefully modifying them and then overwriting them, ensuring they work as before. Now it's just a simple prompt away. I am definitely ramping up my defenses, everything that doesn't need to be Internet connected goes behind double VPN and port knocking schemes, essentially making my infrastructure invisible from the outside, all unknown packet types get hard dropped, end-to-end encryption of all internal traffic, web endpoints behind CDN proxies and WAF providers (as much as I hate doing that), double and tripe security mechanisms everywhere, maximum restriction of privileges, ultrafast automated patching and AI based continuous monitoring, audits and interventions. You should also get used to that, this will surely still get much worse as attacker tooling gets better.
Comment by utopiah 10 hours ago
Everybody wants a $1 Website, avoid security updates, pay nothing for maintenance, and done yesterday.
It has nothing to do with AI or a specific language or stack. As long as cost cutting will be the culture, driven by the economical model of competition, then we will accumulate technical debt BECAUSE it is the strategy.
Comment by daveguy 13 hours ago
Comment by contravariant 12 hours ago
We'd be replacing decades of technical debt with debt we'd have no hope of paying.
Comment by worble 12 hours ago
Comment by jayd16 10 hours ago
Instead we're spending the time churning out busy work fixing bugs that were too low value to care about a year ago. We have a new street lamp so we're working on anything it illuminates.
I haven't really seen the same kind of push towards verifiable software. It's mostly around solutions that will perpetually burn tokens and justify usage of the new toys.
Mine is a cynical take for sure but I honestly really miss meaningful feature work but that's not what the market's interested in right now, I guess
Comment by amelius 13 hours ago
Comment by trkaky 12 hours ago
Comment by _Dimage 9 hours ago
Comment by 0xbadcafebee 5 hours ago
Bridges don't get driven on until they're inspected. And they only pass inspection if they were built according to standards, and meet design specifications that have to take safety into account.
How we get there, how we enforce it, is another conversation. The point is that we have to start getting serious and be real engineers. The bare minimum standard should be that you can drive a truck over it without worry. How much software can you confidently say that about?
Comment by doublerabbit 3 hours ago
It's only been the latest years where software has had a hard crash in quality.
Comment by wickedsight 11 hours ago
Back then, you could log in as admin to any phpBB forum by copying a generic admin cookie from a shady web site. You could often then also compromise the hosting company that was running the forum, because nothing was really separated.
You would try to trick your friends into installing Sub7 so you could open their CD drive.
You could easily capture other people's data on public wifi networks in restaurants and hotels.
You could change the price of the pizza you were ordering through devtools in your browser.
You could infect half the world by emailing a picture of a blonde tennis player.
Sure, there have been quite a lot of pretty serious zero days recently. But many of them have been pretty hard to actually abuse, in part thanks to all these layers we've added over time. So I guess I'm not as pessimistic as many of you are. Especially because we all seem to be incredibly aware of what's happening and are acting on it.
Comment by doublerabbit 9 hours ago
The hack was simple enough that you could change the price in the HTML and it could be purchased for that amount. You were suppose to implement verification yourself on checkout but no one did.
Comment by 1-6 6 hours ago
Comment by logifail 8 hours ago
... "by legacy sites"? :O
That's a pretty spectacular mischaracterisation of PHP's place on the web.
<cough> Wordpress <cough>....
Comment by j45 6 hours ago
I'm not a PHP person, but so much of the world actively runs on it silently.
All technologies have gaps. PHP was affected differently not because it's old, but because it's so ubiquitous. Python is as old, so is Javascript, and many others.
Comment by sevenzero 13 hours ago
Comment by malka1986 13 hours ago
Comment by pressbackbonela 8 hours ago
Comment by j45 9 hours ago
Comment by cmdrk 5 hours ago
Comment by ExoticPearTree 11 hours ago
At my current day job, if I mention “hey, how about you update PHP to at least a supported version” people start whining about how that takes a lot of time and if really anyone would try to attack our public facing services written in… PHP.
Next time we get hit I can sit back, eat some popcorn and say “told ya’”.
Comment by anovikov 12 hours ago
Comment by jmyeet 12 hours ago
I've been bullish on Rust for years because memory safety is simply too important and Rust can essentially prove an implementation correct and solve a certain class of very important bugs. It doesn't solve every problem and there is a complexity cost.
I've also said that writing multithreaded code is simply too complex. In an era of out-of-order execution, different levels of caching, asymmetric cores, hyperthreading (although this is largely dead now?), branch prediction and having to support multiple architectures it's simply too hard for most people to write correct multithreaded code. Period. If you find yourself as a non-library developer creating a mutex or spawning a thread, somebody has screwed up.
This is also why I think the single-threaded async/await model with post-request teardown of PHP is the correct choice for most people.
We're going to go through a dark period of a lot of 0days. As the article notes, this is paying 40 years of technical debt. It will result in a more secure ecosystem however. I believe this will ultimately also hamper black markets, governments and 3 letter agencies who trade in those 0days.
As painful as this all is, it will ultimately be a good thing, one of the few to come out of AI (IMHO).
Comment by SoftTalker 10 hours ago
Comment by with 7 hours ago
Comment by shevy-java 6 hours ago
I think one should, of course, be critical of those who write C code for PHP, but at the end of the day, some part of that criticism also goes to C. C is both a king and a problem. And it seems very hard to fix the problems of C.
Comment by lynx97 7 hours ago
Comment by TheRealPomax 10 hours ago
Wow. Love it or hate it, but tell me you have no idea what PHP's place in the web ecosystem is without telling me you have no idea what PHP's place in the web ecosystem is.
Comment by arbor-group 10 hours ago
Comment by rahulsawhneyonl 10 hours ago
Comment by 0xmattclark 2 hours ago
Comment by SlightGenius 9 hours ago
Comment by lvyafei 10 hours ago
Comment by honkostani 12 hours ago
Comment by marsven_422 11 hours ago
Comment by LoboxPlatform 13 hours ago
Comment by LoganDark 13 hours ago
Keep in mind that every PHP application globally was advised to completely cut network access or entirely shut down until the patch was available, so this was global critical infrastructure failure. I'm surprised lives weren't lost as a result.
And no, from what I've seen, it had nothing to do with the few entities that did rightly ignore the embargo after the exploit was already being circulated, even though the PHP devs are seemingly still putting resources into spinning it that way.
I'm sure the defender community will eventually come up with a way to patch security vulnerabilities just about as fast as they can be discovered and exploited.
Comment by 12312987 12 hours ago
AI does find things, but it is a small percentage the overall vulnerabilities.
IEEE is of course and industry owned organization that shills for BigAI, NSA and any other entity with money or influence.
Comment by IX-103 11 hours ago
Comment by jryle70 12 hours ago
Who do you shill for? because that sounds like someone who want to fuel the flamewar.
Comment by 12ajhg 11 hours ago
Comment by 091273153 3 hours ago
Comment by doublerabbit 9 hours ago
Comment by sam_lowry_ 12 hours ago