The Five Levels: From spicy autocomplete to the dark factory

That's how I've been using and treating it, though I'm not primarily a developer. I work in ops, and LLMs write all sorts of disposable code for me. Primarily one-off scripts or little personal utilities. These don't get shared with anyone else, or put on github, etc. but have been incredibly helpful. SQL queries, some python to clean up or dig through some data sets, log files, etc. to spit out a quick result when something more robust or permanent isn't needed.

Plus, so far, LLMs seem better at writing code to do a thing over directly doing the thing, where it's more likely to hallucinate, especially when it comes to working with large CSV or Json files. "Re-order this CSV file to be in Alphabetical order by the Name field" will make up fake data, but "Write a python script to order the Name filed in this CSV to be alphabetical" will succeed.

My growing (cynical) feeling is that AI-generated code is legacy-code-as-a-service. It is by nature trained on other people and company's legacy code. (There's the training set window which is always in the past. There's the economics question of which companies would ever volunteer to opt-in their best proprietary production code into training sets. Sure there are a few entirely open source companies, but those are still the exception and not the rule.) "Vibe code" is essentially delivered as Day Zero "Legacy Code" in the sense that the person who wrote that code is essentially no longer at the company (even if context windows get extended to incredibly huge sizes and you have great prompt preservation tools, eventually you no longer have the original context and not to mention that the Models themselves retrain and get upgraded every so many months are essentially "different people" each time. But most importantly the Models themselves can't tell you the motivating "how" or "why" of anything, at best maybe good specs documents and prompts do, but even that can be a gamble).

The article starts with a lot of words about how the meaning and nature of "tech debt" are going to change a lot as AI adoption increases and more vibe coding happens, but I think I disagree on what that change means. I don't AI reduces "tech debt". I don't think it is "deflationary" in any way. I think AI are going to gift us a world of tech debt "hyperinflation". When every application in a company is "legacy code" all you have is tech debt.

Having worked in companies with lots of legacy code, the thing you learn is that those apps are never as disposable as you want to believe. The sunk cost fallacy kicks in. (Generative AI Tokens are currently cheap, but cheap isn't free. Budgets still exist.) Various status quo fallacies kick in: "that's how the system has always worked", "we have to ensure every new version is backwards compatible with the old version", "we can't break anyone's existing process/workflow", "we can't require retraining", "we need 1:1 all the same features", and so forth.

You can't just "vibe code" something of equal quality if you can't even figure out what "equal quality" means. That's many the death of a legacy code "rewrite project". By the time you've figured out how every user uses it (including how many bugs are load-bearing features in someone's process) you have too many requirements to consider, not enough time or budget left, and eventually a mandate to quit and "not fix what isn't broken". (Except it was broken enough to start up a discovery process at least once, and may do so again when the next team thinks they can dream up a budget for it.)

Tech debt isn't going away and tech debt isn't getting eliminated. Tech Debt is getting baked into Day Zero of production operations. (Projects may be starting already "in hock to creditors". The article says "Dark Software Factory" but I read "Dark Software Pawn Shop".) Tech debt is potentially increasing at a faster than human scale of understanding it. I feel like Legacy Code skills are going to be in higher demand than ever. It is maybe going to be "deflationary" in cost for those jobs but only because the supply of Legacy Code projects will be so high and software developers will have a buffet to choose from.

Spot on. That's why so far all of the supposed solutions to 'the programmer problem' have failed.

Whether this time it will be different I don't know. But originally compilers were supposed to kill off the programmers. Then it was 3G and 4G languages (70's, 80's). Then it was 'no code' which eventually became 'low code' because those pesky edge cases kept cropping up. Now it is AI, the 'dark factory' and other fearmongering. I'll believe it when I see it.

Another HN'er has pointed me into an interesting direction that I think is more realistic: AI will become a tool in the toolbox that will allow experts to do what they did before but faster and hopefully better. It will also be the tool that will generate a ton of really, really bad code that people will indeed not look at because they can not afford to look at it: you can generate more work for a person in a few seconds of compute time than you can cover in a lifetime. So you end up with half baked buggy and insecure solutions that do sort of work on the happy path but that also include a ton of stuff that wasn't supposed to be there in the first place but that wasn't explicitly spelled out in the test set (which is a pretty good reflection of my typical interaction with AI).

The whole thing hinges on whether or not that can be fixed. But I'm looking forward to reading someone's vibe coded solution that is in production at some presumably secure installation.

I'm going to bet that 'I blame the AI' is a pattern what we will be seeing a lot of.

> There's this notion of software maintenance - that software which serves a purpose must be perennially updated and changed - which is a huge, rancid fallacy. If the software tool performs the task it's designed to perform, and the user gets utility out of it, it doesn't matter if the software is a decade old and hasn't been updated.

If what you are saying is that _maintenance_ is not the same as feature updates and changes, then I agree. If you are literally saying that you think software, once released, doesn't ever need any further changes for maintenance rather than feature reasons, I disagree.

For instance, you mention "security implications," but as a "might" not "will." I think this vastly underestimates security issues inherent in software. I'd go so far say that all software has two categories of security issues -- those that known today, and those that will be uncovered in the future.

Then there's the issue of the runtime environment changing. If it's web-based, changing browser capabilities, for instance. Or APIs it called changing or breaking. Etc.

Software may not be physical, but it's subject to entropy as much as roads, rails, and other good and infrastructure out in the non-digital world.

Sure, but the reason why this is the case is simple: writing software is easy. Writing good software is stupendously hard. So all those manyears that went into maintaining software were effectively just hardening, polishing bug fixes and slow adjustment to changing requirements and new situations. If you throw it all out whenever the requirements change you never and up with something that is secure or as bug free as you can make it.

That's definitely a pattern people are already starting to have good results from - using multiple "agents" (aka multiple system prompts) where one of them is a security reviewer that audits for problems and files issues for other coding agents to then fix.

I don't think this worked at all well six months ago. GPT-5.2 and Opus 4.5 might just be good enough for this pattern to start being effective.

Honestly I’m not sure we’re not there yet, run this prompt as a ralph loop for 2 days on your codebase and see where you at...

Well isnt the point humans wouldn't need to do basically anything?

It would be 'desirable' because the value is in the product of the labour not the labour itself. (Of course the resulting dystopian hellscape might be considered undesirable)