The Tyranny of the Small
I’m in the strange position of feeling like I’m always taking the anti-AI position, despite thinking it’s really cool for it’s own sake, and that it’s something that might might completely change how laymen interact with computers. But, being a contrarian interacting with the constant hype-cannon around this subject forces me into the situation again and again. This is a shame, because I really think LLMs are cool, and I like using them for various tasks. Hell, they’re better at discussing philosophy than the median internet commentator, which is a low bar, but one that regular humans fail to hit about half the time. It’s tiring arguing against something that I think is really cool, but at this point satisfying the hype requires super-intelligence that renders white collar work nearly obsolete.
Today, I’d like to talk about the idea that us humans will move up to working with pure architecture, using LLMs more like compilers that turn feature descriptions into working programs. This is never going to happen. There are so many reasons why it isn’t going to happen, not least of which is that systems managed by non-coding architects always fall over in a pile of bloat and sadness. That, however, isn’t the one that I want to talk to you about today. Today, my argument is much more simple. My argument is that the details matter, and you have to understand them.
The details matter so much that existing compilers leave them for you to decide in most cases, and LLMs, even if they function as futuristic compilers, will have to leave them open for you as well. You’re not going to be high up in the ivory tower thinking about architecture. You’re going to be down in the muck, grappling with details, deciding if a double is the right thing to use in a particular place for business logic. Programming is dominated by the small details. The mechanics of the program. Programming is dominated by a tyranny of small things, because the small things are where things go wrong. Did you use a string in a sql query without escaping it? Your beautiful architecture doesn’t matter, your database is on twitter. Does your business logic require that 0.1 + 0.2 == 0.3? The most perfect module system won’t save you, you need to get down in the muck and figure out if you’re using IEEE-754 floating point numbers, or something else. You, my brave architect, have to care about all those little details that you wanted to abstract away. If you don’t, you’re not an architect or an engineer, you’re a sucker who’s taking liability for a system they don’t understand.
I have taken down production many times in my life. Without fail, it’s always something small. It’s an incorrectly indented yaml block, a trailing slash on a url that shouldn’t be there, an off by one error, an environment variable that has a typo. There are so many degrees of freedom available to screw up software, and it’s always something small that does it, and as long as LLMs have a 5 percent chance of getting something small wrong, we have to review every line that comes out of them. Reading code, famously, takes longer than writing it if you want the same level of understanding, which saps most of the value from LLMs for engineering. And you should be aiming for that level of understanding if you’re going to call yourself an engineer.
I can already hear one particular objection. We catch these failures with tests. To be sure, tests have value, and they can catch a lot, but you’re telling me that you’re going to test that a function that doesn’t interact with a sql database isn’t open to sql injection attacks? You won’t, there’s just too many degrees of freedom to test everything. Testing everything possible is going to end up with petabytes of tests that will take weeks to execute. No, of course not, we only mean tests that are relevant to the situation. Well, how do you know which ones are relevant without a deep understanding of the code? You have to review it. You have to care if it’s using a double or not. You have to care about the format of strings. You aren’t an architect at that point, you’re down in the muck, engaging with the mechanics of the system.
Are there times when the code truly doesn’t matter and you can just generate it all without caring? Sure. Ad-hoc visualization comes to mind. Are there times when it’s genuinely useful? Sure, it speeds up boilerplate, but there are ways to automate most boilerplate without spending several percent of gdp on gpus. This stuff is genuinely cool, but I just really want people to stop saying that they 5x’d their engineering output. You didn’t, you 5x’d your code output, and saying that your engineering output scaled that much is just holding up a sign saying “I don’t understand what I’m shipping”.
Until next time.
- Gene