The Sorry State of Code Quality in AI and Machine Learning
This isn’t a conventional take, but having spent years immersed in applied mathematics, physics, and computer science, starting with languages like C, I’ve formed a strong opinion on the current state of AI and machine learning code. Frankly, today’s AI and machine learning code sucks.
You see this problem everywhere. Look at the code accompanying most research papers – its quality is poor by default. Even widely discussed open-source projects, like TorchTitan, often demonstrate this issue. While some in the community praise them for large-scale training code, digging into it reveals layers of abstraction and inheritance that add no real value and make it nearly impossible for outsiders to contribute without a significant, often wasted, time investment.
There’s an inherent problem in how people write code in this field. I believe Python, despite its ease of use, significantly contributes to this. It’s the “wild west” where you can do almost anything, but the cost is often ignored. People don’t pay attention to what they are doing.
This lack of attention to constraints leads to astonishing waste. I’ve seen projects consume hundreds of gigabytes of storage for very basic tasks. This waste, I suspect, stems from many modern coders never having experienced resource constraints, thus never learning to optimize. There’s an economic angle too: bad software that demands tons of CPU, memory, and storage is good for companies selling hardware. It pushes consumerism in technology, where more powerful machines enable writing “shittier code”. Instead of optimizing, the common approach in AI today is just to throw more compute at the problem.
Another major issue is the lack of standardization or principles in the backbone software for AI. While libraries like PyTorch are well-developed, only a limited handful of people truly understand the full stack well enough to debug or contribute. Too much code is written with the expectation that it will be thrown away, which leads to bad outcomes. This includes things like package installations taking enormous amounts of time, frequent errors, and relying on hacks.
Both inexperienced and experienced engineers fall into traps. Junior people prioritize getting it to work, regardless of how, and aren’t pushed towards best practices. Senior people often gravitate towards unnecessary complexity, abstractions, and solutions that aren’t needed. Abstractions are only valuable if they significantly reduce the work or code lines required; otherwise, they are just personal opinions without value.
We’re also still stuck in the 80s and 90s mindset of inheritance. Composition is a better alternative. Once you understand composition, you realize you often just need data classes with methods, composing them to build complex, yet readable and understandable structures. This avoids navigating confusing inheritance chains.
Looking ahead, AI assistance for coding might make things worse. It’s the classic “garbage in, garbage out” problem. It could make it harder for people to reason deeply and connect different parts of complex systems. Real architectural thinking requires innovation and considering all constraints, something current assistants struggle with.
So, what’s the alternative? While Python is terrible as a primary product language, it’s perfect as a glue language. Build your APIs in Golang, your algorithms in C or another compiled language, and use Python to call them. Just be careful about transferring large amounts of data across these language boundaries.
We need to value code longevity and simplicity. ML code from just five years ago is often obsolete. Contrast this with real software like Redis, written 10-15 years ago, which is still highly relevant and powers much of the internet. A powerful tool doesn’t need to be overly complex. For instance, my own blog runs from a single binary loading content from a single file into memory. It achieves its goal simply, without needing ten different services like a separate database, cache, object store, etc.
The situation is tricky, but understanding these problems and shifting our approach towards simplicity, constraint awareness, composition, and using languages for their strengths is crucial.
Cheers, IE