How local LLMs provide more value than Frontier Models

Slow is smooth, smooth is fast

Ever since LLMs got good enough to output plausible code, the software industry has sprinted into full-blown AI adoption. Quotes like Dario Amodei’s prediction that "90% of code will be written by AI" or Boris Cherny’s declaration that "programming is a solved problem" quickly turned into memes. But while most code written today might touch an LLM, programming itself is far from solved.

We’re starting to see an awakening from the AI euphoria. Projects like OpenJDK and Zig have banned AI contributions entirely, while others like the Linux kernel team, have instituted strict rules around their use.

I run a software development agency which means balancing developer craft with client urgency, of which I wear multiple hats throughout the day. By trait and heart, I am a developer. I love to code and build, shall I say “craft” things. In day‑to‑day work, being efficient matters a lot, especially in an agency where many clients might wait for our feedback at the same time. AI promises instant gratification, but immediate solutions often trade away the learning experience of manual execution and introduce a good chunk of technical debt. Oracle puts it best, "LLMs are good at generating code and tests that look correct but are poorly designed once you look closer."

Early experiments with cloud-based coding agents left us underwhelmed. At first, when we tried the usefulness of AI‑Coding Agents, we were not convinced by the results and discarded them. I even positioned myself against Dario’s claim that 90% of code will be written by AI. Amodei’s "90%" stat might hold true for broad public metrics like overall GitHub commits, but it falls apart when looking at real client codebases with actual stakes.

As a business owner, I believe that the core functions of an agency cannot and should not be outsourced. Since Coding‑Agents became an important part of development work, my drive to bring the technology in‑house is high. Dependence on external platforms leaves you vulnerable to sudden service shifts or lock-outs. We could already see it with The Fable posse that you can just be cut off. This realisation pushed me toward local models. Already in late 2025 I noticed that local LLMs are not to be underestimated. And in mid‑2026, I am testing a development workflow that relies solely on local models, using the latest Apple Chips.

Compared to the cloud‑powered LLMs from the big vendors, a local LLM is smaller and slower. One would think that it would be less capable. Well, that depends. A small, specialised model, coupled with the relevant context and prompt, can be as effective for coding as the frontier models are. The biggest difference being that the user will need to know which model to pick for the task, and if the task is even suitable for local processing at all. One-shot prompts like "build me a 3d game like Sims - no mistakes!" will not work on a local model. The frontier models can be understood as do-it-all machines. While local models are less universal because of their reduced size.

The loss is a gain

The loss in speed – local models tops out around 11 - 60 tokens per second compared to a frontier model’s 100+ t/s.

I highly criticise the approach to software development that was introduced by frontier models. Frontier models spew code faster than human comprehension can keep up. They generate code so fast that it is overwhelming for the developer to thoroughly review the result. A frontier model easily generates hundreds of files in less than ten minutes, making it impossible to read, understand or even correct the architecture decisions that are baked into this solution. Real code review collapses into a rubber-stamped "looks good to me" as long as it works in a manual test and gets the job done.

But this is how slop creeps into a codebase.

Now compare this to the workflow with a coding agent, powered by a local LLM.

Local models force discipline. Since the local model has to deal with the hardware limitations of a consumer‑grade laptop, it cannot one‑shot a solution of that size. Tasks that are supposed to be handed over to a model, need to be small, clearly defined and contain relevant context and skills. This costs a significant chunk of the available context.

It leads to more human involvement in the overall process because under these circumstances, one can keep up. Tasks get planned thoroughly before implementation and the chunks of generated code are small enough to be reviewed properly. Changes to the architecture are still possible as I review early and more often. This workflow lowers the implementation time while creating maintainable solutions.

Only local LLMs from now on?

No. Frontier LLMs still have their place. The power of a data‑center‑driven LLM does have other valid applications, mostly where code quality is a lesser concern. When starting a new project, creating an MVP that can be demonstrated and discussed with the client is an easy task for a frontier model. This throw‑away code does not need to be maintainable. It serves the purpose of shortening the time spent in a discovery‑phase while communicating better. AI can deliver a testable MVP of an idea in minutes instead of days which translates into a massive reduction of time‑to‑market.

Does AI lower the cost of producing code, yes. But it doesn’t lower the cost of making good architectural decisions. I believe architectural thinking may become one of the most important human skills in software engineering. The next generation of engineers won’t be valued primarily for how quickly they can write code or navigate a compiler’s edge cases. Rather, they will be valued for their ability to design the blueprint, knowing what to build, how the pieces should fit together, which trade-offs to make, and what will remain maintainable years from now.