The Harness is the Moat

Training an LLM has become such a common practice now that people are doing it on pre-1930s text just for a laugh. There are absolute benefits to having a massive compute cluster, a whole team of talented staff and a big pile of money - but there are limits to how much good data you can squeeze into a model, and the reinforcement learning pipeline is likely to bring diminishing returns as most software optimisation eventually does.

Which is why Anthropic are leaning hard into new Claude Code features. By making Claude the #1 choice for developers they make it less likely you'll bounce to other models. Let's be honest, most of us are switching between Anthropic and OpenAI models depending on which one we have remaining usage with. But if Claude Code comes with a bunch of tools and services which make it nicer to work with, then I'm less likely to switch.

It also takes the pressure of the model team to always have something new. Even if the model is just "good enough" then a lot of developers will stick with it because they like the tooling. That also becomes important as the gap between model capabilities shrink.

I've long thought that the harness will become the moat, especially as the open source models grow in capability. I think Microsoft and Apple know that too, which is why Copilot and the upcoming Siri features are model agnostic. If the app you use finds the right photos, writes the todo app you wanted, or surfaces the time of your mums flight - you won't really care about the model underneath.

We're so early in the process of working out the right set of tools, determining what the model can and can't do or training our staff how to take advantage of them. But we're also emotional animals and developers will especially stick with the tool they feel comfortable with, even if it's not the "best" option, but just the first one they used.

Longer term I think the larger development shops will start building their own harness. I can already see how a pre-packaged interface with the right tools, skills, knowledge access and guardrails would benefit our team. Anyone with a large codebase is going to benefit from the context being tuned depending on the part of the system they're working on.

I assume there are companies working on that problem already. Which is probably why you can really only use your Claude subscription with Claude Code.