Build the Box

I don't know why I think it's a box.

It's just the picture I have in my head. The box as a container.

If you're a developer, you're not used to building the box. You're used to building the thing inside the box.

But development is different now. We're writing less and less code by hand and instead having agents do it for us.

Projecting from where we were a year ago and the quality we've seen from the flagship models, I suggest that 100% of your code is going to be written by agents before too long.

If you're working as part of any reasonably sized development team then there's a good chance you only understand a small percentage of the code which makes up your project. So what difference does it make if it was written by agents rather than humans?

When we do see the code of others it's often in the context of a peer review.

But it's not going to be long before you start asking what are they for. Agents are better at finding issues in review than your team. If you're looking for security holes, Claude understands them better. If you're worried about system design or "taste" then write a skill for Sol so it follows how you want things laid out.

You may think it's crazy now, but it's not going to be long before your agent is writing code that goes straight to production without human eyes on it.

And that's why you need to build the box.

And not just a box to contain what the agent produces. But a box that what you're building fits perfectly within. A box with a foam insert.

Coding is the most suited task for an LLM because it's possible to write a test to check whether or not the output is correct. Unlike a song or a book or a legal paper, code has a correctness that can be defined.

"Correct" doesn't mean "done".

We're all defining what we want - a feature, a bug fix - and we're giving that to our agents. We're then letting it write tests, and then code that passes those tests.

We're doing this because TDD is what we used to do.

But if the agent made the wrong assumptions then the tests do nothing except prove that the code the agent wrote passed the tests.

This is especially true in larger developments. If you have a significant new feature you want to add to a product, with multiple screens, database models and functionality then the amount of definition required is larger than the model context.

You're therefore tempted to break it down into smaller tasks. And then go through each task, red/green TDD as you go.

But what you really want to do is build the box. Give it the full context. Let it break it down itself. And let the box guide it.

If you define the box well enough then only the correctly shaped code will fit inside it. And when it doesn't fit, your agent can try again.

Building the box isn't easy.

It's going to take multiple departments across your business to do it right.

Customer success, product design, sales - they're all going to have to chip in. You're going to have to re-organise your business. The lines between departments in a software business no longer make sense.

We don't want fewer staff. We just need to find new ways of making everyone work together.

You all need to decide what the software should do and more importantly, how to measure whether it does.

Decisions need to be made in advance, not at implementation.

The engineers can work out how to prove that the code is well designed - typing tests and mutation tests and complexity tests.

But can the rest of the team find a way to prove that the code is what your customers actually wanted? That it's easy to use? That the UI isn't awful?

That's the new software development challenge.

Go beyond the initial development and your support staff need to put the right tools into the box so that the agent is able to debug issues, access data in a secure way, and help you resolve customer issues.

But the box can't do it all.

It can't tell you whether what you're doing is worth building.

Yet.