Build the Thing That Builds the Product
By Matthew Kim
Published
If only engineers can use AI to build software, every other team goes back to waiting in line. A better design has engineers build the platform that makes it safe for other teams to build for themselves.
Goldman Sachs has reportedly limited its advanced AI coding tools to engineers and strats, according to Fast Company, which cites Bloomberg. Traders and bankers lost access. The firm still offers a chatbot to all employees, and a spokesperson described the limit as part of an experimental period.
The worry behind it is reasonable. Bank of America CEO Brian Moynihan described the risk in September as "letting it start getting answers without humans checking to make sure the answer was right." A program keeps acting once it runs, so a mistake can repeat until someone catches it.
Restricting who can use the tool is one way to put the control. I think a better one is to build the checks into the system that makes the software. Most of the ideas below already exist in software engineering. What is new, and what I am proposing, is applying them to people who aren't engineers, working with AI that writes the code.
What happens when only engineers can build
Dana leads trade operations at a hypothetical bank. Every morning she compares unmatched trade confirmations across three systems by hand. A small tool could do it. She knows exactly what it should show and which exceptions matter.
Dana isn't an engineer, so she files a request. It joins the requests from every other desk.
That is a waterfall process. Work moves through stages in order, and each stage is usually owned by a different function. The business writes requirements and waits. Engineering builds from a ticket, and then testing and release take their turns. The people who understand the problem are not the people building the answer.
In my experience, this arrangement tends to produce three problems:
- Backlogs. The engineering group's capacity is fixed, and demand grows with every team that learns what AI can do.
- Firefighting. Urgent requests push aside planned work, so engineers spend their time reacting.
- Systems nobody owns. The engineer who built Dana's tool has moved to the next ticket. Dana knows what the tool should do now but can't change it, and nobody else knows why it was built that way.
These are my observations, not research findings.
From waterfall to agile to agentic agile
Agile development was a response to this problem. One principle in the Agile Manifesto says "Business people and developers must work together daily throughout the project." Agile teams put the functions they need into one small cross-functional team so the team can deliver on its own.
Business teams have also built their own tools for a long time, in spreadsheets and low-code platforms. This is often called citizen development. The people closest to the problem build quickly, and the results can be hard to govern: Microsoft's Power Platform guidance describes apps and flows left ownerless when their makers leave.
AI changes what those people can build. An agentic system is an AI that can plan and carry out multi-step work, such as writing, testing and changing code. It can produce far more capable software than a spreadsheet, which raises both the value and the risk. I'll call a team that builds this way agentic agile. In the best case, the team's owner describes what is needed, reviews what comes back and asks for changes, much as they would with an engineer.
That only works if the system runs inside guardrails, and engineering sets them: what the system can access, which security checks and tests every change must pass, and how a change gets rolled back.
Compare it with the agile picture above. The Build box is now an agentic system, and testing and security checks move into the guardrails. The team owns the problem and the app, and engineering owns the guardrails.
Engineers build the system that builds the product
Two existing ideas support the engineering half of this.
The first is platform engineering. In Team Topologies, Matthew Skelton and Manuel Pais describe a platform team that provides self-service tools so other teams can deliver on their own, and treats those teams as its customers. The New Stack summarizes the model.
The second is Zach Lloyd's version for AI. Lloyd is the CEO of Warp. In an interview with Latent Space, he said: "I think it can be extremely interesting if you view the job as meta-engineering: building the system that builds the product." That gives this article its title. He also gave a talk, "How to Build a Software Factory That Improves Itself", on the AI Engineer channel.
Lloyd says the most valuable thing to automate is "the main loop of software engineering: triage, specification, implementation, review, verification, shipping and monitoring," with teams deciding "the points where humans should be brought into the loop." His subject is a software company's own engineering work, and the people using his factory are engineers. Extending the idea to traders, bankers and operations staff is my proposal, not his claim.
Let the person with the problem build
Suppose Dana's bank has built that platform. She describes the tool she needs. The platform lets her tool read the three systems but not change them. Every change has to pass checks before release, and the platform records who owns the tool.
One caution: tests written by the same AI that wrote the tool are not enough on their own. The platform needs checks the builder can't edit, such as comparing the tool's output with known results, and a human reviewer for anything that touches money.
Once the tool is released, Dana owns it. Amazon's CTO Werner Vogels described a similar principle in a 2006 interview: "You build it, you run it." For Dana, that means deciding when to change the tool, roll it back or retire it, because she knows who uses it and when it has stopped being useful. A named owner on every app also addresses the ownerless-app problem described earlier.
Engineers own the platform: the access rules, security checks, approvals for anything that touches money, required tests, rollback and the owner record. When a tool breaks in a way a test should have caught, they add a check, and every app gets it. The builder owns the app, and engineers own the platform.
Who builds on the platform
In this model each team is a self-contained unit with its own agentic system and someone who can build with it. I can see two ways to staff that role. A team could train its own people, such as a trader who builds a tool for her desk. Or it could have an embedded builder, a person on the team whose job is building tools for it, much like an agile team with one expert building on behalf of the others. Either way, the team owns what gets built, and the engineering group does not take it back.
Who runs the platform
Platform engineering already treats the platform as a product, with customers and a roadmap. Someone has to decide what the platform should let people build, what "safe" means for each kind of app, and whether it works: whether Dana's problem was solved, how many tools were rolled back, and how many apps have no owner. What may be new is that the platform's customers are non-engineers building with AI. I think that makes this a new variety of platform product manager. That is my view, and I haven't seen evidence that settles it.
What exists and what is unproven
Already established: waterfall and agile as ways to organize work, platform teams that provide self-service tools, teams that run what they build, and governance for citizen-built apps, including owners and cleanup.
Still unproven: whether an AI agent can reliably build software for non-engineers inside guardrails, whether those guardrails can be automatic and trustworthy, and whether risk and compliance teams in a regulated bank will accept them. I don't know whether this works for a regulated bank.
A first test could be narrow: read-only tools built by a few trained people on one desk. Measure how often released tools needed rollback, how often tests missed a problem a reviewer later found, and how many tools ended up without an owner. If tests keep missing problems or ownerless tools pile up, the platform isn't ready for wider use.
The two designs side by side
Here is the same AI under both designs. On the left, waterfall: engineers do every step for every request in turn. On the right, agentic agile: engineers set the guardrails once, and each team builds inside them. Both sides run the same requests through the same steps: build, tests and security. The right side assumes the guardrails run tests and security checks automatically.
Matthew Kim is a principal at Neuralworks.