Sharpening The Thesis With Domain Experts Keeps Prototypes From Failing In Production

Building a working prototype used to be the expensive part of a project. A frontier model will now write the pipeline, wire up a dashboard, and return something that demos well enough to win budget. The trouble starts afterwards, when the same system has to run against incomplete data, serve a thousand people at once, and satisfy a regulator who wants to know who approved the output. Speed at the build stage doesn't help with any of that. Whether the idea was worth building, and whether anyone can answer for what it produces, are both settled by people who understand the work.

Neeraj Mathur is the Chief Information Officer at SickKids Foundation, where he runs the technology strategy, operations, and infrastructure behind a $2.5 billion precision child health campaign. He is accountable for AI governance and cybersecurity. Mathur previously held technology leadership roles at Sanofi and started his career at Deloitte Consulting. The foundation is already running agents in production and building its project management office on Claude, so the questions he works through are the ones his own deployments raise.

"Most of these models are designed to build something if you ask them to build it, whether it's built right or not. That is where you have to apply your domain expertise," said Mathur. A model asked to produce a supply network graph will produce one, and the output arrives with the same confidence whether the underlying data supports it or not. Mathur ran into this issue on a side project he built outside his work at the foundation.

• Domain expertise validates the output: The project is called RUNWAY, and it assembles a supply network graph for a drug out of public data, then reads signals across the nodes to flag a shortage before it happens. Mathur built the pipelines with Claude Code, and nearly a decade in supply chain at Sanofi told him which of its answers were plausible. "If you're using AI, it tends to be overly optimistic, and this is where you have to apply domain knowledge," Mathur said. "Knowing what I know about supply chains and how drug supply chains work, I was able to challenge the model's outputs from time to time as they came back."

• Data quality limits reliability: RUNWAY worked better for some drugs than others, and the difference came down to how much had been published about each one. Products with enough public information behind them produced a usable proxy for the supply network. Others had too little published about them, and the graph came back with nodes the data did not support. "The quality of the data really was a big factor in the reliability of the output," Mathur noted.

Mathur reads any unevenness as an argument for settling whether an idea holds up before anyone commits engineering time to it. He spent most of his time on RUNWAY cutting the scope down to the drugs where public data could actually carry the method. "Sharpen the thesis before you build it," Mathur advised. "If you want to burn your expensive tokens, burn them in sharpening the thesis, because the build can come later. The build is now trivial, but having the right idea is the harder thing to do."

• Experts before product teams: Deciding that question early changes who joins a project and when. An expert brought in at the start sets the scope of the system, the cases it won't cover, and the data it depends on. The product and engineering teams then build against that definition. "The real domain experts need to be involved right from the get-go, because the domain expert is the one who will validate whether there is merit to the idea," Mathur said. "The product teams, the dev teams, all of them should follow after the idea has been validated."

• Containing what the model knows: A validated idea still has to reach production, where it raises a security problem that deterministic systems may have missed. A team could specify which records an API was allowed to return and be confident the boundary held. A foundation model offers no equivalent guarantee, since the exposure runs through the prompt. "With a language model you don't know, because you're feeding it data," Mathur explained. "Somebody could have a prompt somewhere written in a way that it could expose the data, and you wouldn't even be able to stop it." Building the application is only half the job, and the controls around it are the other half.

In healthcare, those controls answer to a regulator. Mathur expects agents in his sector to stop short of acting on their own for some time yet, because a person has to be accountable for the consequences of the decision. A regulator won't accept "the model decided" as an answer, so an agent can go as far as recommending a decision and no further. "AI augmentation to me is probably the answer, not AI automation," Mathur said. "AI can automate the data flows, but from a decision perspective it still has to augment the human who has to make the call."

• Earning the handover: Mathur starts from curated data, since a model drawing on whatever it can reach produces recommendations no one can trace back to a source. From there the method is the one quantitative teams have used for years: back-testing first then watching the system work alongside the people already doing the job. He widens what the system decides on its own as the agreement rate holds up. "Have it run in parallel, have the human monitor the outcomes and say, well, if I was to do this and the model also says that it would do X, and I was also going to do X, then that's validation that the model is trending in the correct direction," Mathur explained.

A system has to show agreement for longer than a person would, and Mathur is clear about why. A colleague who gets something wrong is forgiven and the work moves on. A system that gets something wrong is discounted, and the goodwill it built over months goes with it. Mathur treats that asymmetry as a design constraint. The parallel runs, the curated data, and the narrow first use case are all there to build a record long enough to survive one bad output. "The burden of proof becomes higher on an AI system to prove that it consistently gives good results before that trust is built," Mathur concluded. "As humans we sometimes have difficulty handing control."