Agentic AI’s Real Test Starts After Launch, When Data Quality And Security Take Over

A working agentic AI demo proves almost nothing about whether the system will survive in production. The prototype is fast and cheap because it skips the corporate data that hasn't been cleaned and the connections that haven't been vetted. Those are exactly the conditions that break agents once they go live.

Claudionor Coelho is Senior Fellow for AI at Majestic Labs, an AI infrastructure startup founded by former Google and Meta silicon leaders to build memory-heavy servers for large AI workloads. He has spent years inside enterprise AI and security, with earlier turns as Chief AI Officer at Zscaler and as head of AI Labs at Palo Alto Networks, where he helped advance neuro-symbolic systems for security operations. That vantage point, close to the data and the connections that decide whether an agent holds up after launch, has him tracking a number most companies look past.

"You shouldn't look at how many agents people are deploying. You should look at how many of those renew, because that is the better metric," Coelho said. Renewals are where the truth shows up, because a clean proof of concept hides how often the production version gets quietly shelved within six months, and the first cracks usually show in the code itself.

• A breach over coffee: Coding agents hand software creation to employees who have never shipped production code, and almost none of what they generate passes a security review. The result is a quiet sprawl of unvetted servers running on machines never built to be servers. "Imagine a marketing person sets up a server to analyze his marketing data, then goes to a Starbucks and connects to public Wi-Fi. He is on an unsafe network with disposable code running in the background, software that was never vetted for security. Someone could get into the laptop through that open connection and leave something dormant, so when the laptop comes back to the office, a massive attack begins," Coelho said.

The risk compounds with adoption. It repeats on every laptop that can now write its own code, and the perimeter security teams have to defend stretches from a handful of monitored servers to the entire workforce. Few of them are staffed to test at that scale.

In early 2026, the Financial Times reported that Amazon linked a run of site outages to AI-assisted code changes and began requiring senior engineers to approve that code before deployment. Amazon disputed the account, saying only one incident involved AI tooling and that it introduced no such rule. The episode still points to a real tension, because any mandate to review machine-generated code assumes a level of attention that automation steadily erodes. It plays out in self-driving cars, where regulators insist on hands and eyes on the road even as drivers disengage.

• Hands off the wheel: "Once you turn on self-driving, your attention span on the technology drops. The same thing happens with coding agents. If a bug ships, you tell yourself you did not write it. Claude Code did," Coelho said. Over time, the cost compounds, as teams that lean on agents lose familiarity with their own code and the ability to debug it quickly when an incident hits.

• The 35% problem: Chaining agents in sequence works against reliability. In a 10-step workflow where each step succeeds 90% of the time, the full sequence completes correctly only about 35% of the time, and tighter coupling between steps pushes it lower. Running several models in parallel and letting them vote helps only when their mistakes are independent, and they often aren't. "The models are trained on the same web-scale data, so they tend to make the same mistakes at the same time," Coelho said. Other limits sit beyond the reach of scale entirely, problems with no efficient solution, no matter how large the model gets. Reliable results are an engineering outcome, earned by building systems that assume failure and catch it early.

• Cost of being wrong: The stakes of a mistake vary enormously from one use case to the next. A flawed marketing recommendation is recoverable, something a team can catch and correct. The same automation pointed at a medical device or a weapons system leaves no room for a single bad call. "Once the cost of a hallucination crosses a certain threshold, you cannot trust the response. That's the first thing you have to establish as a baseline," Coelho said. Setting that baseline turns deployment into a triage decision. Leaders rank their pipelines by what a failure would cost and grant autonomy only where that cost stays survivable. For an attacker, the logic inverts. A wrong answer costs almost nothing, so an agent can fail repeatedly and still win on the attempt that finally lands, while the defender carries the full cost of every miss.

None of this yields to a one-problem-at-a-time fix. The quality of the data determines how reliable the agents are, and the connections carrying that data determine how exposed the company is. Fixing one while the other stays broken changes little. Getting an agent to hold up in production is disciplined governance work, the kind a convincing demo never has to do.

What makes that work urgent is the fact that the threat wears a helpful face. The agent pulling data out and quietly shaping what comes back is the same one a team welcomes in to move faster. "Consider Morpheus in the Matrix," said Coelho. "He hops into the network, pulls out confidential information, and feeds false information back in. That's what the coming years look like."