Enterprise AI’s Next Breakthrough Depends On Getting Context Under Control

The views and opinions expressed are those of Aman Sharma and do not represent the official policy or position of any organization.

Context is becoming the control plane for enterprise AI. A model’s built-in amnesia may be harmless in a one-off prompt, but it becomes a serious architecture problem when agents move through regulated workflows, connected platforms, and production systems that need to be governed, observed, and audited. That’s where prompt engineering stops being enough and context engineering becomes enterprise infrastructure.

Aman Sharma works on responsible AI governance frameworks and governance architectures for production AI workloads, also running an independent research program on safety and governance in clinical AI systems. He is a Principal Enterprise Architect AI/ML for Blue Shield of California, and he has multiple papers under review at IEEE and targeting NeurIPS 2026

"Every AI session has amnesia. It doesn't recognize what was sent to it before, and it doesn't recognize what comes after. Every AI tool that is getting called is just one endpoint call, and that is what prompt engineering is," said Sharma. He drew a hard line between prompt engineering and context engineering, arguing that conflating the two is where most enterprise programs go wrong.

• Configuration, not prompting: "Context engineering is the art of setting up systems so that you can dynamically pass context to the LLMs so that they understand what was told to them before," Sharma said. Prompts are one component of a much larger configuration ecosystem. With that configuration in place, hallucination goes down and the system operates like infrastructure as code. Without it, teams fall into trial and error. "You can make 50 prompt changes per day. You guess, the system hallucinates, you try again," he said.

• The missing architectural layer: Sharma compared the moment to early API adoption, when enterprises realized they needed gateways to enforce policy at the edge. "Now companies have realized they need the same thing for context engineering," he said. Major cloud providers have launched dedicated AI gateways and agent gateway platforms for that purpose, and Sharma called it the key component that had been missing. That gateway, paired with middleware orchestration, creates a single governed entry and exit point for all AI traffic. "The policy and governance that need to be applied have to be at the enterprise level," he said.

Sharma pointed to a friction point that has shifted from internal IT coordination to a growing divide between business and technology teams.

• Business versus IT: "The biggest friction I see now is business versus IT," Sharma said. Business leaders assumed that access to AI tools meant they could run workflows without enterprise solutions. "That sometimes scares me, because to have AI work, you have to have the foundation layers," he said. His organization opened the door for personal productivity use cases but drew a clear line: complex solutions still had to run through the enterprise platform.

• The POC treadmill: Without a production data strategy, enterprises end up trapped in a loop. "If you don't have data platforms and a strategy that works in production, not the POCs, then you are just in a trap of moving one step forward and two steps back," he said.

Governance and observability drew Sharma's sharpest warnings. As enterprise AI governance moves from policy documents into production infrastructure, his research examined what happens when multiple agents communicate across a pipeline.

• Multi-agent hallucination: "When multiple agents are talking to each other, they will agree on a fake drug and confirm that it is the right treatment," Sharma said. Individual agents might catch errors alone, but in a chain, they reinforced each other's mistakes. In regulated healthcare, that failure mode is not theoretical.

• Layer-by-layer observability: Sharma described a nine-layer stack spanning API management, routing, orchestration, context creation, and the LLM itself. Observability at the start and end points is not enough. "What goes in, what comes out, you have to track it at each and every layer," he said. He recalls his organization failing an internal audit because it lacked the observability to track where LLM tokens were going and who was consuming them.

For CIOs evaluating whether their AI architecture can scale, Sharma's position was direct: invest in system design before use case velocity and treat context as a governed infrastructure problem rather than an application-layer optimization. "We are not there yet," he said. But the organizations that absorb the cost now will have the architecture in place when the rest of the market is still running experiments. "If you don't control your context, you do not have an AI system in place."