To Succeed With Agentic AI, You Need An Orchestration Strategy

If you, as many of your customers, business partners and competitors, are engaging in the use of AI agents (usually referred to as “agentic AI”), you have surely heard consultants, service providers and industry analysts mention the term “orchestration.”

This is one of those words that have suddenly risen to popularity in IT circles, even if it is often used out of turn or without a clear understanding of what it refers to. However, in agentic AI, the term orchestration typically refers to coordinating AI agents, traditional application systems, data sources, business partners, devices, and human activities to accomplish a business goal.

More specifically, it consists of the following aspects:

• Integrating AI agents with enterprise applications and data
• Enabling AI agents to work with each other (multiagent system)

Injecting AI agents into classic, “deterministic” business processes (AI Agent workflows)

1. Integrating AI Agents with Enterprise Applications and Data

An AI agent is a software component that applies AI techniques (e.g., LLMs, but also business rules and others) to achieve a desired business goal by:

• Sensing events happening in the digital or physical world
• Deciding dynamically, what is the best course of action needed to respond to a given event
• Performing the decided course of actions

AI agents operate in your “environment”, that is, your organization’s data, applications, documents, devices, processes, ecosystems, partners and, of course, people.

An AI agent operates in your environment through a well-defined set of actions you have to specify: which data and business applications it can access and how, which business processes it can execute, the human approvals and sign-offs it requires, and the authentication and authorization rules it must follow. This set of actions effectively establishes what the AI agent “is capable of doing” in relation to the environment. This is why it is often referred to as the AI agent’s “enterprise skills” (or “e-skills” for short). An enterprise skill is basically a self-contained piece of deterministic business logic implementing a meaningful business capability (for example, “Tax Calculation”, “Invoice Validation Against PO”, “Execute Domestic Money Transfer,” and “Calculate Claims Settlement”). The e-skill must also provide a well-defined interface and expose a clear and precise description. Via such a description, the AI agent can “discover” what e-skills it has available and “understand” the effect of each e-skill execution. Finally, the AI agent can trigger e-skills via its interface, increasingly using MCP, which is becoming the standard protocol for AI agent-to-environment communication.

Behind the scenes, the AI agent decides “the best course of action” to respond to a certain event based on the event itself and the AI agent's “context”, that is, all the information the AI agent has available to determine what to do. Context includes:

• The AI agent's conversation history 
• Its job description (often called “system instructions”) specifies what the AI agent can and cannot do
• A knowledge base of policies, procedures, constraints, and business and compliance rules
• The AI agent’s memory, essentially a “log” of its previous activities
• Other external knowledge

Once it has devised “the best course of action”, the AI agent triggers the e-skills needed to make the intended impact on the environment. Because the “best course of action” is not typically predefined, but “decided” at runtime based on context, often AI agents are referred to as “non-deterministic”.

In summary, Orchestration is required to implement "enterprise skills" and curate the necessary contextual data, thus ensuring that AI agents can reliably operate within existing enterprise applications and data sources.

2. Enabling AI Agents to Work with Each Other (Multiagent Systems)

“Multiagent systems” or “collaborative AI agents” refer to an orchestration-enabled architecture by which multiple specialized AI agents work together to achieve a specific goal. With this approach, AI agents can combine their knowledge, skills, and understanding to address more complex problems.

The benefits of this architecture are notable, including the well-known advantages of modular software design, such as simplified development, testing and operations. This architecture also provides AI practitioners the opportunity to reuse custom and 3rd-party AI agents across multiple scenarios. Moreover, a set of medium granularity AI agents is more predictable and less prone to hallucination than a single coarse-grained monolithic AI Agent. However, there is no free lunch in multiagent systems either: orchestration of AI agents from different vendors is complicated, and the necessary technology (e.g., the A2A protocol) remains immature. Multiple AI agents working in unison expose a security attack surface larger than a functionally equivalent monolithic AI agent. Additional testing, observability and governance issues could emerge, and the operational costs could be higher.

In summary, Orchestration is key to enabling collaborative interactions between multiple agents. Still, you will have to balance the multiagent systems' benefits against the increased security risks, complexity, and operational challenges.

3. Orchestrating AI Agent Workflows

Ultimately, the goal of an AI agent is to support a certain business goal of the enterprise, but that doesn’t mean the AI agent itself will own every task required to achieve it. In many cases, AI agents handle only specific tasks, whereas traditional software applications, such as your ERP, CRM, SCM, and HCM, support others. 

Thus, AI agents can certainly act as the non-deterministic “process orchestrators” to address complex, uncertain, and dynamically changing scenarios. However, AI agents can also participate in traditional business processes orchestrated using classic “deterministic” automation techniques. In other words, a classic deterministic business process can “delegate” some of its tasks to AI agents, for example, to assist decision makers or to create complex documents by aggregating data from multiple sources.

In summary, Orchestration is needed to effectively integrate AI agents into structured, deterministic business processes as tasks.

What Is The Enterprise AI Orchestration Strategy … (and Why You Need One)

To support the three aspects of enterprise AI orchestration—integrating AI agents with their environment, multiagent systems, and AI agent workflows—you need to deploy technologies enabling AI agents to:

• Work on a rich and up-to-date context by moving documents and data from the environment to the AI agent's “context graph” in batch or real-time.
• Receive the events they must react to via "prompts" entered by humans, documents, messages, API calls, or event streams.
• Trigger actions in the environment via enterprise skills by setting off API calls, publishing events, sending files, and kicking off business processes.
• Coordinate multiagent systems via APIs, webhooks, event streams, and file exchanges.
• Implement enterprise skills by composing actions and transactions into atomic deterministic processes using orchestration/integration capabilities.
• Inject AI agents into deterministic business processes by enabling these processes to activate AI agents and retrieve their outputs via events or APIs.

You can relatively easily address these challenges for a single “proof of concept” AI agent by writing custom code, transferring files, and manually uploading spreadsheets and documents. However, you cannot adopt this opportunistic approach when you must implement and deploy dozens, hundreds or maybe thousands of AI agents across your organization. Simply put, opportunistic “spaghetti orchestration” doesn’t scale and becomes an obstacle to the widespread adoption of agentic AI within your organization.

The spaghetti approach means you end up with a myriad of partially overlapping tools; hundreds, if not thousands, of orchestrations you don’t know how to manage or control; patchy or nonexistent governance processes; and rapidly escalating costs. Ultimately, you will accumulate significant technical debt, which will prove impossible for you to handle and very hard to retrofit.

To avoid these risks and keep your costs under control, you should define and implement an enterprise AI orchestration strategy based on two crucial pillars:

• The technology platform - The integrated set of tools that your teams share to support your organization’s agentic AI initiatives and projects. This typically consists of a combination of classic enterprise orchestration technologies (for example, iPaaS, data integration tools, API management platforms and process automation tools) extended with AI orchestration capabilities, as well as specific technologies like model gateways, MCP gateways and agent gateways.
• The delivery model - A clear, unambiguous and explicit definition of who is responsible for selecting and delivering the technology platform, who is in charge of implementing the orchestrations and who designs and enforces the key governance processes and the associated KPIs (for example, shorter time-to-value for new agentic AI applications or reduced LLM token utilization). This model could range from fully centralized (a central team delivers enterprise AI orchestration projects for the whole organization) to fully democratized (business teams implement orchestrations themselves under the supervision and support of the central team) to a hybrid combination of the two. The choice of delivery model depends on many factors (for example, business goals, available skills, your company’s organizational and technical maturity), and it can evolve (for example, from centralized to democratized) as your company's business goals change.

Implement Your Strategy To Defeat The Spaghetti Approach and Reap The Benefits

As CIOs, IT leaders or AI leaders, you cannot underestimate the crucial role of orchestration for the success of your agentic AI initiatives. By designing and implementing your enterprise AI orchestration strategy, you will:

• Minimize acquisition, operations and maintenance costs by reducing the number of orchestration technologies to a lowest common denominator
• Stop the proliferation of point solutions that inevitably lead to the accumulation of notable technical debt
• Build up a cohesive set of skills, best practices and governance processes that will accelerate time-to-value for orchestration projects, optimize resource allocation and help you keep costs under control
• Ultimately, it helps you avoid the risks and costs of the spaghetti approach

Enterprise AI Orchestration is not optional; it’s a critical enabler for scaling your agentic AI strategy and avoiding the chaos of fragmentation.

‍
Massimo Pezzini is an independent strategic advisor and Workato Head of Research, formerly a Gartner analyst specializing in integration, APIs, cloud computing, and composable business. Provides strategic IT advice to global organizations and is a frequent speaker on integration technologies, architecture, and industry trends. Fluent in English and Italian, with interests in food, wine, music, history, art, and travel.