The architectural demands of an agentic system depend on where the complexity lies. A system handling dense, ambiguous requests needs to resolve language and intent before execution. One handling a large volume of operational lookups may instead need to minimise the cost of execution. These differences have direct consequences on the architecture.
In fact, the architecture’s complexity should be driven by the business problem and the way users need to interact with the system. It does not, however, eliminate the need to address four commitments: reusability, whether the system can change without being rebuilt; accuracy, whether each component behaves as intended; performance, whether the system can operate under its real workload and budget; and coherence, whether its components act on a shared interpretation of the request. These properties need to be considered early, alongside the readiness of the underlying data. I made this argument in Shift-Left for Agentic Systems , and this article traces how we arrived at it. The commitments emerged across two enterprise builds. The first established reusability, accuracy and performance; the second brought coherence into focus.
Architecture begins with problem formulation
Agentic design often starts from the wrong end. A common design mistake is to select an agentic pattern before establishing what the problem requires. An orchestrator with workers, a ReAct loop, or specialised agents may all be reasonable choices, but their applicability depends on the nature of the work. Problem formulation must, in fact, establish where deterministic processing is sufficient and where autonomy is actually needed. I discussed this discipline in my earlier article on the mental model for agentic systems .
Formulating the problem means scoping it along two axes. The data-source axis identifies what the system must reason over: structured tables may require query generation, APIs may require tool calling, and documents may require retrieval. The capability axis identifies what the user expects, whether that is a factual answer, a comparison against a benchmark, a recommendation, or a proactive signal. Considering both axes helps establish which mechanisms are actually needed and explains why the two projects described below ended up with very different designs.
Project one: A fleet-management system
The first project was a conversational search framework for a commercial-vehicle fleet-management platform, whose fleet owners and managers needed operational answers about where a vehicle was, what its mileage was, how it performed against a benchmark, or which vehicles were underperforming. The interaction was conversational, but no single response format could serve it.
A factual question called for concise prose, a trend or distribution called for a chart and a decisive operational value called for a prominent key–value result, as shown in Figure 1.
Figure 1: Mock UI of the fleet management assistant
Problem formulation led directly to the initial architecture. Because the platform exposed structured tables, service APIs and operational documents, one orchestrator coordinated among a SQL agent, an API agent, and a retrieval component; and a response generator assembled their results. Interpreting the request was the easy part. The relevant entities were generally a vehicle identifier and a date or date range, which a lightweight slot-filling step could resolve once and pass to every downstream component.
From the start, the framework was built for reusability, because it had to serve more than one future use case, and its prompts, models and pre- and post-processing were written as independently adaptable modules; none of them tied to a single application. It was also held to accuracy. Each component was evaluated against golden datasets, the end-to-end flow was evaluated across multi-turn conversations and those results governed prompt tuning, which gave us a sound baseline for a system that had not yet met production load.
Figure 2: Initial architecture of the fleet-management system with significant architectural effort going into orchestrating SQL, API, and retrieval agents across heterogeneous enterprise data.
What concurrent use exposed
Under concurrent, continuous use, the fleet system exposed a performance constraint we had not treated as a design requirement. The most demanding queries, like recommendations to improve operational efficiency of fleet, took more than thirty seconds to get a response to. More importantly, ordinary lookups, like mileage of a specific truck, travelled through the same expensive agentic route designed for multi-step analysis. The architecture did not account for the distribution of query complexity, and model-level tuning could not address that mismatch. The example queries and required responses are shown in Figure 1.
The correction had to be architectural. We split orchestration into a lightweight simple orchestrator, which routed straightforward lookups to direct tool calls and a complex orchestrator, which kept sequential agent coordination for the questions that actually needed it. We also placed a cache-augmented generation (CAG) component alongside the SQL agent so that recurring query structures stopped incurring generation cost each time, used prompt caching to cut the cost of repeated context and streamed intermediate responses to improve perceived latency. Together, these changes brought approximately 80–85% of questions below three seconds and perceived latency under one second, while keeping a longer path available for complex analytical requests.
Another workload would need different mechanisms, but the variables that governed performance here generalise: where reasoning occurs, how many model calls a request activates and which work runs sequentially. All three are fixed by the structure of the system, so once unacceptable cost or latency shows up in production, fixing it usually means changing that structure. Performance became the third commitment and we learned it by redesigning the system, because the project scoping had missed it.
Figure 3: Performance improved when the system reserved expensive reasoning for the requests that warranted it and stopped making every request pay for the most complex path.
Project two: Campaign planning under human control
The second project, a campaign-planning assistant for advertising operations, was defined by human control, since a planner had to verify every targeting decision before it took effect. Its user experience combined an interactive filter panel, a map showing candidate advertising frames by latitude and longitude and a grid for selecting the final set of frames by different criteria. Planners made their decisions through these elements, which meant they could not be treated as a presentation layer added after the system had reasoned.
Figure 4: Mock-UI of the campaign planning assistant- the chat window processes user messages, extracted entities are populated in the filter panel at the top, resulting frames are visualised on a map to showcase geographical spread and are shown in grid for planners to select from.
The system had to extract entities from the planner's request and populate the filter panel (shown in the Figure 3) early enough for the planner to verify or correct them, because a slow panel would have undermined the human-in-the-loop design. We started with all three commitments from the first project in place: a fast, economical model was selected from the outset, a prompt library supported multi-region configuration, entities were extracted before the orchestrator fired so that the interface stayed responsive and the slot-filling preprocessor from the fleet system was extended to the advertising domain. The initial design used independent parallel extractors, one per entity type, which was reasonable from a latency perspective since each extractor's results could feed the filter panel and the orchestrator as soon as they were ready. It did not, however, fit the problem we had.
Figure 5. The second project shifted the design centre of gravity upstream, prioritising early entity capture so that human verification could remain part of the operational loop.
Where the extractors disagreed
The campaign-planning domain was heavy on entities and on language understanding. There were roughly five broad entity classes, each with tens of subclasses and they had to be drawn out of dense, ambiguous requests. For instance, the mention of a single coffee shop could plausibly be read as a media venue, a point of interest, or an audience attribute. Each parallel extractor was accurate in isolation, yet the system as a whole could still be wrong, because every extractor was free to construct its own interpretation of what the planner meant. In the fleet system, with its small entity surface, resolving a vehicle identifier and a date independently had been safe. In campaign planning, entities take their meaning from their relation to one another and from the planner's evolving intent, so one component had to own that interpretation before the system distributed any work.We introduced a message-enrichment router for this purpose. It performed intent and entity recognition, decided which extractors should run, disambiguated across entity classes and treated conversation history as an incremental delta, updating its existing reading of the conversation with each turn. The router established a shared interpretation of the request before downstream extraction began. That interpretation was carried through the orchestrator and response generator, rather than allowing each component to independently reconstruct user intent. We came to treat this consistency of interpretation as coherence. Improving extractor accuracy alone would not have addressed the problem; the system needed a shared representation of the request that every component could consume.
Figure 6. The enrichment router resolves ambiguity across entity classes before the request is distributed to downstream components.
Shift all four left
Understanding where the complexity in a problem lies, matters much more than the choice of agentic patterns. The fleet system’s requests were narrow enough for slot filling, with the real complexity emerging after interpretation, that is, in routing, execution and caching under load. The campaign-planning system, on the other hand, had conventional execution requirements. However, its requests were ambiguous enough to warrant a dedicated upstream layer for entity disambiguation. Both systems honoured the same four commitments, but their architectural choices differed because the problems they had to address were different. The choice of patterns must therefore follow an assessment of where the system encounters uncertainty, variability, cost or ambiguity. It is this assessment that should determine the mechanism we choose, with the least unnecessary autonomy.
In both projects, every expensive redesign traced back to a commitment discovered after the architecture had begun to harden. Performance, deferred to load testing, became a re-architecture of orchestration. Coherence, deferred to user testing, meant inserting a new layer upstream of extractors that the filter panel and the orchestrator already depended on. The work arrived either way, only later, after other components had been built around the original design.
The corrective is to ask the scoping questions for every commitment alongside the questions about data sources and capabilities, while the architecture is still amenable and inexpensive to change. Underlying all these considerations is a more fundamental question: is the enterprise data ready to be reasoned over in the first place? I explore this in a separate blog: The AI-Ready Enterprise: Preparing Data for the Age of AI . The problem designs the system only when it is understood deeply enough. In both projects, the commitments we discovered late were already visible at design time and that is the case for raising all four in the first architectural conversation and letting the problem decide how much design effort each one receives.
