JOURNAL / SYSTEM DESIGN

Before building an agent,
name the decision.

If the steps are already known, write them down as a workflow. Reach for an agent when the next useful step depends on what the system learns along the way—and only inside limits you can enforce.

An aerial view of roads crossing and joining in an urban interchange.SYSTEM DESIGN / PATHS / SYMPLOKA
CHOOSE THE ROUTE THAT FITS THE WORK

A more complex path is not automatically a better one.

Let the task determine how much discretion the system needs. Keep predictable steps in software and bound the decisions that vary.

01 / AGREE ON THE DISTINCTION

The key question is who controls the next step.

People use “agent” in different ways. Here, the useful engineering distinction is whether the software follows a sequence written in code or lets a model choose the next step and tool call from the results it observes.

Ordinary automationRules determine each step. Use it when conditions are explicit, actions need to be exact, and exceptions can be handed off.
LLM workflowA model interprets text or produces a structured result inside a sequence that the application controls. The order and permission checks remain defined.
Agent loopThe model can choose a tool, inspect its result, and decide whether another step is needed to reach the task’s goal.

These patterns can be combined. An application might use a model to read an unstructured request, run a predictable workflow, and reserve agentic steps for exceptions that need more context.

02 / LOOK FOR A REAL DECISION

Use an agent when the path genuinely depends on new information.

  • The task needs interpretation across messages, documents, or changing system state.
  • The next step depends on what a search or tool returns, rather than a fixed branch.
  • Exceptions make the existing rule set difficult to maintain, and the model can choose among bounded actions.
  • The result can be checked against a clear finish condition.
  • A person or safe fallback can take over when evidence is missing or a limit is reached.

If the process fits a short checklist, keep it as a workflow. If only one step requires language understanding, put a model there and keep the rest deterministic. An agent loop is useful only when its flexibility improves the outcome enough to justify extra runtime, review, and failure handling.

OpenAI’s practical guide identifies context-sensitive decisions, difficult-to-maintain rules, and unstructured data as promising places to evaluate agents. It also advises validating the use case first; a deterministic solution may be enough.

03 / WALK THROUGH AN EXAMPLE

Keep the task, then choose the smallest useful architecture.

Suppose a team receives supplier documents and needs to check them against an internal policy.

Known fields and fixed policyExtract named fields, validate the expected format in code, compare direct rules, and send missing or invalid data to a person.
Documents vary, steps stay knownUse a model to find relevant passages or normalize wording, then run the same policy checks in a fixed workflow and keep source references.
Unclear exception, several possible checksConsider a bounded agent that can consult approved sources and choose the next investigation step, then return its evidence and recommendation for review.

Do not let the model convert a recommendation into approval by itself. If a decision changes a contract, sends money, or creates another consequential side effect, enforce that authority in the application and keep the required human check.

04 / LIMIT WHAT IT CAN DO

Give the loop an explicit boundary and a stop condition.

Task boundaryName what the agent may accomplish and which outcomes it must leave to a person.
Tool boundaryExpose only the actions and data required for this task; enforce permissions where each tool runs.
Run limitsSet a maximum number of steps, time, retries, and resources. Define what happens when the limit is reached.
Recovery pathPreserve a useful account of the work and return control when a tool fails, evidence conflicts, or the goal cannot be verified.

Start with one agent and a small tool set. Split work across agents only if testing reveals a clear separation that improves results; extra handoffs add more places to lose context and more behavior to evaluate.

Read where agent permissions belong ↗
05 / TEST BEFORE EXPANDING

Compare the agent with the simpler route.

Run the same representative tasks through the current workflow and the proposed design. Check whether the agent completes more of the work to the required standard, what it costs in time and tools, how much review people need, and whether it stays within its permissions.

Include routine cases, ambiguous inputs, missing information, hostile source material, and tool failures. A successful demo shows that a path is possible; repeated evaluation shows whether the path is dependable enough for the intended use.

Build a task-level agent evaluation ↗
Measure the full development workflow ↗
ARCHITECTURE GUIDANCEOpenAI, A Practical Guide to Building AI Agents (2026) ↗Anthropic, Building Effective Agents ↗Anthropic, How We Built Our Multi-Agent Research System ↗