Most engineering teams begin their agentic journey with the Agent Loop: a sequence of reasoning, acting, and observing where an LLM reads a growing context window to decide what to do next. In production, the "loop" often hides the very state we need to manage. As we move toward more autonomous systems, the industry is shifting from treating agents as chatbots to treating them as distributed execution systems governed by explicit workflow graphs.
Loops Hide State
The classic loop is effectively a "single-ready-unit scheduler". At any given moment, only one action is active, and the choice of what happens next lives inside an opaque context window rather than an inspectable policy.
This creates three structural weaknesses:
- Implicit Dependencies: The fact that "Step B" requires "Step A" exists only in the LLM's memory; there is no structural guard to prevent out-of-order execution.
- Unbounded Retries: When a step fails, the LLM autonomously decides whether to retry or skip, often leading to infinite retry cycles without an explicit contract for failure types.
- Mutable History: If an agent revises its plan mid-execution, the original plan is often overwritten in the context window, making it impossible to audit what plan governed which actions.
- Hard to Replay: Because the state is entangled in a conversational history, replaying a failure to debug it becomes nearly impossible.
Workflow Graphs Are Stateful DAGs
Production agent orchestration is about deciding which work can run in parallel, which work must be serialized, and which component owns authoritative state after each transition.
Some work is naturally parallelizable:
- web searches
- document summarization
- independent code analysis
- data extraction
- candidate generation
These tasks have low side effects, low shared-state contention, and low merge cost. They are good candidates for fan-out execution. But other work must be serialized because the state is authoritative:
- database schema changes
- financial transactions
- shared memory updates
- user-facing commitments such as sending emails, placing orders, or deleting data
These are not merely “tasks.” They are state mutations. If multiple agents mutate authoritative state concurrently, the system risks corruption, stale memory, duplicate actions, or irreversible user-facing mistakes.
The scheduler’s job is not simply to keep agents busy. It must preserve ordering, authority, and correctness across state transitions.
Production-grade agents therefore need workflow graphs: stateful Directed Acyclic Graphs (DAGs) that define execution explicitly. Each node has dependencies, inputs, outputs, retry rules, and contracts. The runtime maintains a ready set: nodes whose dependencies have been satisfied and are eligible for dispatch.
"The parallelism, dependency tracking, and bounded recovery are not LLM decisions—they are structural properties of the DAG." — From Agent Loops to Structured Graphs: A Scheduler-Theoretic Framework for LLM Agent Execution
Once execution is represented as a graph, parallelism becomes structural. The graph topology defines what can run next; the state ownership model defines what can run safely. If two tasks have no dependency edge and do not contend for the same authoritative state, the scheduler can dispatch them concurrently. If a node mutates authoritative state, the graph can force sequencing, approval, rollback, or bounded retries.
Workflow graphs are not merely an orchestration convenience. They are the structural mechanism that makes dependencies, state ownership, execution ordering, and recovery semantics explicit.
Version the Plan
If the workflow graph is the execution plan, then the plan itself must become a versioned artifact. Otherwise, every replan becomes a silent rewrite of history.
Once a task graph is generated and validated, its structure is frozen for that execution version.
A versioned plan can be represented as:
Notation | Field | Meaning |
id | plan_id | execution identity |
version | version | plan revision |
V | nodes | executable units |
E | edges | dependencies |
node config | tool/model/runtime config | |
contracts | expected outputs |
If the plan needs to change due to a reasoning failure, the system must trigger a formal replan protocol, generating a new versioned graph. This ensures a clean audit trail where every state transition is attributable to a specific version of a specific plan.
Schedule by Side Effects
Versioning tells us which plan is being executed. Side-effect classification tells the scheduler how each node is allowed to run. Not all tasks in a graph are created equal. Advanced schedulers classify executable units by their side-effect profile, and the scheduling policy respects these boundaries.
Node Type | Scheduling Rule |
Read-only | Freely parallelizable and retryable |
Idempotent | Safe for retry; results are consistent |
Reversible Write | Allowed with formal rollback/compensation protocols |
Destructive | Sequenced, strictly approved, and limited retry budgets |
By encoding side-effect levels into the graph, we prevent the scheduler from speculatively dispatching "dangerous" operations in parallel—a risk that grows as we scale team size.
Use Deterministic Checks
Side-effect-aware scheduling limits when a node can run. Deterministic validation decides whether it should run at all.
For hard constraints—like "Is this voltage level safe?" or "Does this code pass the type-checker?"—engineering teams are moving away from using LLMs as judges. Instead, they use deterministic safeguards.
Before an agent triggers an action, its proposed action is passed to a domain-specific translator that converts natural language into a formal mathematical representation. This is checked against hard-coded rules or formal methods. If the safeguard itself is AI-based, we are merely compounding uncertainty; deterministic checks provide the only true containment for hallucinated data propagation.
Takeaway
The production abstraction for AI is not a “smarter agent loop.” It is a stateful execution graph: a system organized around explicit state transitions, dependency-aware scheduling, versioned plans, deterministic control boundaries, and carefully managed side effects.
Intelligence alone is insufficient. Production reliability emerges from how execution, authority, recovery, and state mutation are structured across the graph.
- Author:Fan Luo
- URL:https://fanluo.me/article/production-agents-need-workflow-graphs
- Copyright:All articles in this blog adopt BY-NC-SA agreement. Please indicate the source!
上一篇
[Leetcode 240] 搜索二维矩阵 II
下一篇
Building Auditable LLM Workflows for Medical Coding