The model is not the agent.
In production, an agent’s behavior is shaped by its runtime: which tools are visible, when retrieval happens, how retries are handled, whether actions are idempotent, what state is persisted, and who is allowed to commit mutations. This is why two systems using the same model can behave very differently. One feels reliable; the other loops, over-calls tools, violates policy, or becomes impossible to debug. The difference is rarely the prompt alone—it is the execution environment around the model.
As engineering teams move from simple chat interfaces to autonomous agents managing Kubernetes clusters or complex financial workflows, the focus is shifting from "prompting" to the rigorous engineering of agentic runtimes.
The following takeaways highlight why reliability is an architectural property, not a model one.
1. Framework semantics leak into model behavior
We often treat agent frameworks as neutral wrappers, but their internal conventions dictate how a model perceives and reacts to the world. System-level evaluations suggest that framework choice impacts performance as significantly as model choice within the same capability tier.
For example, benchmarking reveals that a framework's default behavior—such as forcing a tool call at every single turn—can cause high-tier models like GPT-5-mini to enter redundant clarification loops. In some cases, the model may misinterpret framework-specific error messages (like "max turns reached") and retry the same failed strategy repeatedly, consuming 10x more tokens than other model-framework combinations. These are not "hallucinations" in the traditional sense; they are system-level collisions where framework rules trigger model-specific failure modes.
"Implementation decisions substantially impact performance, including choices such as topology, orchestration logic, and error handling... framework conventions can combine with model tendencies to produce failures that neither component would exhibit in isolation."
2. LLMs should propose; runtimes should commit
A critical architectural flaw in early agent design is giving the model direct authority over a filesystem or API. Truly advanced systems adopt a layered architecture that separates the LLM proposal layer from the authoritative execution engine.
In this design, the model generates a structured patch or a suggested action (the proposal), but it never executes it directly. The engine validates this proposal against structural schemas, security policies (e.g., directory restrictions), and phase consistency before performing an atomic apply to the state. This ensures that generative non-determinism does not translate into uncontrolled state changes.
3. Tool calls need executable contracts
Traditional software uses type systems to enforce boundaries; autonomous agents require Behavioral Contracts to do the same at runtime. A production-grade contract moves beyond the prompt by defining a formal tuple of constraints:
C = (P, I, G, R) Where
P is Preconditions, I is Invariants, G is Governance, and R is Recovery. For example, a refund tool contract might require:
- Preconditions: The order status must be verified as "delivered" before invocation.
- Invariants: The refund amount must remain below a specific approval threshold.
- Governance: The action must be idempotent; a unique transaction ID is required.
- Recovery: If the tool fails, the system follows a specific logic for supervised recovery (e.g., alert a human operator or trigger a fallback path) instead of retrying without a fresh state read.
By making these metrics explicit, you transform "agent drift" from a mysterious failure into a measurable distributional shift.
4. Mutation requires evidence, not confidence
Advanced teams are shifting from "confidence-based" execution (where the model proceeds if it's sure) to "evidence-based" execution. This draws from Test-Driven Development (TDD) as a governance protocol: an agent is systemically blocked from implementing a change until it has first produced a failing test (the RED phase) that defines correctness.
This protocol prevents the model from speculating or adding unnecessary features. The system acts as a gatekeeper: it requires evidence of a failure before accepting a behavior-changing diff, and it enforces a bounded repair loop (typically
N=3 attempts) to prevent the agent from burning tokens in repetitive retry cycles."Development discipline shapes the trajectory of model generation rather than being applied after the fact... prompt engineering [is] a mechanism for encoding software engineering process invariants."
5. Multi-agent systems need execution semantics
Scaling to multiple agents requires moving away from static scripts toward hierarchical task management. This architecture treats agents like processes and threads, employing explicit parent-child task splitting.
For the system to scale effectively, the runtime must manage:
- Bounded Delegation: Parents recruit children for subtasks, but child resources (tokens, memory) are constrained by the parent’s budget limits.
- Join Semantics: Serial chains suffer from a "broken telephone" effect; for example, a 5-agent chain where each agent is 95% reliable results in only end-to-end reliability.
- Mutual Exclusion: The runtime must prevent multiple compute-intensive agents from over-burdening physical processors, coordinating their execution based on real-time resource availability.
6. Reliability lives in the runtime monitor
The final takeaway is that transparency is an architectural choice. When agents are deployed with behavioral contracts, they often appear to have "lower" compliance than uncontracted agents. This is the Transparency Effect: the contract reveals violations (tone degradation, threshold breaches) that were previously silent and unmeasurable.
By implementing a runtime monitor that tracks distributional drift (using metrics like Jensen-Shannon divergence), engineers can detect shifts in the agent's action space before they manifest as hard failures. This allows for corrective actions—like re-prompting with specific constraints or escalating to human oversight—long before the system enters an unrecoverable state.
"The value of contracts is not that they eliminate violations, but that they make violations measurable... revealing performance gaps that were always present but previously unobservable."
Looking Ahead
The next generation of agent systems will not be differentiated only by model quality. It will be differentiated by the runtime around the model: what tools are visible, what actions are allowed, what state is trusted, what mutations require evidence, and what happens when the system starts to drift.
The model can generate a proposal.
The runtime decides whether that proposal becomes reality.
That boundary is where production agent reliability lives.
- Author:Fan Luo
- URL:https://fanluo.me/article/agent-reliability-lives-in-the-runtime
- Copyright:All articles in this blog adopt BY-NC-SA agreement. Please indicate the source!
上一篇
Production Agents Run on an Autonomy Spectrum
下一篇
Design Agents Around Workflows, Not Chat Turns