Building a Simple Invoice-processing Agent

Building a reliable AI agent rarely requires complex prompt engineering or endlessly appending text to a chat history. While early experimental patterns relied heavily on raw text-based loops — where a model freely generated thoughts, actions, and inputs via the traditional ReAct pattern — this approach lacks the structural predictability required for production applications.

An agent architecture is fundamentally a highly structured workflow engine. The LLM does not control the loop; it acts as a constrained planner that proposes one next step from the current state. The surrounding runtime handles the deterministic state transition logic, boundary checking, and tool execution isolation.

I will use a small invoice-processing agent as a running example: parse an invoice, verify it against a ledger, request approval if needed, and write the final entry only after validation.

1. Start with state

An agent's state should never be a generic array of chat messages. It is a structured Workflow Object that represents the single source of truth for the system's operational health, constraints, and business context.

When designing state, start with explicit, typed fields whenever possible. Only resort to generic payloads when the workflow shape is completely unknown. This keeps the data contract clean and easy to test.

2. Define tool schemas & registry

We also need to define the boundaries of what the system is capable of doing. Every tool represents a hard system boundary. To keep this completely deterministic, I split tool management into two parts:

Pydantic Schemas: To enforce strict, static input contracts for each capability.

A Centralized Registry: A single source of truth that pairs these data contracts with their actual execution primitives (handlers).

By declaring this "universe of capabilities" upfront, we ensure that no tool can be invoked without passing through a strict validation gate first.

The mock tools above make the runtime example reproducible without external dependencies. In a real implementation, the registry would keep the same contract but replace each lambda with a concrete tool implementation, as specified in Appendix. The point is that the runtime does not care whether a tool is a mock function, a local Python class, a database client, an MCP server, or a SaaS integration. As long as the tool exposes a schema and returns a structured observation, the surrounding loop remains unchanged.

In production, a tool is a system boundary. It has a schema, permissions, side effects, latency, error modes, audit requirements, and a state update contract. Not all tools are equal. Read-only tools can often be called automatically. Write tools need permission boundaries. Irreversible tools need confirmation, audit, and rollback strategy.

Here I use Local Tool Execution with registry. With local tools, the agent imports Python functions directly into the runtime. However, because our architecture completely decouples the Planner's proposal from the tool's internal mechanics, you can easily pivot this setup to modern standards like Anthropic's Model Context Protocol (MCP). With MCP, the agent talks to a decoupled tool server over a standardized protocol. This makes tool execution completely replaceable, auditable, and language/runtime independent—without changing a single line of code in your core execution loop, validator, or reducer.

3. Derive allowed actions

Instead of overwhelming the LLM by passing all tools at once—which degrades reasoning accuracy and wastes tokens, the runtime evaluates the current WorkflowState and derives a highly localized, permitted action surface.

This is a deterministic control boundary: for a fixed state and ruleset, the same workflow state should produce the same allowed actions. Deriving allowed actions is preemptive guidance. It narrows the action surface before the planner makes a decision. The runtime is effectively telling the planner: “At this workflow step, these are the only actions you may choose from.” This matters because the planner should not decide what it is allowed to do. It should only choose from the actions the runtime has made available.

In practice, deriving allowed actions serves three purposes:

First, it acts as dynamic sandboxing. The runtime prunes the model’s action space based on the current workflow step. If current_step == "extract_invoice", the runtime dynamically constructs and exposes only the JSON Schema for the parse_pdf_tool. It does not need to inject schemas for write tools, deletion tools, payment tools, or other high-risk capabilities that are completely irrelevant to the current step.

Second, it acts as a business-state gate. High-risk tool schemas should only appear when the state satisfies explicit conditions.

For example, the schema for write_ledger_tool should only be appended to the API payload when approval_status == "approved". That decision belongs in backend control logic, not in the model’s judgment or free-text tracking.

Third, it improves prompt and inference efficiency. If the system has 50 tools, sending all 50 full JSON schemas at every turn wastes prompt tokens, increases latency, and adds massive routing noise. Most steps only need a highly restricted action surface. Exposing two or three relevant tool schemas natively usually improves both cost and planner accuracy via constrained logit decoding.

Importantly, the allowed tools schema is a derived value — it should not be a core field of the persisted state. The runtime should recompute this clean schema payload at every iteration from the latest state using derive_allowed_tools_schema.

Example of allowed tools schema:

4. Ask the planner for one proposal

The LLM acts as the Planner. Because the allowed tools schema is computed outside the persisted state, the runtime passes it explicitly into the planner prompt. This forces the model to choose from the current action surface instead of inventing a tool or jumping ahead in the workflow.

Think of derive_allowed_tools_schema as the train tracks, while the LLM (Planner) is the locomotive engine. The engine provides the forward momentum and parameterizes the next movement, but the exact rails the train can physically travel on—the restricted set of tool schemas exposed in the API payload—are determined entirely by the hard-coded, rule-based track-switching logic of the runtime. This structural division of labor is the core foundation that enables state-driven architectures to handle mission-critical business workflows with absolute predictability.

We use native OpenAI Tool Calling strictly as a parameter-generation mechanism. The runtime passes the allowed tools schema. By enforcing strict: True and matching the schema to our internal state requirements, the model is stripped of its ability to return open-ended conversational text. Instead, it returns a structured, predictable payload such as:

The model returns a native tool call containing a tool name and JSON arguments. At this step, the planner has completed two tasks:

Action Routing: choose exactly one action_name from allowed_actions based on the current step. It acts as a switch operator on a pre-laid set of tracks—it cannot steer the train off the rails.

Parameter Extraction: Dynamically extracting the required parameters from the current WorkflowState context and place them into the arguments object.

Instead of allowing that call to execute immediately, the runtime intercepts it and converts it into an decoupled ActionProposal. We also enforce that the planner provides a concise decision_rationale inside this proposal—just enough to make the step auditable and inspectable, but not enough to turn your clean state history into an unstructured chat transcript.

Only after this proposal passes our strict validation gates does the runtime proceed to execute the actual tool. This allows us to harness the greatest strength of native tool calling—reliable, structured argument generation—while completely preventing the model from acting as the execution controller.

While this layout leverages native OpenAI Tool Calling paired with Pydantic to keep these underlying mechanics fully transparent,it remains 100% compatible with instructor. Moving to instructor for production is a seamless upgrade: it wipes away the manual serialization glue code and handles structured parsing natively, all while perfectly preserving this exact same decoupled validation architecture.

5. Validate before execution

An LLM proposal is fundamentally untrusted input. Before triggering any tool, the control logic acts as a strict firewall, intercepting and validating the proposal. Validation before execution checks whether the planner’s actual proposal complied with the allowed action surface and satisfies the current state constraints. This is where the runtime verifies the selected action, argument shape, approval status, permissions, and side-effect rules before any tool is executed. This boundary protects the system.

In a fuller implementation, validation failures should become structured observations so the planner can recover instead of silently retrying. This gate should also validate the argument schema against the tool's signature, execution permissions, side-effect levels, and human approval statuses.

6. Execute tools

Tools do not emit conversational text or markdown summaries meant for users. They must return strict, machine-readable data payloads—Observations—that the system runtime can reliably parse.

Inversion of Control for Failures: Notice that the try-except block completely isolates the runtime from buggy tool code or network flakes. If a database query times out or an API returns a 500 Internal Server Error, the system does not panic or crash. Instead, the exception is caught, containerized, and returned as a structured data payload (status="failed").

State-Driven Recovery: Because errors are captured as data states within the Observation, the downstream orchestration logic can handle them deterministically—choosing to decrement an error budget, trigger a back-off retry, or cleanly route the workflow to a human-in-the-loop remediation queue.

7. Update state with a reducer

The planner can only produce an ActionProposal. A tool can only return an Observation. The reducer is the only component allowed to translate that observation into a new WorkflowState.

The raw Observation returned by a tool is passed to a pure, deterministic reducer function. The reducer determines exactly how to advance the WorkflowState, increment error counts, or transition the current_step programmatically.

The reducer should be a pure state transition function. It does not mutate the incoming state object in place. It copies the current state, applies a narrow transition based on the action and observation, and returns a new state.

This is the critical divergence from traditional text-based ReAct loops: the model does not mutate history, rewrite state, or directly interpret tool output into workflow progress.

That separation prevents a common failure mode in agent systems: letting the model or a tool rewrite workflow state directly. If the LLM has an open-ended update_state tool, it can hallucinate fields, overwrite structured objects with prose, or mark a workflow complete because it “feels” done. If tools mutate global state directly, they become tightly coupled to one workflow and difficult to reuse.

8. Put the loop together

Finally, let's tie these modular components into a cohesive workflow engine.

Notice how the stop conditions are controlled entirely by concrete state attributes (error_count, max_loops, is_complete). These act as absolute hard boundaries, protecting application against infinite execution loops and cascading system failures.

sample_invoice.pdf

42.6 KiB

The smallest useful agent is not an open-ended chat loop. It is a controlled state transition loop:

State constrains actions

Planner proposes one

Validation gates it

Tools return observations

Reducer updates state

Runtime decides whether to stop

This is the core pattern. Everything else — MCP, workflow frameworks, multi-agent orchestration, long-term memory, observability, and evaluation — should build on top of this loop, not replace it.

Conclusion

By structuring the agent as a state-driven workflow loop rather than an open-ended conversationalist, we draw a sharp line between the system's non-deterministic and deterministic layers:

The Non-Deterministic Component (The LLM) is tightly constrained within the planner_propose_step boundary. It has no authority to corrupt the execution state, invent unauthorized tools, or bypass business logic gates. It acts strictly as a parameter-generation engine.

The Deterministic State Transition Logic (The Runtime) governs action spaces, enforces schema constraints, tracks error budgets, and drives workflow progression via predictable, immutable state mutations.

Understanding where these architectural boundaries sit clarifies the exact roles of modern agent tooling, transforming a fragmented ecosystem into a coherent tech stack:

Pydantic defines the strict, immutable argument contract.

Instructor handles the low-level parsing of structured model decisions cleanly.

MCP (Model Context Protocol) decouples the agent runtime from remote, independent tool servers.

Composio manages authentication-heavy, enterprise-grade SaaS integrations.

LangChain community tools and LlamaHub provide instantly reusable, off-the-shelf tool wrappers.

LangGraph manages complex, stateful, multi-step multi-agent orchestration.

By building a minimal runtime from scratch using native building blocks, we gain absolute visibility into the execution loop. This decoupling makes it trivial to drop in, swap out, or scale any of these specialized enterprise components as production needs evolve.

For a granular, engineering-focused classification of off-the-shelf tool wrapper catalogs, enterprise SaaS authentication platforms, and high-risk browser runtime suites, see Appendix B: The Agent Tooling Landscape.

Before reaching for heavy orchestration frameworks or complex multi-agent libraries, always start by defining the core state contract. By keeping execution loops explicit, lightweight, and fully validated, we retain absolute architectural control over the system boundaries, non-deterministic risks, and data lineages.

Appendix

A. Concrete Tool Implementations

PDF parser tool

The PDF parser can internally call the structured extraction pipeline we built earlier. From the agent runtime’s perspective, it is still just a tool that returns a typed observation.

Database query tool

Add to Tool Registry

A tool registry entry does not need to be a local Python function. It can wrap an internal service, an MCP server, or a Composio-managed SaaS integration. The runtime contract remains the same: expose a schema, validate the proposal, execute only after policy approval, and return a structured observation.

B. The Agent Tooling Landscape & Boundaries

When transitioning from a minimal, from-scratch runtime to a scaled production stack, you do not need to reinvent every connector. However, from an architectural standpoint, every external tool library is a system boundary that introduces distinct liabilities regarding state mutation, authentication, and execution risk.

We classify the modern agent tooling ecosystem into three operational tiers:

1. Tool Wrapper Catalogs (Stateless Utilities)

These are pre-packaged collections of atomic capabilities, highly effective for granting your agent rapid, off-the-shelf access to data utilities. While you can easily import individual wrappers from these catalogs to accelerate development, your deterministic runtime must still capture their schemas, validate inputs, and strictly govern their execution loop. Never pass an unvalidated, black-box framework toolkit directly to the LLM.

Toolkit / Catalog	Typical Engineering Use Case
LangChain Tools / Toolkits	General-purpose utilities (Google Search, SQL databases, Python REPL, third-party API wrappers). LangChain explicitly defines a toolkit as a collection of tools designed to be used together for specific domains.
LlamaIndex Tools / LlamaHub	Data-centric RAG utilities, data loaders, and query engine interfaces. LlamaIndex specializes in exposing complex indexing and document search pipelines directly to the agent as standard tools.
Haystack Components / Tools	High-performance enterprise RAG pipelines, advanced document retrieval, web search, and structured document QA.
Semantic Kernel Plugins	Function and plugin abstractions optimized heavily for the Microsoft, .NET, and Azure enterprise ecosystems.

2. Integration & Authentication Platforms (Managed SaaS Boundaries)

When your agent needs to interact with real-world enterprise SaaS applications (e.g., Gmail, Slack, GitHub, Jira, Salesforce), the bottleneck shifts from writing Python wrappers to managing secure infrastructure: OAuth flows, per-user sessions, token refreshing, fine-grained permission scopes, and immutable audit logs.

Platform / Toolkit	Production Architecture Role
Composio	A production-grade agent integration platform providing 1000+ managed SaaS toolkits, built-in OAuth/session handling, event triggers, and an execution workbench. Fully supports both native API delivery and MCP server architectures.
Zapier MCP / AI Actions	Business-logic-focused SaaS automation, enabling agents to tap into thousands of existing cross-platform Zapier workflows.
Pipedream	Developer-centric API workflows and serverless SaaS integrations that can be exposed cleanly as secure agent tool backends.
n8n	A powerful, self-hosted workflow automation engine that can expose complex, multi-step business logic endpoints directly to an agent loop.
Workato / Tray.io	High-end enterprise iPaaS (Integration Platform as a Service) systems used to bridge agents with rigid, legacy corporate IT systems.

3. High-Risk Boundaries: Browser & Computer-Use Toolkits

Unlike stateless APIs or read-only database queries, browser automation and GUI-manipulation tools allow the agent to execute arbitrary, stateful, and often irreversible visual actions on the open web. Because these tools operate outside structured, predictable API boundaries, they carry a severe execution risk and mandate strict sandbox isolation, zero network privileges to internal corporate infrastructure, and mandatory human-in-the-loop validation gates.

Toolkit / Framework	Execution Profile & Risk Mitigation
Playwright	High-speed browser automation. Ideal for programmatic web scraping, end-to-end frontend testing, and dynamic form submission. Requires strict DOM-element verification.
Selenium	Industry-standard browser automation framework built for robust, multi-browser cross-compatibility.
Browserbase	Fully hosted, secure, and ephemeral browser automation infrastructure designed specifically to offload the security risks of running headless browsers locally.
Stagehand	An AI-first browser automation abstraction layer built on top of Playwright and Browserbase, optimizing element selection via LLM vision.
OpenAI / Anthropic Computer-Use	Model-driven OS and desktop GUI manipulation (mouse clicks, keystrokes, screen capturing). Mandates an isolated virtual machine container (sandbox) and strict confirmation checkpoints.