From Decision to Action: Implementing Tool Calling, Idempotency, and Observability in AI Agents

In the first part of this series we laid out a principle: an agent's autonomy and its authority to execute actions are separate things, governed independently.

Here we bring that principle down to code: how you implement that boundary in a real system, without relying on the model to "behave."

How do you connect an AI agent to a real system without giving it arbitrary access?

By never letting the model generate free-form code or queries against your database or API. The correct pattern is to expose a closed set of tools with an explicit schema — tool calling or function calling — where the model can only request an action with an exact parameter shape, never execute anything outside that contract:

export const createPurchaseOrderTool = {
  name: "create_purchase_order",
  description: "Creates a purchase order for an authorized supplier.",
  parameters: {
    type: "object",
    properties: {
      sku: { type: "string", description: "Unique inventory SKU identifier" },
      quantity: { type: "integer", minimum: 1 },
      supplierId: { type: "string" },
      expectedDeliveryDate: { type: "string", format: "date" }
    },
    required: ["sku", "quantity", "supplierId", "expectedDeliveryDate"],
    additionalProperties: false
  }
} as const;

Here's a nuance that's easy to miss: a validated schema is not the same thing as a correct decision. Anthropic's documentation on structured outputs is explicit about this — guaranteeing that a model's output satisfies a JSON Schema ensures the shape of the response, not the accuracy of its content; the model can still generate a response that's perfectly valid and wrong at the same time. The schema is a barrier of form, not a barrier of truth. Content verification — does this supplier exist? does this quantity make sense? is this SKU active? — has to live in a separate layer, after schema validation and before execution.

What happens if an agent executes the same action twice?

It happens more often than you'd think, and almost always for the same reason: the agent sends a request, the operation completes server-side, but the response gets lost to a network timeout. The agent reads that as a failure and retries. Result: two purchase orders where there should have been one.

The fix isn't new — it's the same one payment APIs have used for over a decade. Stripe solves this with an idempotency key: a unique identifier the client generates for each operation; if the API receives two requests with the same key, the second one never re-executes — it simply returns the stored result of the first, even if that first request was an error.

For an agent, the natural key combines the run's identifier with the identifier of the specific action inside it:

// 1. Deterministic key generation per run and action step
const idempotencyKey = `${runId}:${actionId}`;

// 2. HTTP header dispatched to the transactional API (ERP / Gateway)
const requestHeaders = {
  "Idempotency-Key": idempotencyKey, // e.g. "run_8f31:create_po"
  "X-Agent-Step": "po_replenishment"
};

That detail — which sounds like a footnote — is exactly what separates a demo from a production system. In a demo, nobody retries anything. In production, networks fail constantly, and any action with real side effects needs this protection implemented in the service that executes the operation, never relying on the agent to "remember" it already did it.

Where should an agent's state live: in the conversation or in the database?

Neither one exclusively — and mixing them is one of the most common mistakes when moving from a prototype to a real system. It helps to separate three things that look alike but aren't the same:

• Business state — inventory, orders, suppliers, contracts. It lives in the usual transactional systems (ERP, database), not in the agent. It's the source of truth.
• Execution state — which step the flow is on, which tools were called, which approvals are pending, how many retries have happened. This does need to be persisted so an execution can be paused and resumed later — while waiting on a human approval, for instance.
• Memory and context — instructions, documentation, prior results relevant to interpreting the current task.

Treating conversational memory as if it were the system's operational state is what makes an agent grow erratic over time: the context window fills up with noise that shouldn't be there, and the business's source of truth ends up depending on what the model "remembers" doing instead of what the transactional system confirms actually happened.

How do you audit an agent's decision without relying on its chain of thought?

By not treating the model's internal reasoning as an audit mechanism — it's unstructured text, unverifiable, and its format can change from one call to the next. What is auditable is a structured trace of the execution: what data was queried, what tools were used, what policy was applied, what decision resulted, and what authorization it received.

{
  "run_id": "run_8f31",
  "workflow": "inventory_replenishment",
  "sku": "ABC-123",
  "tools_used": [
    "get_inventory",
    "get_demand_forecast",
    "calculate_reorder_point"
  ],
  "decision": {
    "action": "CREATE_PURCHASE_ORDER",
    "quantity": 500,
    "supplier": "SUP-42"
  },
  "policy_applied": {
    "max_order_value": 10000,
    "requires_approval": true
  },
  "approval": {
    "approved_by": "operations_manager",
    "approved_at": "2026-09-10T14:32:00Z"
  },
  "execution": {
    "idempotency_key": "run_8f31:create_po",
    "result": "success",
    "erp_reference": "PO-10452"
  }
}

This isn't a nice-to-have anymore — it's turning into a de facto standard. OpenTelemetry, the reference project for observability in distributed systems, publishes semantic conventions specifically for AI agents that standardize how tool calls, step-by-step reasoning, and inter-agent coordination get logged — precisely so these traces stay comparable across different providers and frameworks.

How do you know if an agentic system is ready for production?

With different metrics than the ones you'd use to evaluate a chatbot. Conversational response quality says nothing about whether an agent can safely operate against a real system. The ones that matter:

• Human intervention rate — what percentage of actions needed review.
• Rejection rate — how many of the agent's proposals an operator turned down.
• Tool-call success rate — how many tool calls succeeded on the first try.
• Duplicate-operation rate — the metric that confirms whether idempotency is actually working.
• Cost per complete workflow — not the cost of a single model call, but of the entire cycle through to the final action.
• Real operational impact — did stockouts, excess inventory, or rush orders actually go down? This is the only metric the business truly cares about; every other one exists to diagnose why it's moving.

A system that requires human review on 90% of its actions isn't failing technically — but it isn't automating anything either. That intervention rate is the most honest signal of how well-calibrated the risk-tiered autonomy architecture from the first part really is.

What happens when two parts of the system recommend different actions?

This shows up as soon as more than one signal source feeds the same decision — a demand-forecasting model suggesting you buy more, and a budget rule suggesting you wait. The robust way to resolve it isn't letting the model itself "decide which argument is more convincing," but turning each recommendation into a structured object with its own evidence and confidence level, and letting an explicit rules engine — not the LLM — apply the final policy:

IF   stockout_forecast < 14 days
AND  supplier_approved == true
AND  order_value < budget_limit
THEN
  approval = "automatic"
ELSE
  approval = "mandatory_human_review"

The model interprets and proposes. The policy, written as deterministic code, decides what's allowed. That separation is what keeps the system auditable and predictable even as the model behind it changes versions.

Frequently asked questions

What is tool calling (or function calling) in an AI agent?

It's the mechanism by which a language model requests a specific action — with parameters that satisfy a predefined schema — instead of generating free-form code or queries against a system. It's what lets you connect an agent to an ERP or any other API without granting it unrestricted access.

What is an idempotency key, and why does an agent need one?

It's a unique identifier tied to a specific operation that keeps a retry after a network failure from duplicating the action — the receiving system recognizes it already processed that key and returns the original result instead of repeating the operation. It's essential for any agent action with real side effects, like creating an order or sending a payment.

Why shouldn't a model's chain of thought be used as an audit log?

Because it's unstructured text, unverifiable, and its format can vary between runs. A defensible audit trail relies on a structured trace: what data was queried, what tools were used, what policy was applied, and what result the execution produced — not the natural-language explanation the model generates about its own reasoning.

What metrics indicate an agentic system is ready for production?

At minimum: human intervention rate, rejection rate of its proposals, percentage of successful tool calls, duplicate-operation rate, cost per complete workflow, and above all, whether it actually moved a real operational metric for the business.

Should an agent's state be stored alongside the conversation history?

No. Business state lives in the transactional systems, execution state (which step the flow is on) is persisted separately so it can be paused and resumed, and conversational memory is just supporting context — mixing these layers is one of the most common causes of erratic behavior in agents that have been running for a while.

This is the second part of the series on agentic system architecture. If you haven't read the first one — autonomy versus execution authority — that's where the governance principle behind all of this lives. Both pieces continue the thread from our earlier guide on agentic development for software teams.

Technical sources

• Structured outputs — Claude Docs (Anthropic): docs.claude.com
• Idempotent requests — Stripe Documentation: docs.stripe.com
• Semantic Conventions for GenAI agent spans — OpenTelemetry: opentelemetry.io