How to build an agent that survives production

A chatbot has a prompt. An agent has a surface of actions, and that surface is the most expensive design decision in the project. The customer channel in PÍDELO has eight tools and none of them is called handleRequest: prepareOrder, createOrder, sendMenuImage, sendLocation, sendSchedule, sendWelcomeMessage, fetchKnowledgePackage, escalateToHumanAgent. The name is documentation the model actually reads. If I cannot say in five words what a tool does, the model will not know when to call it either.

Bounding matters as much as naming. The admin channel is a second agent with three read-only tools—live orders, business report, financial breakdown—and one written prohibition: no orders are taken there. Splitting them is not architectural purism. The worst case of a reporting agent is an incomplete number; the worst case of an agent that writes orders is an order nobody placed. Those two risks do not belong in the same place.

In Ch4t.ai Finance the catalog is larger: 18 agent tools for creating transactions, transfers, categories, budgets, investments, savings goals, debts and receivables, plus the listings, the reports, and the daily snapshot. That is why the catalog itself became context. A custom MCP server exposes 25 tools with hybrid authentication and preloads the accounts, categories, and debts into the tool descriptions. The model does not guess which account exists: it reads it where it is already looking.

Everything leaving the loop is validated with Zod, and the knowledge base is never queried with free text: the agent picks a topic from a closed list. It is the same idea three times over. The narrower the door, the fewer ways there are for the model to invent something on the way through, and the easier it is to read back later, cold, exactly what it did and on what data.

PÍDELO's tool-calling loop uses the Vercel AI SDK, with generateText and toolChoice on auto. The interesting part is not that it calls tools: it is that the stop is declared. stopWhen combines a step ceiling with a terminal-tool condition, so the loop ends for one of two reasons I wrote down, not because the model ran out of things to say. An agent with no stop condition is not an agent. It is an open invoice.

Stopping is a product decision too. In Finance, when a write tool succeeds, the loop halts right there by contract and the turn goes back to the person for confirmation. It is slower and it is correct: nobody wants an agent chaining three money movements in a single turn because the message was ambiguous. Speed can be won back elsewhere. A wrong entry in someone's books cannot.

What the agent decides depends on what it reads, so the prompt cannot be a string buried in the code. In Finance the system prompt is built in five layers: platform rules, persona by role, execution context, response format, and tool catalog. In PÍDELO the prompts live in a versioned CMS—draft, published, archived—with variants and dynamic variables. Changing the tone of one business stops being a deploy, and every publication is dated, which is what makes it possible to argue later about why the agent answered the way it did.

Before a message goes out to the customer it goes through quality passes. If the text comes back empty, it gets synthesized. If it carries markup or words written in another alphabet, it gets corrected. If the tool flow fails, a tool-free recovery at least leaves the customer with a written answer instead of silence. None of the three is elegant. All three exist because the channel is WhatsApp and the person on the other side is hungry.

The conversation is stored turn by turn in DynamoDB, tool calls and their results included, and rehydrated between retries. That part is always underrated: if the retry cannot see what the previous attempt already did, the agent repeats actions that have effects in the world. Memory is not there to make the agent look attentive. It is there so it does not create the same order twice.

Providers go down as well. Every model has its own circuit breaker in Redis, driven by atomic Lua scripts: closed, open, half-open, with a recovery probe and exponential decay on the cooldown. When one opens, the multi-provider fallback chain carries the traffic to the next one. The atomic part matters: several workers compete for the same state, and a breaker that can be read half-written is worse than no breaker at all.

escalateToHumanAgent is a first-class tool, not a fallback. It carries four reasons and no others: the customer asks for a human, the customer repeats a complaint, frustration is evident, or the request is out of scope. Closing that list does two things at once. The model knows exactly when it is allowed to give up, and the business can measure why it gives up. An agent that cannot let go is not autonomous. It is stubborn.

Cost is a requirement, not an end-of-month surprise. Workers share a budget through a double bucket, requests per minute and tokens per minute, so a spike at one business does not eat the capacity of the rest. That gets decided before the first prompt is written, because afterward you are making it with live customers on the line. The useful question is not what one message costs. It is what the worst day of the month can cost.

Latency is handled with architecture, not with hope. The path is Meta webhook, SQS, BullMQ, worker, model, Meta API, WebSocket, with distributed locks, deduplication, and a fairness gate between chatbots. Voice notes are transcribed with Whisper and enter through that same path. The customer notices none of it, and that is precisely the point: the only thing they should notice is a fast, correct answer.

All of this holds only if it can be watched. Tracing goes to Langfuse over OpenTelemetry, with separate tracers per channel so the admin agent does not muddy the reading of the customer agent, and on top of that there are custom metrics for cost, tokens, and errors per tool. When somebody asks why the bot said that at eleven at night, the answer is not a theory. It is a trace.