Lecture: Designing a Production Agent Harness with the Claude Messages API
A complete walkthrough of the seven-component application-owned agent harness: tool schemas, ID pairing, parallel tool calls, timeout recovery, budget guards, JSONL logging, and truthful partial answers.
Most tutorials teach tool use in 20 lines of code: define a toy weather function, call messages.create in a while loop, and print the final string. That works for a five-minute demo, and it breaks immediately in production. Real tools time out, models occasionally emit malformed arguments or multiple parallel tool calls, and unbounded loops can burn through an API budget in seconds.
The solution is not a heavy black-box agent framework that hides every prompt and retry behind layers of abstraction. It is a clean, application-owned agent harness—roughly 200 to 300 lines of well-structured Python or TypeScript that you control completely. In this lecture, we design that harness piece by piece, from tool schema contracts to fault-injection tests.
The 7 Components of an Application-Owned Harness
Every reliable agent runtime—whether it powers a coding assistant, a research bot, or a customer-support workflow—separates concerns into seven small, testable modules:
- Model Adapter: Wraps the official SDK (
anthropic.Anthropic), pins model IDs andmax_tokens, catches transient SDK errors (RateLimitError,APIStatusError), and normalizesusageandstop_reasontelemetry. - Run State & Transcript: Owns the
messageslist, enforces strict turn alternation andtool_use_idpairing, and records step-by-step trace events. - Tool Registry & Policy: Defines JSON schemas for the model and maps allowlisted tool names to local Python callables with strict input validation.
- Executor: Runs allowlisted tools with wall-clock timeouts, catches exceptions, truncates oversized payloads, and formats structured
tool_resultblocks. - Budget Guard: Tracks cumulative iterations, elapsed wall-clock seconds, and estimated USD spend—halting tool execution before limits are breached.
- Persistence: Saves structured JSONL call logs, run transcripts, and scoped user memory safely to disk or database.
- Trace & Eval Hooks: Captures deterministic assertions and latency/cost metrics so every production failure can become a regression test.
1. Defining Strict Tool Schemas
Your tool schema is the API contract between your application and the model. Every field in input_schema serves two audiences: the description guides the model's reasoning, while the JSON Schema types and constraints guide your own validator. Keep descriptions concise and behavioral—state what the tool does, what it returns, and whether it is read-only.
SEARCH_CORPUS_SCHEMA = {
"name": "search_corpus",
"description": (
"Search the indexed documentation corpus for passages matching a query. "
"Returns file paths, line ranges, and text snippets. Read-only."
),
"input_schema": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "Concise search keywords or exact symbol name."
},
"max_hits": {
"type": "integer",
"description": "Maximum passages to return (1 to 10). Defaults to 5.",
"minimum": 1,
"maximum": 10
}
},
"required": ["query"],
"additionalProperties": False
}
}Notice "additionalProperties": False and explicit bounds (minimum: 1, maximum: 10). Even with those schema hints, never assume the model respected them. JSON Schema in a tool definition is guidance to the model, not runtime enforcement. Always validate and clamp inputs inside your application code before touching disk, network, or databases.
2. Application-Side Allowlists and Input Validation
Treat every tool_use block returned by the model as untrusted external input. A model can hallucinate a tool name, omit a required field, pass a boolean where an integer was expected, or pass max_hits: 50000. Never dynamically dispatch function names from model output using getattr, eval, or unguarded dictionary lookups on module globals.
from typing import Callable
def validate_search_corpus(raw_input: dict) -> dict:
if not isinstance(raw_input, dict):
raise ValueError("Tool input must be a JSON object")
query = raw_input.get("query")
if not isinstance(query, str) or not query.strip():
raise ValueError("'query' must be a non-empty string")
if len(query) > 500:
raise ValueError("'query' exceeds 500 character limit")
raw_max = raw_input.get("max_hits", 5)
# Note: in Python, isinstance(True, int) is True, so check bool explicitly!
if not isinstance(raw_max, int) or isinstance(raw_max, bool):
raise ValueError("'max_hits' must be an integer")
clamped_hits = max(1, min(raw_max, 10))
return {"query": query.strip(), "max_hits": clamped_hits}
TOOL_REGISTRY: dict[str, tuple[Callable, Callable]] = {
"search_corpus": (validate_search_corpus, run_search_corpus),
}If a tool name is not in TOOL_REGISTRY, do not raise an unhandled exception that crashes the server. Return an error tool_result telling the model that the requested tool is unavailable and listing only the allowed tools so it can self-correct on the next turn.
3. Handling Sequential and Parallel Tool Calls
In a single assistant turn, Claude may emit one tool_use block or several parallel tool_use blocks (for example, searching two different topics or reading three files at once). Your harness must handle both sequential multi-turn chains and parallel single-turn batches cleanly:
- Iterate over every block in
response.contentwhereblock.type == "tool_use". - Collect one
tool_resultobject perblock.id, preserving the exacttool_use_id. - Append all
tool_resultblocks together inside oneusermessage—never send separateusermessages for each parallel tool result, because consecutiveuserturns violate the Messages API alternation rule. - Never drop a
tool_use_ideven if an earlier parallel tool failed; every ID in the assistant turn requires a matching result in the next user turn.
tool_results = []
for block in response.content:
if block.type != "tool_use":
continue
result_payload = execute_with_policy(
name=block.name,
raw_input=block.input,
timeout_sec=5.0,
)
tool_results.append({
"type": "tool_result",
"tool_use_id": block.id,
"content": result_payload["content"],
"is_error": result_payload["is_error"],
})
messages.append({"role": "user", "content": tool_results})4. Timeouts, Exceptions, and Untrusted Tool Outputs
External APIs hang, databases lock, and files go missing. If a tool raises an exception or exceeds its per-call timeout, catch it inside the executor and return a structured error with "is_error": True. That turns a fatal crash into recoverable feedback for the model.
import concurrent.futures
import json
def execute_with_policy(name: str, raw_input: dict, timeout_sec: float = 5.0) -> dict:
if name not in TOOL_REGISTRY:
return {
"is_error": True,
"content": f"Denied: tool '{name}' is not in the allowlist."
}
validator, fn = TOOL_REGISTRY[name]
try:
clean_input = validator(raw_input)
except Exception as exc:
return {"is_error": True, "content": f"Invalid arguments: {exc}"}
with concurrent.futures.ThreadPoolExecutor(max_workers=1) as pool:
future = pool.submit(fn, **clean_input)
try:
output = future.result(timeout=timeout_sec)
serialized = json.dumps(output, ensure_ascii=False)
return {"is_error": False, "content": serialized[:12000]}
except concurrent.futures.TimeoutError:
return {
"is_error": True,
"content": f"Tool '{name}' timed out after {timeout_sec}s. Narrow your query or proceed without it."
}
except Exception as exc:
return {"is_error": True, "content": f"Tool '{name}' failed: {exc}"}Notice two subtle details here: first, capping serialized output at 12,000 characters prevents a single noisy tool call from blowing up the context window; second, returning JSON-encoded output reinforces to the model that tool responses are structured data—not instructions to obey.
5. Handling SDK Errors and Structured JSONL Call Logging
Tool failures aren't the only thing that can go wrong; the model API itself can return rate limits or transient 5xx responses. Wrap client.messages.create in explicit exception handlers for anthropic.AuthenticationError (fail fast—never retry bad keys), anthropic.RateLimitError (bounded exponential backoff with jitter), and anthropic.APIStatusError (log HTTP status and request ID).
On every single API call—success or failure—write one structured JSONL record to your telemetry log. Keeping one line per call makes post-mortem debugging, latency percentiles, and cost audits trivial with jq or DuckDB:
import json, time
from datetime import datetime, timezone
def log_jsonl_event(log_path: str, run_id: str, iteration: int, response, latency_ms: float, call_usd: float) -> None:
record = {
"ts": datetime.now(timezone.utc).isoformat(),
"run_id": run_id,
"iteration": iteration,
"model": response.model,
"stop_reason": response.stop_reason,
"input_tokens": response.usage.input_tokens,
"output_tokens": response.usage.output_tokens,
"latency_ms": round(latency_ms, 1),
"call_usd": round(call_usd, 6),
}
with open(log_path, "a", encoding="utf-8") as f:
f.write(json.dumps(record) + "
")6. Application Budgets vs. max_tokens
Many developers mistakenly treat max_tokens as a run budget. In reality, max_tokens only caps the length of a single assistant response—if a loop runs 25 iterations, you still pay for 25 full turns of growing input context. Worse, if max_tokens is set too low and the model hits stop_reason == "max_tokens" in the middle of emitting a tool_use JSON block, that block may be incomplete or truncated.
A production harness enforces three independent application-owned caps:
- Iteration cap (
max_iterations): Hard ceiling on loop turns (for example, 6–10 turns). - Wall-clock cap (
max_wall_seconds): Prevents slow retries from holding HTTP connections open past gateway timeouts. - Cumulative USD spend cap (
max_spend_usd): Computed after every API response usingresponse.usage.input_tokens,output_tokens, and cache read/write token counts against your dated model pricing table.
Emitting Truthful Partials When a Cap Hits
When a budget cap or max_tokens truncation triggers mid-loop, never silently return an empty string or pretend the task finished. Instead, stop exposing tools (omit tools on one final synthesis call if spend allows, or return a deterministic partial summary) and instruct the model to report only what was verified so far and explicitly list what remains unchecked.
def finalize_partial(client, system_prompt: str, messages: list[dict], reason: str, total_usd: float) -> dict:
# Do not pass tools on the finalization step so the model cannot loop further
closing_note = {
"role": "user",
"content": (
f"System limit reached ({reason}). Do not call any tools. "
"Summarize only what you verified from completed tool outputs, "
"and clearly state what remains unfinished."
),
}
# If the last assistant message had unfulfilled tool_use blocks, strip or resolve them first
safe_history = strip_unanswered_tool_use(messages) + [closing_note]
final_resp = client.messages.create(
model="claude-sonnet-4-20250514",
max_tokens=512,
system=system_prompt,
messages=safe_history,
)
return {
"status": "partial",
"reason": reason,
"text": extract_text(final_resp),
"spend_usd": round(total_usd, 6),
}Notice strip_unanswered_tool_use(messages): if you hit a budget cap right after the model emitted a tool_use block (before you executed it), you cannot append a plain user message without either supplying a tool_result or stripping the unanswered tool_use block—otherwise the API will reject the request with a 400 error.
7. Fault-Injection Testing Before You Ship
Before deploying an agent harness, run deterministic fault-injection tests against it using a mocked model adapter—no live API key required:
| Fault Scenario | Simulated Condition | Expected Harness Behavior |
|---|---|---|
| Denied tool name | Model emits tool_use for run_shell |
Executor returns is_error: true; no exception raised |
| Oversized parameter | Model passes max_hits: 9999 |
Validator clamps to 10 or rejects cleanly |
| Tool timeout | Stubbed tool sleeps past timeout_sec |
Executor catches TimeoutError, returns is_error: true |
Mid-tool max_tokens |
Response stops with stop_reason == "max_tokens" |
Harness stops looping and emits a truthful partial |
| Spend cap exceeded | Cumulative cost crosses max_spend_usd |
Tools disabled; final response marked status: "partial" |
Once your harness passes these five fault-injection checks, swapping models, adding new tools, or wiring up MCP servers becomes routine engineering instead of guesswork.
8. Streaming Responses Without Breaking Tool Execution
Users expect tokens to stream into the UI immediately, especially when an agent is synthesizing a long answer after several tool calls. When you switch from client.messages.create() to client.messages.stream(), keep one strict rule in mind: never execute a tool call from partial JSON deltas.
During a streamed turn, the model emits content_block_start for a tool_use block, followed by incremental input_json_delta chunks, and finally content_block_stop. Only after the full stream completes (stream.get_final_message()) should your harness inspect response.stop_reason and dispatch validated tool calls. While the model is streaming text blocks, forward those text deltas straight to your UI; while it is streaming tool_use input JSON, show a clean status indicator (such as "Searching corpus...") instead of raw JSON fragments.
with client.messages.stream(
model=budget["model"],
max_tokens=budget["max_tokens"],
system=system_prompt,
tools=[SEARCH_CORPUS_SCHEMA],
messages=messages,
) as stream:
for text_chunk in stream.text_stream:
emit_ui_token(text_chunk)
response = stream.get_final_message()This gives you sub-second time-to-first-token for final answers while preserving identical validation, ID pairing, and spend tracking on the completed message object.
Harness Deployment Checklist
Before promoting a new agent workflow to production traffic, confirm every item below is enforced in code—not just in system prompt instructions:
- Allowlist enforced at dispatch: Unknown tool names return
is_error: truewithout raising. - Arguments validated and clamped: Types, string lengths, and pagination caps are checked in Python/TypeScript.
- Per-tool timeouts active: Slow external APIs fail cleanly within 5–10 seconds.
- All three budgets wired: Iteration count, wall-clock seconds, and cumulative USD spend are tracked per run.
- Structured JSONL logs enabled: Every call logs model, tokens, latency, stop reason, and cost.
- Truthful partial fallback tested: Hitting any cap disables tools and reports verified vs. unfinished work.
Once these six checks are automated in your test suite, your agent harness becomes a predictable, observable piece of backend infrastructure. You can swap underlying Claude models as new versions release, extend the tool registry with new capabilities, or connect external MCP servers without worrying about silent regressions, runaway token bills, or unhandled tool exceptions taking down your worker processes.