Tool Response Corruption (Schema Drift + Truncation) + Code

Quick take

• Tool output is the agent’s reality; validate it like untrusted input.
• Enforce size + content-type before parsing; strict parse + schema + invariants.
• Fail closed (or degrade) on mismatch — never “best-effort” parse for writes.
• Track tool_output_invalid as a first-class metric.

Problem-first intro

Your tool returns JSON.

Until it doesn’t.

Maybe a proxy injects HTML. Maybe the response is truncated. Maybe the tool version changed and a field renamed.

The model sees “something” and tries to keep going. That’s how you get:

• wrong decisions
• wrong writes
• and the worst kind of bug: “it didn’t crash, it just did the wrong thing”

Tool response corruption is a production failure because tool output is the agent’s reality. If reality is corrupted, decisions are corrupted.

Why this fails in production

1) Tool output is treated as trusted and well-formed

Most teams validate inputs and ignore outputs. That’s backwards for agents:

• outputs are what the agent uses to decide actions
• outputs are the easiest place for silent corruption

2) Partial responses happen

Timeouts don’t always fail cleanly. Sometimes you get:

• half a JSON document
• a 200 with an error payload
• an empty body with a success status

3) Schema drift is constant

Internal APIs change. Vendors change. Even your own tools change.

If your agent doesn’t validate outputs, you’ll find out from user-facing mistakes.

4) The model tries to “smooth over” corruption

Humans see invalid JSON and stop. Models see “close enough” and fill gaps with hallucinations.

That’s a feature for text generation. It’s a bug for tool-mediated actions.

Implementation example (real code)

The safe pattern:

1. enforce response size limits
2. enforce content-type
3. validate schema + invariants
4. fail closed (or degrade) on mismatch

import json
from typing import Any


class ToolOutputInvalid(RuntimeError):
  pass


def parse_json_strict(raw: str, *, max_chars: int = 200_000) -> Any:
  if len(raw) > max_chars:
      raise ToolOutputInvalid("tool output too large")
  try:
      return json.loads(raw)
  except Exception as e:
      raise ToolOutputInvalid(f"invalid JSON: {type(e).__name__}")


def validate_user_profile(obj: Any) -> dict[str, Any]:
  if not isinstance(obj, dict):
      raise ToolOutputInvalid("expected object")
  if "user_id" not in obj or not isinstance(obj["user_id"], str):
      raise ToolOutputInvalid("missing user_id")
  if "plan" in obj and obj["plan"] not in {"free", "pro", "enterprise"}:
      raise ToolOutputInvalid("invalid plan enum")
  return obj


def get_user_profile(user_id: str) -> dict[str, Any]:
  raw = http_get(f"https://api.internal/users/{user_id}")  # (pseudo)
  obj = parse_json_strict(raw)
  return validate_user_profile(obj)

export class ToolOutputInvalid extends Error {}

export function parseJsonStrict(raw, { maxChars = 200_000 } = {}) {
if (raw.length > maxChars) throw new ToolOutputInvalid("tool output too large");
try {
  return JSON.parse(raw);
} catch (e) {
  throw new ToolOutputInvalid("invalid JSON: " + (e && e.name ? e.name : "Error"));
}
}

export function validateUserProfile(obj) {
if (!obj || typeof obj !== "object") throw new ToolOutputInvalid("expected object");
if (typeof obj.user_id !== "string") throw new ToolOutputInvalid("missing user_id");
if ("plan" in obj && !["free", "pro", "enterprise"].includes(obj.plan)) {
  throw new ToolOutputInvalid("invalid plan enum");
}
return obj;
}

export async function getUserProfile(userId, { httpGet } = {}) {
if (!httpGet) throw new Error("missing httpGet implementation");
const raw = await httpGet("https://api.internal/users/" + encodeURIComponent(userId)); // (pseudo)
const obj = parseJsonStrict(raw);
return validateUserProfile(obj);
}

This is intentionally strict. If the tool output is corrupted, the correct response is usually:

• stop
• return partial results
• and log the failure class

Not: “guess what the tool meant”.

Example incident (numbers are illustrative)

Example: an agent that updated CRM notes based on user profile + recent events.

The user.profile tool started occasionally returning HTML error pages with a 200 status (proxy misconfig). The model “read” the HTML and extracted a “plan = enterprise” string from a random banner.

Impact:

• 23 CRM records tagged with the wrong plan
• sales team wasted ~3 hours chasing “enterprise leads” that weren’t
• we had to run a cleanup job and restore from logs (which were incomplete)

Fix:

1. content-type checks + strict JSON parse
2. schema validation + enum checks
3. fail closed + safe-mode (skip write when profile invalid)
4. metrics: tool_output_invalid rate

Tools lie in boring ways. Your agent has to call them out.

Trade-offs

• Strict validation increases hard failures during tool drift (good: you see it).
• Fail closed reduces success rate short-term, prevents silent corruption.
• Schema maintenance is work. Silent data corruption is worse work.

When NOT to use

• If the tool is already strongly typed end-to-end and you control it fully, you can validate less (still keep size limits).
• If the tool returns free-form text by design, you should wrap it with an extractor that produces structured output.
• If you can’t tolerate stopping, you need fallback tools, not looser validation.

Copy-paste checklist

• [ ] Enforce max response size
• [ ] Check content-type (JSON vs HTML)
• [ ] Strict parse (no “best effort”)
• [ ] Schema validation + enum constraints
• [ ] Invariant checks (ids, counts, ranges)
• [ ] Fail closed or degrade (no writes on invalid output)
• [ ] Metrics/alerts on invalid output rate
• [ ] Log args hash + tool version + error class

Safe default config snippet (JSON/YAML)

validation:
  tool_output:
    fail_closed: true
    max_chars: 200000
    require_content_type: "application/json"
    enforce_enums: true
safe_mode:
  on_invalid_output: "skip_writes"
metrics:
  track: ["tool_output_invalid_rate"]

FAQ (3–5)

Related pages (3–6 links)

• Foundations: How agents use tools · What makes an agent production-ready
• Failure: Prompt injection attacks · Hallucinated sources
• Governance: Tool permissions (allowlists)
• Production stack: Production agent stack