Anthropic Claude Models Emit Invalid Tool Calls with Extra Fields

TL;DR

Newer Anthropic Claude models (Opus 4.8 and Sonnet 5) frequently produce edit‑tool calls that contain extra, invented fields in the edits[] array, causing Pi’s edit tool to reject the call. The regression appears tied to post‑training on Claude Code’s forgiving harness, which tolerates malformed calls and thus reinforces sloppy schemas.

The Symptom: Unexpected Keys in Edit Calls

When invoking Pi’s edit tool, the expected payload looks like:

{
  "path": "some/file.py",
  "edits": [
    {"oldText": "text to replace", "newText": "replacement text"}
  ]
}

In practice, Opus 4.8 and Sonnet 5 sometimes emit objects such as:

{
  "oldText": "...",
  "newText": "...",
  "requireUnique": true
}

or even:

{
  "oldText": "...",
  "newText": "...",
  "oldText2": "",
  "newText2": ""
}

The extra keys (requireUnique, oldText2, type, id, matchCase, …) are completely invented and not part of Pi’s schema. The core oldText/newText values are correct, but the presence of any unknown field causes Pi to reject the call and request a retry.

Why It Happens Only With Newer Models

Older Anthropic models (e.g., Opus 4.5) rarely exhibit this behavior. The regression correlates with the introduction of Claude Code’s post‑training harness, which:

  1. Accepts a wide range of sloppy parameters – aliases (old_str → oldText), Unicode repairs, and silent filtering of unknown keys.
  2. Retries automatically – when a malformed call is detected, the harness rewrites the transcript and asks the model to try again.
  3. Rewards successful task completion – reinforcement learning (RL) treats a call that ultimately results in a correct edit, even if intermediate keys were wrong, as a success.

Because the RL signal does not penalize the extra fields, the model learns a strong prior that some optional field may be present. When presented with a different schema (Pi’s nested edits[]), the model has no learned name for the optional slot and fabricates one, leading to a wide variety of random keys.

Tool Calls Are Plain Text, Not Magic

LLM tool invocation is implemented by embedding a special markup (often called ANTML) into the model’s output stream. A simplified representation looks like:

<antml:function_calls>
  <antml:invoke name="edit">
    <antml:parameter name="path">some/file.py</antml:parameter>
    <antml:parameter name="edits">
      [
        {"oldText":"...","newText":"..."}
      ]
    </antml:parameter>
  </antml:invoke>
</antml:function_calls>

The model generates this markup as plain text; the server parses it and validates the JSON payload. If validation fails, the server returns an error and the model attempts another generation.

Two Approaches to Enforcing Correct JSON

  1. Post‑generation validation – let the model emit any JSON, then reject malformed calls. This is the default behavior of many harnesses, including Pi.
  2. Grammar‑aware (constrained) decoding – mask out tokens that would violate the JSON schema during generation. Anthropic’s strict mode implements this by refusing to sample keys not present in the schema.

Enabling strict mode eliminates the spurious keys, confirming that the issue stems from unconstrained sampling.

Community Workarounds

  • Improved error messages – as noted by a commenter, providing clear guidance on the expected schema lets the model quickly self‑correct on the next turn.
  • Self‑healing extensions – some users patch Pi’s edit tool to strip unknown fields before validation, reducing round‑trips.
  • Alternative tool formats – using curl‑style commands or flat edit schemas (e.g., Claude Code’s file_path, old_string, new_string) sidesteps the nested‑array problem.

Implications for Harness Designers

  1. Schema choice matters – a tool schema that diverges from the model’s post‑training distribution will see higher failure rates.
  2. Closed‑source harnesses hide biases – without visibility into Claude Code’s internal repairs, third‑party developers cannot anticipate which sloppiness the model tolerates.
  3. Strict grammar enforcement is a practical mitigation – enabling strict mode or implementing your own constrained decoder prevents the model from emitting unknown keys.
  4. Future RL fine‑tuning may cement these biases – if providers continue to reward successful task completion despite malformed calls, models will become increasingly locked into the provider’s preferred schema.

Takeaway

The regression of newer Claude models adding spurious fields to tool calls is not a random glitch but a training artifact caused by reinforcement learning on a forgiving harness (Claude Code). Harnesses that rely on alternative schemas must either adopt strict grammar‑constrained decoding or accept that the model will continue to inject invented keys, leading to unnecessary retries and reduced reliability.

Sources

Related