Skip to content

< all problems22 · Level 04, Tool Calling

Refuse the Arguments a Model Invented

medium · implement · Tool Calling

You gave the model a schema. It is not obliged to follow it: a string where you declared a number, a missing required field, a parameter you never declared, all arriving as ordinary keyword arguments.

Implement checked_call(registry, name, arguments, schema) returning (ok, value):

  • (False, message) when the arguments do not satisfy the schema
  • (True, result) when they do and the call succeeds

Validate three things against the JSON Schema fragment:

  1. Every name in required is present.
  2. No key appears that is not in properties.
  3. Each value matches its type: "number" accepts int and float, "string" accepts str, "boolean" accepts bool.

The catch: isinstance(True, int) is True in Python, so a naive number check accepts True as a number.

The message goes back to the model, so name the offending field; the tests check for it. "Invalid arguments" tells it nothing.