Validate LLM Tool Calls With JSON Schema and Business Rules

Structured tool calling can constrain output shape, but a syntactically valid object can still request an unauthorized refund, a nonexistent project, or a destructive default. Treat every model-produced argument as untrusted input and place a deterministic policy layer between generation and execution.

The practical model

Treat validate llm tool calls with json schema and business rules as a system with four review points: make the schema narrow, validate beyond json schema, separate planning from commitment, handle repair safely. Give each point an owner, a measurable acceptance condition, and a rollback trigger. Thresholds should come from observed workloads, contracts, or risk tolerance rather than an unexplained default.

Make the schema narrow

Use required properties, explicit types, enums, numeric bounds, string formats, and additionalProperties false. Prefer separate tools for materially different actions instead of a single action field with dozens of optional parameters. Describe units and time zones in field descriptions and names.

Validate beyond JSON Schema

Resolve IDs within the authenticated tenant, verify object state, enforce role and monetary limits, and calculate authoritative values server-side. A model should select a customer or quantity, not supply a final price, permission, or ownership claim. Normalize dates and URLs before policy evaluation.

Separate planning from commitment

For impactful actions, first return a preview containing the resolved target, computed effects, and an opaque confirmation token. Require explicit user approval before the commit endpoint. Bind the token to the user, exact arguments, policy version, and short expiry so it cannot approve a modified request.

Handle repair safely

Return compact machine-readable validation errors to the model and cap repair attempts. Never relax a constraint because generation repeatedly fails. Log schema version, validation outcome, policy decision, tool latency, and redacted result; do not log secrets hidden in arguments.

Common pitfalls

• Executing unknown fields that the deserializer ignored.

• Trusting a model-supplied tenant ID.

• Using a default true value for destructive options.

• Retrying a non-idempotent tool after an ambiguous timeout.

A focused test plan

1. Make the schema narrow: verify that use required properties, explicit types, enums, numeric bounds, string formats, and additionalProperties false. Then introduce the failure case “executing unknown fields that the deserializer ignored” and confirm that the system fails visibly, stays within its safety boundary, and has a documented recovery action.

1. Validate beyond JSON Schema: verify that resolve IDs within the authenticated tenant, verify object state, enforce role and monetary limits, and calculate authoritative values server-side. Then introduce the failure case “trusting a model-supplied tenant id” and confirm that the system fails visibly, stays within its safety boundary, and has a documented recovery action.

1. Separate planning from commitment: verify that for impactful actions, first return a preview containing the resolved target, computed effects, and an opaque confirmation token. Then introduce the failure case “using a default true value for destructive options” and confirm that the system fails visibly, stays within its safety boundary, and has a documented recovery action.

1. Handle repair safely: verify that return compact machine-readable validation errors to the model and cap repair attempts. Then introduce the failure case “retrying a non-idempotent tool after an ambiguous timeout” and confirm that the system fails visibly, stays within its safety boundary, and has a documented recovery action.

Primary references

• JSON Schema documentation

Conclusion

Tool safety is deterministic software around a probabilistic planner. Constrain shape, re-derive authority and prices, preview consequential actions, and make execution idempotent.