Tool-schema vs prompt-only structured output
What you'll learn
If code is going to parse the output, define a tool whose schema is the output shape. Tool-schemas use constrained decoding — the model can't break the format. Prompts can only ask politely.
- Prompt-only structured output is a soft request. Fails ~3–10% silently.
- Tool-schema structured output uses constrained decoding. Schema-controlled fields cannot violate the schema.
- Decision rule: machine-consumed output → tool-schema. Human-consumed output → prompt-only OK.
When you need machine-consumable structured output (JSON, tabular records, schema-conforming objects), the API offers two paths: ask in the prompt ('respond in JSON, no trailing commas') or define a tool whose input schema *is* the desired output shape, then have the model 'call' that tool. They look similar but have very different reliability profiles.
Prompt-only structured output is a soft constraint. The model is being *asked* to produce JSON; nothing in the sampling process prevents it from emitting an extra comma, an unescaped quote, or extra prose around the JSON. It works ~90–97% of the time and fails silently the rest. For one-off use that's tolerable; for any pipeline that parses thousands of responses, the failure rate compounds — every percent is real bugs.
Tool-use with a schema is a hard constraint via *constrained decoding*. The model literally cannot emit tokens that violate the schema for the schema-controlled fields. Trailing commas, missing keys, type mismatches — these are eliminated structurally, not asked-for politely. The trade-off is API-side complexity (define the tool, parse the tool_use block) but the parse-failure rate effectively goes to zero for the schema-conforming portion.
The decision rule: if the output is consumed by code (parser, downstream service, database write), use the tool-schema. If the output is consumed by a human reading a chat reply, the prompt-only path is fine. The exam frames this exactly: 'occasional malformed JSON breaks downstream parsing — what is the most reliable fix?' The right answer is structured output via tool use, not stricter prompt wording.
Key points
- Prompt-only structured output is a soft request. Fails ~3–10% silently.
- Tool-schema structured output uses constrained decoding. Schema-controlled fields cannot violate the schema.
- Decision rule: machine-consumed output → tool-schema. Human-consumed output → prompt-only OK.
- The reliability gap is structural, not a prompt-quality issue.
Examples
system: 'Return JSON: {name, email, score}.' user: '<record>' → 1-in-30 responses has a trailing comma or unescaped quote that breaks JSON.parse. The team adds 'no trailing commas, valid JSON only' — failure rate doesn't move materially. The fix is structural.
tools: [{name: 'submit_record', input_schema: {type: 'object', required: ['name','email','score'], properties: {...}}}] user: '<record>' → model emits tool_use(submit_record, {name, email, score}) with schema-validated fields. Parse the tool_use input directly; failure rate effectively zero.
Pitfalls
- Iterating prompt wording to fix the last 3% of malformed output. The 3% is structural; you're chasing a tail you can't reach with prompts.
- Wrapping prompt-only output in retry-with-validation logic. Works, but is more code than just defining a tool-schema.
- Using tool-schema and then ignoring the tool_use block, parsing free-text instead. You've added complexity for no benefit.
- Forgetting that constrained decoding only applies to the schema-controlled fields. Free-text inside a string field is still free text.
Source notes: 00-academy-basics/notes/04-claude-api.md
Ask Claude
Build a prompt with this lesson + your question, copy it, and open the Claude Project in a new tab.