Abstract
Deploying large language models in production often requires constraining outputs to structured formats such as JSON or XML, and prior work treats the resulting accuracy loss as an inherent `structure tax'. We re-examine this claim by evaluating a battery of models, datasets and schemas, measuring task accuracy, confidence calibration, and hidden-state geometry. The tax turns out to depend on schema design rather than on structure per se: reasoning-first field ordering matches or exceeds free-form accuracy, while answer-first ordering causes steep drops, particularly in smaller models. Format sensitivity scales inversely with a task's own structural constraints, and schemas that preserve reasoning order also improve calibration with CKA showing greater separability between correct and incorrect representations in middle transformer layers. Our findings indicate that properly designed structured formats can match or exceed free-form performance, reframing the critical question from `whether to structure' to `how to structure' for optimal reasoning preservation.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Kumar, V., Kanishka, & Mandora, B. (2026). Structure Tax: How Structured Output affects LLMs Performance. https://omanscience.com/en/articles/structure-tax-how-structured-output-affects-llms-performance
MLA 9
Kumar, Vineet, et al. "Structure Tax: How Structured Output affects LLMs Performance." https://omanscience.com/en/articles/structure-tax-how-structured-output-affects-llms-performance.
Chicago (author–date)
Kumar, Vineet, Kanishka, and Bhuvanesh Mandora. 2026. "Structure Tax: How Structured Output affects LLMs Performance." https://omanscience.com/en/articles/structure-tax-how-structured-output-affects-llms-performance.
Harvard
Kumar, V., Kanishka and Mandora, B. (2026) 'Structure Tax: How Structured Output affects LLMs Performance', Available at: https://omanscience.com/en/articles/structure-tax-how-structured-output-affects-llms-performance.
Vancouver
Kumar V, Kanishka, Mandora B. Structure Tax: How Structured Output affects LLMs Performance. https://omanscience.com/en/articles/structure-tax-how-structured-output-affects-llms-performance
IEEE
V. Kumar, Kanishka, and B. Mandora, "Structure Tax: How Structured Output affects LLMs Performance," https://omanscience.com/en/articles/structure-tax-how-structured-output-affects-llms-performance.