الملخص
Language models exhibit strong reasoning capabilities, yet adapting them to structured domains remains challenging and can yield inconsistent outcomes. We identify representation compatibility, the extent to which a model effectively processes a representation for a structured task, as a key factor in adaptation. We study this in chess, which provides a controlled testbed with precise semantics, computable optimal actions, and multiple state representations, including a symbolic encoding (FEN) and a spatial format (ASCII). We find that models often process semantically equivalent inputs substantially differently, affecting both learning and generalization. Building on this observation, we propose representation-aligned auxiliary supervision, which uses environment-derived tasks expressed in compatible representations to improve adaptation to structured domains. Across models and representations, auxiliary supervision consistently improves optimal-move prediction relative to target-only training under identical target data. Tasks that expose environment dynamics provide larger and most consistent gains than surface-level or static supervision, while remaining competitive with substantially increasing the amount of target-task data. Moreover, ASCII-trained models transfer more effectively to FEN than FEN-trained models do to ASCII, even surpassing the FEN target-only baseline on FEN evaluation. The gains also extend beyond optimal-move prediction to open-ended, factually grounded commentary generation. Overall, our results show that auxiliary supervision in model-compatible representations can enable effective adaptation in structured domains.
الكلمات المفتاحية
بيانات النشر
- المجلة
- غير متاح
- وصول مفتوح
- وصول مفتوح أخضر
اقتبس هذه المقالة
APA 7
Kim, K., Sheng, P., Hebbar, A., Xu, P., Xie, Y., Wang, K., Xin, R., Wei, C., Wang, Z., Shin, J., Viswanath, P., & Oh, S. (2026). Representation-Aligned Auxiliary Supervision for Language Model Adaptation. https://omanscience.com/ar/articles/representation-aligned-auxiliary-supervision-for-language-model-adaptation
MLA 9
Kim, Kyuyoung, et al. "Representation-Aligned Auxiliary Supervision for Language Model Adaptation." https://omanscience.com/ar/articles/representation-aligned-auxiliary-supervision-for-language-model-adaptation.
شيكاغو (المؤلف–التاريخ)
Kim, Kyuyoung, Peiyao Sheng, Ashwin Hebbar, Peiyang Xu, Yunfei Xie, Kevin Wang, Rui Xin, Chen Wei, Zhangyang Wang, Jinwoo Shin, Pramod Viswanath, and Sewoong Oh. 2026. "Representation-Aligned Auxiliary Supervision for Language Model Adaptation." https://omanscience.com/ar/articles/representation-aligned-auxiliary-supervision-for-language-model-adaptation.
هارفارد
Kim, K., Sheng, P., Hebbar, A., Xu, P., Xie, Y., Wang, K., Xin, R., Wei, C., Wang, Z., Shin, J., Viswanath, P. and Oh, S. (2026) 'Representation-Aligned Auxiliary Supervision for Language Model Adaptation', Available at: https://omanscience.com/ar/articles/representation-aligned-auxiliary-supervision-for-language-model-adaptation.
فانكوفر
Kim K, Sheng P, Hebbar A, Xu P, Xie Y, Wang K, et al. Representation-Aligned Auxiliary Supervision for Language Model Adaptation. https://omanscience.com/ar/articles/representation-aligned-auxiliary-supervision-for-language-model-adaptation
IEEE
K. Kim, P. Sheng, A. Hebbar, P. Xu, Y. Xie, K. Wang, R. Xin, C. Wei, Z. Wang, J. Shin, P. Viswanath, and S. Oh, "Representation-Aligned Auxiliary Supervision for Language Model Adaptation," https://omanscience.com/ar/articles/representation-aligned-auxiliary-supervision-for-language-model-adaptation.