[
    {
        "id": "osp-20913",
        "type": "article-journal",
        "title": "Representation-Aligned Auxiliary Supervision for Language Model Adaptation",
        "author": [
            {
                "family": "Kim",
                "given": "Kyuyoung"
            },
            {
                "family": "Sheng",
                "given": "Peiyao"
            },
            {
                "family": "Hebbar",
                "given": "Ashwin"
            },
            {
                "family": "Xu",
                "given": "Peiyang"
            },
            {
                "family": "Xie",
                "given": "Yunfei"
            },
            {
                "family": "Wang",
                "given": "Kevin"
            },
            {
                "family": "Xin",
                "given": "Rui"
            },
            {
                "family": "Wei",
                "given": "Chen"
            },
            {
                "family": "Wang",
                "given": "Zhangyang"
            },
            {
                "family": "Shin",
                "given": "Jinwoo"
            },
            {
                "family": "Viswanath",
                "given": "Pramod"
            },
            {
                "family": "Oh",
                "given": "Sewoong"
            }
        ],
        "URL": "https://omanscience.com/ar/articles/representation-aligned-auxiliary-supervision-for-language-model-adaptation",
        "language": "en",
        "issued": {
            "date-parts": [
                [
                    2026
                ]
            ]
        },
        "abstract": "Language models exhibit strong reasoning capabilities, yet adapting them to structured domains remains challenging and can yield inconsistent outcomes. We identify representation compatibility, the extent to which a model effectively processes a representation for a structured task, as a key factor in adaptation. We study this in chess, which provides a controlled testbed with precise semantics, computable optimal actions, and multiple state representations, including a symbolic encoding (FEN) and a spatial format (ASCII). We find that models often process semantically equivalent inputs substantially differently, affecting both learning and generalization. Building on this observation, we propose representation-aligned auxiliary supervision, which uses environment-derived tasks expressed in compatible representations to improve adaptation to structured domains. Across models and representations, auxiliary supervision consistently improves optimal-move prediction relative to target-only training under identical target data. Tasks that expose environment dynamics provide larger and most consistent gains than surface-level or static supervision, while remaining competitive with substantially increasing the amount of target-task data. Moreover, ASCII-trained models transfer more effectively to FEN than FEN-trained models do to ASCII, even surpassing the FEN target-only baseline on FEN evaluation. The gains also extend beyond optimal-move prediction to open-ended, factually grounded commentary generation. Overall, our results show that auxiliary supervision in model-compatible representations can enable effective adaptation in structured domains."
    }
]