Abstract
LLM agents for coding, search, and workplace tasks increasingly rely on long-context capabilities to effectively aggregate and reason over extended interaction histories. Recent work has incorporated agent trajectories into mid-training stage, drawing on their naturally long and interaction-rich structure. Yet how to organize the information within these trajectories into effective mid-training supervision remains underexplored. In this work, we investigate the relationship between long-context and agent atomic capabilities and introduce TrajLong, a novel framework that compiles trajectories into long-context training tasks with dense supervision, targeting three representative atomic capabilities: evidence grounding, cross-evidence aggregation, and temporal state maintenance. We mid-train Qwen3-14B-Base and Qwen3-30B-A3B-Base with data compiled by TrajLong, followed by supervised fine-tuning. Experiments on 6 long-context and 12 agent benchmarks demonstrate broad performance gains, with controlled ablations showing improvements over raw and masked trajectory baselines. Capability-level analyses further reveal task-dependent associations between long-context and agent atomic capabilities. These findings suggest that the shared capability demands of long-context reasoning and agent execution provide a principled basis for designing mid-training data to develop downstream agent capabilities.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Peng, M., Zhang, Q., Chen, N., Li, Y., Yan, G., Gu, X., Wu, H., Wang, H., Huang, L., Zhang, W., & Li, J. (2026). TrajLong: Co-Designing Agentic and Long-Context Supervision for Mid-Training. https://omanscience.com/en/articles/trajlong-co-designing-agentic-and-long-context-supervision-for-mid-training
MLA 9
Peng, Miao, et al. "TrajLong: Co-Designing Agentic and Long-Context Supervision for Mid-Training." https://omanscience.com/en/articles/trajlong-co-designing-agentic-and-long-context-supervision-for-mid-training.
Chicago (author–date)
Peng, Miao, Qintong Zhang, Nuo Chen, Yuhan Li, Guochen Yan, Xinran Gu, Hongqiu Wu, Hai Wang, Lydell Huang, Wentao Zhang, and Jia Li. 2026. "TrajLong: Co-Designing Agentic and Long-Context Supervision for Mid-Training." https://omanscience.com/en/articles/trajlong-co-designing-agentic-and-long-context-supervision-for-mid-training.
Harvard
Peng, M., Zhang, Q., Chen, N., Li, Y., Yan, G., Gu, X., Wu, H., Wang, H., Huang, L., Zhang, W. and Li, J. (2026) 'TrajLong: Co-Designing Agentic and Long-Context Supervision for Mid-Training', Available at: https://omanscience.com/en/articles/trajlong-co-designing-agentic-and-long-context-supervision-for-mid-training.
Vancouver
Peng M, Zhang Q, Chen N, Li Y, Yan G, Gu X, et al. TrajLong: Co-Designing Agentic and Long-Context Supervision for Mid-Training. https://omanscience.com/en/articles/trajlong-co-designing-agentic-and-long-context-supervision-for-mid-training
IEEE
M. Peng, Q. Zhang, N. Chen, Y. Li, G. Yan, X. Gu, H. Wu, H. Wang, L. Huang, W. Zhang, and J. Li, "TrajLong: Co-Designing Agentic and Long-Context Supervision for Mid-Training," https://omanscience.com/en/articles/trajlong-co-designing-agentic-and-long-context-supervision-for-mid-training.