Abstract
Large language models (LLMs) excel at natural language processing but struggle to interact with external environments. Tool learning provides a promising way to extend LLMs into actionable agents, where tool selection is a critical prerequisite for successful tool use. Existing work often assumes a small or predefined set of tools, leaving large-scale tool selection underexplored. Real-world repositories contain a vast and diverse array of tools, making it difficult for LLMs to effectively search, distinguish, and compose tools under context-length constraints. We identify large-scale tool selection as a new challenge for agentic reinforcement learning, highlighting that existing RL methods for knowledge-based question answering are inadequate for selecting tools while considering compatibility. To address this challenge, we propose ToolSearcher, a novel RL framework for effective multi-turn search and fine-grained optimization in large-scale tool selection. Specifically, we introduce category-constrained tool discrimination to improve the model's ability to distinguish functionally similar tools, event-level search modeling to explicitly optimize the discovery of target tools during multi-turn search, and trajectory-aligned credit allocation to provide fine-grained reward signals for different stages of the search-selection process. Extensive experiments on large-scale tool selection benchmarks demonstrate that ToolSearcher consistently outperforms a set of strong baselines in challenging settings involving iterative search and complex tool composition.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Dai, Z., Song, X., Wang, Z., Niu, T., Liu, J., Wang, W., Tang, X., Wu, S., Yao, C., & Chen, J. (2026). ToolSearcher: Optimizing Tool Selection at Scale via Reinforcement Learning. https://omanscience.com/en/articles/toolsearcher-optimizing-tool-selection-at-scale-via-reinforcement-learning
MLA 9
Dai, Zhenlong, et al. "ToolSearcher: Optimizing Tool Selection at Scale via Reinforcement Learning." https://omanscience.com/en/articles/toolsearcher-optimizing-tool-selection-at-scale-via-reinforcement-learning.
Chicago (author–date)
Dai, Zhenlong, Xujie Song, Zitong Wang, Tong Niu, Jian Liu, Weiqiang Wang, Xiu Tang, Sai Wu, Chang Yao, and Jingyuan Chen. 2026. "ToolSearcher: Optimizing Tool Selection at Scale via Reinforcement Learning." https://omanscience.com/en/articles/toolsearcher-optimizing-tool-selection-at-scale-via-reinforcement-learning.
Harvard
Dai, Z., Song, X., Wang, Z., Niu, T., Liu, J., Wang, W., Tang, X., Wu, S., Yao, C. and Chen, J. (2026) 'ToolSearcher: Optimizing Tool Selection at Scale via Reinforcement Learning', Available at: https://omanscience.com/en/articles/toolsearcher-optimizing-tool-selection-at-scale-via-reinforcement-learning.
Vancouver
Dai Z, Song X, Wang Z, Niu T, Liu J, Wang W, et al. ToolSearcher: Optimizing Tool Selection at Scale via Reinforcement Learning. https://omanscience.com/en/articles/toolsearcher-optimizing-tool-selection-at-scale-via-reinforcement-learning
IEEE
Z. Dai, X. Song, Z. Wang, T. Niu, J. Liu, W. Wang, X. Tang, S. Wu, C. Yao, and J. Chen, "ToolSearcher: Optimizing Tool Selection at Scale via Reinforcement Learning," https://omanscience.com/en/articles/toolsearcher-optimizing-tool-selection-at-scale-via-reinforcement-learning.