Abstract
Robot agents must connect their intended actions to observed outcomes while retaining the context needed to revise their choices over repeated attempts. Existing interfaces often leave these choices inside predefined tools or require agents to manage detailed execution code and its growing history. We introduce RobotUse, a robot agent harness that organizes computation, context, and decisions around specifying and revising physical actions. Agents visually select targets and poses, while the backend handles geometry, motion planning, and control. Subagents retain detailed interactions within each subgoal and return the information needed for subsequent decisions. Continual harnessing lets agents learn from execution by updating a persistent playbook. On RoboLab, RobotUse achieves 45% task success, outperforming CaP-X by 6.7 percentage points while maintaining compact decision contexts and reducing reliance on predefined action abstractions. Furthermore, we show that RobotUse learns from real-world execution despite imperfect feedback and transfers what it learns to subsequent tasks. Project page is available at https://robotuse-team.github.io/.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Lee, J., Baek, I., Kim, S., Jeon, S., Kim, M., Kim, B., & Kwak, N. (2026). RobotUse: Allocating Computation, Context, and Decisions. https://omanscience.com/en/articles/robotuse-allocating-computation-context-and-decisions
MLA 9
Lee, Junhoo, et al. "RobotUse: Allocating Computation, Context, and Decisions." https://omanscience.com/en/articles/robotuse-allocating-computation-context-and-decisions.
Chicago (author–date)
Lee, Junhoo, Injun Baek, Seungyeon Kim, Suhyun Jeon, Minkyu Kim, Baekseung Kim, and Nojun Kwak. 2026. "RobotUse: Allocating Computation, Context, and Decisions." https://omanscience.com/en/articles/robotuse-allocating-computation-context-and-decisions.
Harvard
Lee, J., Baek, I., Kim, S., Jeon, S., Kim, M., Kim, B. and Kwak, N. (2026) 'RobotUse: Allocating Computation, Context, and Decisions', Available at: https://omanscience.com/en/articles/robotuse-allocating-computation-context-and-decisions.
Vancouver
Lee J, Baek I, Kim S, Jeon S, Kim M, Kim B, et al. RobotUse: Allocating Computation, Context, and Decisions. https://omanscience.com/en/articles/robotuse-allocating-computation-context-and-decisions
IEEE
J. Lee, I. Baek, S. Kim, S. Jeon, M. Kim, B. Kim, and N. Kwak, "RobotUse: Allocating Computation, Context, and Decisions," https://omanscience.com/en/articles/robotuse-allocating-computation-context-and-decisions.