Abstract
Existing reactive Graphical User Interface (GUI) agents often fail in long-horizon, dynamic scenarios, where unexpected disturbances trigger attention-diverting and cascading failures. To address this, we propose PrecogUI, a pre-cognitive architecture that shifts the paradigm from reactive execution to proactive decision-making. Specifically, we design a Proactive Experience Pool (PEP), which caches recurring anomaly and success patterns as "state-action-result" tuples in a dual-memory repository. Furthermore, we introduce a Proactive Simulation Executor (PSE) that learns to forecast the next symbolic UI layout given a candidate action, enabling early anomaly avoidance and ranking candidate actions by predicted reliability. Finally, a Pre-cognitive Execution Controller (PEC) fuses these priors and predictions, prioritizes handling of foreseen anomalies, and ensures execution robustness through a closed-loop error correction mechanism. For robust evaluation, we develop AutoTraj, an automatic data-generation engine, to construct InterfereBench, a benchmark for long-horizon tasks with strong disturbances. Experiments demonstrate that PrecogUI surpasses state-of-the-art methods on InterfereBench while maintaining competitive performance on public benchmarks. The code will be publicly available.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Kang, B., Ouyang, J., Jiang, L., Chen, B., & Tian, Z. (2026). PrecogUI: Proactive GUI Agents via Pre-cognitive Simulation and Experience Retrieval. https://omanscience.com/en/articles/precogui-proactive-gui-agents-via-pre-cognitive-simulation-and-experience-retrieval
MLA 9
Kang, Bin, et al. "PrecogUI: Proactive GUI Agents via Pre-cognitive Simulation and Experience Retrieval." https://omanscience.com/en/articles/precogui-proactive-gui-agents-via-pre-cognitive-simulation-and-experience-retrieval.
Chicago (author–date)
Kang, Bin, Jiarui Ouyang, Li Jiang, Bin Chen, and Zhuotao Tian. 2026. "PrecogUI: Proactive GUI Agents via Pre-cognitive Simulation and Experience Retrieval." https://omanscience.com/en/articles/precogui-proactive-gui-agents-via-pre-cognitive-simulation-and-experience-retrieval.
Harvard
Kang, B., Ouyang, J., Jiang, L., Chen, B. and Tian, Z. (2026) 'PrecogUI: Proactive GUI Agents via Pre-cognitive Simulation and Experience Retrieval', Available at: https://omanscience.com/en/articles/precogui-proactive-gui-agents-via-pre-cognitive-simulation-and-experience-retrieval.
Vancouver
Kang B, Ouyang J, Jiang L, Chen B, Tian Z. PrecogUI: Proactive GUI Agents via Pre-cognitive Simulation and Experience Retrieval. https://omanscience.com/en/articles/precogui-proactive-gui-agents-via-pre-cognitive-simulation-and-experience-retrieval
IEEE
B. Kang, J. Ouyang, L. Jiang, B. Chen, and Z. Tian, "PrecogUI: Proactive GUI Agents via Pre-cognitive Simulation and Experience Retrieval," https://omanscience.com/en/articles/precogui-proactive-gui-agents-via-pre-cognitive-simulation-and-experience-retrieval.