الملخص
Evaluating agents by outcomes alone can obscure the capabilities that produce them. This problem is especially pronounced in evolving environments, where outcomes reflect a closed-loop interaction between agent behavior and changing external conditions. We introduce LiveMACEBench, a process-aware benchmark that uses live financial markets as a naturally evolving testbed for persistent LLM agents. Five frontier LLMs operate along continuous trajectories under matched Tool Use, Persistent Memory, Rule Following, and Multi-Agent Collaboration configurations. We evaluate them through both realized outcomes and mechanism-specific diagnostics derived from complete decision traces. Across 30 days of live evaluation, we find a pronounced outcome-capability gap: realized returns often diverge from capability-specific measurements, and similar outcomes can arise from markedly different patterns of mechanism use. Trace-level diagnostics further expose distinct bottlenecks across capabilities, demonstrating that mechanism access, effective mechanism use, and downstream performance are not interchangeable measures of agent capability. LiveMACEBench makes this distinction measurable, turning live markets from a performance leaderboard into a diagnostic environment for agent capability
الكلمات المفتاحية
الموضوع
بيانات النشر
- المجلة
- غير متاح
- وصول مفتوح
- وصول مفتوح أخضر
اقتبس هذه المقالة
APA 7
Zhao, J., Fu, L., Wen, Y., Wang, Y., Liu, X., Shu, Y., Lu, Y., Xing, X., Tong, J., Xu, H., Zhang, Q., & Huang, X. (2026). LiveMACE: Process-Aware Evaluation of LLM Agent Capabilities in Evolving Markets. https://omanscience.com/ar/articles/livemace-process-aware-evaluation-of-llm-agent-capabilities-in-evolving-markets
MLA 9
Zhao, Jun, et al. "LiveMACE: Process-Aware Evaluation of LLM Agent Capabilities in Evolving Markets." https://omanscience.com/ar/articles/livemace-process-aware-evaluation-of-llm-agent-capabilities-in-evolving-markets.
شيكاغو (المؤلف–التاريخ)
Zhao, Jun, Leiming Fu, Yanbo Wen, Yiding Wang, Xuantong Liu, Yang Shu, Yuyang Lu, Xuanran Xing, Jingqi Tong, Hao Xu, Qi Zhang, and Xuanjing Huang. 2026. "LiveMACE: Process-Aware Evaluation of LLM Agent Capabilities in Evolving Markets." https://omanscience.com/ar/articles/livemace-process-aware-evaluation-of-llm-agent-capabilities-in-evolving-markets.
هارفارد
Zhao, J., Fu, L., Wen, Y., Wang, Y., Liu, X., Shu, Y., Lu, Y., Xing, X., Tong, J., Xu, H., Zhang, Q. and Huang, X. (2026) 'LiveMACE: Process-Aware Evaluation of LLM Agent Capabilities in Evolving Markets', Available at: https://omanscience.com/ar/articles/livemace-process-aware-evaluation-of-llm-agent-capabilities-in-evolving-markets.
فانكوفر
Zhao J, Fu L, Wen Y, Wang Y, Liu X, Shu Y, et al. LiveMACE: Process-Aware Evaluation of LLM Agent Capabilities in Evolving Markets. https://omanscience.com/ar/articles/livemace-process-aware-evaluation-of-llm-agent-capabilities-in-evolving-markets
IEEE
J. Zhao, L. Fu, Y. Wen, Y. Wang, X. Liu, Y. Shu, Y. Lu, X. Xing, J. Tong, H. Xu, Q. Zhang, and X. Huang, "LiveMACE: Process-Aware Evaluation of LLM Agent Capabilities in Evolving Markets," https://omanscience.com/ar/articles/livemace-process-aware-evaluation-of-llm-agent-capabilities-in-evolving-markets.