Abstract
As language models (LMs) exhibit increasingly consciousness-like behaviors, evaluating their cognitive abilities becomes essential. We introduce AwarenessBench, the first comprehensive benchmark for assessing the cognitive abilities of LMs in four dimensions: metacognition, self-awareness, social awareness, and situational awareness, covering 15 cognitive functions and 14,381 samples. Evaluating 18 state-of-the-art LMs, we find that all consistently surpass random baselines, with more advanced models performing better. We further compare LMs with human performance across three demographic groups, where the best-performing model surpasses human averages overall, but most still fall markedly short in metacognition and self-awareness. Finally, we show that awareness is a distinct capability: progress in language modeling or reasoning does not necessarily translate into improved cognition.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Li, X., Xu, R., Zhang, T., Wang, Y., Chen, S., Lyu, Q., Zhang, B., Yang, P., Chen, K. X., Shi, H., Wang, Y., & Xu, W. (2026). AwarenessBench: Assessing Cognitive Capabilities of Language Models. https://omanscience.com/en/articles/awarenessbench-assessing-cognitive-capabilities-of-language-models
MLA 9
Li, Xiaojian, et al. "AwarenessBench: Assessing Cognitive Capabilities of Language Models." https://omanscience.com/en/articles/awarenessbench-assessing-cognitive-capabilities-of-language-models.
Chicago (author–date)
Li, Xiaojian, Rongwu Xu, Tianyun Zhang, Yue Wang, Shuo Chen, Qiner Lyu, Briana Zhang, Peiran Yang, Kyle Xue Chen, Haoyuan Shi, Yu Wang, and Wei Xu. 2026. "AwarenessBench: Assessing Cognitive Capabilities of Language Models." https://omanscience.com/en/articles/awarenessbench-assessing-cognitive-capabilities-of-language-models.
Harvard
Li, X., Xu, R., Zhang, T., Wang, Y., Chen, S., Lyu, Q., Zhang, B., Yang, P., Chen, K. X., Shi, H., Wang, Y. and Xu, W. (2026) 'AwarenessBench: Assessing Cognitive Capabilities of Language Models', Available at: https://omanscience.com/en/articles/awarenessbench-assessing-cognitive-capabilities-of-language-models.
Vancouver
Li X, Xu R, Zhang T, Wang Y, Chen S, Lyu Q, et al. AwarenessBench: Assessing Cognitive Capabilities of Language Models. https://omanscience.com/en/articles/awarenessbench-assessing-cognitive-capabilities-of-language-models
IEEE
X. Li, R. Xu, T. Zhang, Y. Wang, S. Chen, Q. Lyu, B. Zhang, P. Yang, K. X. Chen, H. Shi, Y. Wang, and W. Xu, "AwarenessBench: Assessing Cognitive Capabilities of Language Models," https://omanscience.com/en/articles/awarenessbench-assessing-cognitive-capabilities-of-language-models.