A Survey on Evaluation of Embodied AI

The rapid progression of Multimodal Large Language Models and Vision-Language-Action models has substantially propelled Embodied AI. Nevertheless, comprehensively evaluating these systems remains challenging, primarily due to the representational gaps between semantic understanding and physical grounding, alongside inherent limitations within specific modules of current agents. Existing evaluations reveal that agents face significant deficits not only in individual capabilities like perception and planning but also in the dynamic system integration required for reliable real-world deployment. To address these challenges, this review establishes a systematic evaluation framework structured around the complete Perception-Cognition-Planning-Action loop. First, regarding evaluation targets, we dissect four core capabilities ranging from spatial perception to action execution, and further analyze the system's trustworthiness across dimensions of Safety, Robustness, and Generalization. Second, regarding evaluation platforms, we systematically summarize representative simulators, datasets, and benchmarks, highlighting the technological transition from rigid physics engines to scalable generative environments to assist researchers in selecting appropriate testbeds. Third, regarding evaluation methodologies, we examine the critical shift from outcome-oriented metrics to multidimensional assessments that emphasize process quality and physical safety. Finally, we identify grand challenges and advocate for a closedloop Evaluation-Diagnosis-Enhancement paradigm. This work aims to facilitate the bridging of the gap between semantic understanding and physical grounding, providing a rigorous reference for standardizing the evaluation of General Embodied Intelligence. We consistently maintain the related open-source materials at: https://github.com/EmbodiedAISurvey/Embodied-AI-Eval-Survey. Supplementary Material File (a_sample_article_using_ieeetran_cls_for_ieee_journals_and_transactions.pdf) - Download - 2.48 MB Information & Authors Information Version history Copyright This work is licensed under a Non Exclusive No Reuse License.

Authors Metrics & Citations Metrics Article Usage 812views 406downloads Citations Download citation Liyu Hou, Linyuan Gao, Yuan Wu, et al. A Survey on Evaluation of Embodied AI. Authorea. 04 February 2026. DOI: https://doi.org/10.22541/au.177023340.02874343/v1 DOI: https://doi.org/10.22541/au.177023340.02874343/v1 If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download. For more information or tips please see 'Downloading to a citation manager' in the Help menu. Cited by - Pose-based embodied interaction for digital Dunhuang dance heritage, npj Heritage Science, 14, 1, (2026).https://doi.org/10.1038/s40494-026-02470-2 Loading...