A meta-analytic investigation of the psychometric evidence of language-based machine learning personality assessment

This paper presents a meta-analytic review of the multidimensional psychometric evidence of language-based machine learning (ML) supported personality assessment, examining the reliability and construct validity, specifically convergent and discriminant validity, of the extracted scores for the big five personality domains derived from ML natural language processing (NLP) techniques. Moreover, factors that may potentially moderate the effect size correlations between traditional personality judgment using self-reports and machine-generated judgment from NLP algorithms, such as types of language data source, types of algorithms, and types of personality scales used. This study uncovered that personality scores derived from textual data using ML and NLP approaches are only partially consistent with those from traditional personality assessment, and that much psychometric evidence is lacking in existing language-based ML personality assessment applications.