Being accurate about verbal credibility assessment

Purpose: Verbal credibility assessments examine language to discern lie from the truth. These tests are used for the scientific study of the language of lies in US Presidential candidates and fraudulent scientists, but also in criminal proceedings for evaluating allegations of child sexual abuse. The dominant approach in psycholegal deception research to date (used in 81% of recent studies that report on accuracy) to estimate the accuracy of a method is to find the optimal statistical separation between lies and truths in a single dataset. However, this method lacks safeguards against accuracy overestimation. Method & Results: A simulation study and empirical data show that this procedure produces overoptimistic accuracy rates that, especially for small sample size studies typical of this field, yield misleading conclusions up to the point that a non-diagnostic tool can be shown to be a valid one. Cross-validation is a cheap and easy remedy to this problem. Conclusions: We propose a set of guidelines to make the psycholegal research meet highest standards and encourage researchers to be more accurate about accuracy and to conduct proper validation when reporting accuracy rates.