Survey Item Validation

In the social sciences, validity refers to the adequacy of a survey (or other mode of assessment) for its intended purpose. Validation refers to the activities undertaken during and after the construction of the survey to evaluate and improve validity. Item validation refers here to procedures for evaluating and improving respondents’ understanding of the questions and response options included in a survey. Ver- bal probing techniques such as cognitive interviews can be used to understand respondents’ response process, that is, what they are thinking as they answer the survey items. Although cognitive interviews can provide evidence for the validity of survey items, they are time-consuming and thus rarely used in practice. The Response Process Evaluation (RPE) method is a newly-developed technique that utilizes open-ended meta-surveys to rapidly collect evidence of validity across a population of interest, make quick revisions to items, and immediately test these revisions across new samples of respondents. Like cognitive interviews, the RPE method focuses on how participants interpret the item and select a re- sponse. The chapter demonstrates the process of validating one survey item taken from the Inventory of Non-Ordinary Experiences (INOE).