Lessons and Recommendations from a Research Audit for the Transparent Psi Project (TPP)

The lessons and recommendations described here were developed from conducting a research audit of the Transparent Psi Project (TPP). The TPP was intended as a potential model and learning experience for optimal research practices. The topics discussed here are among the potential lessons from the TPP. The recommendations are: (1) Validation of data collection software is essential. (2) Evaluation of power or operating characteristics (inferential errors) is essential, including for sequential Bayesian analyses. (3) An online repository copy of data is very valuable, but easy-to-use, reliable, secure processes remain to be developed. (4) A good research audit is much more extensive than current peer review for publication, including Registered Reports. (5) Quality control (QC) comes before auditing. (6) Good research practice guidelines and/or standard operating procedures (SOPs) for general use would be very valuable for confirmatory psychological research. (7) It is time to implement routine measures to prevent experimenter fraud. (8) Git appears to be not optimal for software tracking to prevent programming fraud.