When indices disagree: facing conceptual and practical challenges

Hypothesis testing requires meaningful ways to quantify relevant biological phenomena and account for alternative mechanisms that could explain the same pattern. Researchers combine experiments, statistics, and indices to account for these confounding mechanisms. Key concepts in ecology and evolution, like niche breadth or fitness, can be represented by several indices, which often provide uncorrelated estimates. Is this because the indices use different types of noisy data or because the targeted phenomenon is complex and multidimensional? We discuss implications of these scenarios and propose five steps to aid researchers in identifying and combining indices, experiments, and statistics. Supported by efforts to build databases of hypotheses and indices and document assumptions, these steps help provide a formal strategy to reduce self-confirmatory bias.