The reporting of study limitations and hedging in articles with and without causal diagrams

Abstract Causal directed acyclic graphs (DAG) are known to constitute a useful tool to improve the methodological and reporting quality of observational studies. At the same time, there are concerns that their (mis)use may lead to overconfident inferences. In this study, we investigate the reporting of study bias and limitations, and the use of hedging in articles with and without causal DAG. We performed a keyword search in two electronic databases (Medline and Web of Science) and one full-text repository (PubMed Central). The search strategy aimed to identify biomedical studies with an analytical observational design, that reported use of causal DAG. We selected a random sample of the articles that presented a graphic structure. We pair-matched individually 130 DAG with 130-Non-DAG papers on journal and issue. The number of research limitations acknowledged by the authors and of hedging terms per 100 words were analyzed using multiple linear regression models. From 2000 to 2020, we identified 2256 publications reporting the use of causal DAG, of which 1131 (50%) contained the actual graph structure. The number of study limitations acknowledged by the authors and the frequency of hedging were found to be similar in both samples. However, quantitative evaluations of study limitations were more frequently reported in studies with DAG (51 % vs. 25 %), while bias modeling techniques were uncommon in both groups (18% in DAG and 6% in non-DAG). In order to contextualize our findings, we used causal diagrams to discuss potential mechanisms for the communication of causal assumptions and related uncertainties in observational studies.