PRDA package: Enhancing Statistical Inference via Prospective and Retrospective Design Analysis.

Abstract

There is growing awareness about the importance of performing power analysis to define an appropriate sample size when planning an experiment. However, statistical power is not the only relevant aspect of the study design. Other related inferential risks, such as the probability of estimating the effect in the wrong direction (Type S [sign] error) or the average overestimation of the actual effect (Type M [magnitude] error), are also important. Statistical power, Type M and Type S errors can be evaluated in what Gelman and Carlin (2014) defined as Design Analysis, a process that may inform the planning stage of an experiment and the evaluation of studies’ results. We introduce the PRDA (Prospective and Retrospective Design Analysis) R-package that allows researchers to perform a “Design Analysis” under different experimental scenarios (Altoè et al., 2020). Considering a plausible effect size (or its prior distribution), researchers can evaluate either the inferential risks for a given sample size or the required sample size to obtain a given statistical power. The main aim of PRDA package is to enhance researcher reasoning about inferential risks avoiding automated decisions. Previously, PRDA functions were limited to mean differences between groups considering Cohen’s d in the Neyman-Pearson (N-P) framework. Now, we present the newly developed features that include other effect sizes (such as Pearson’s correlation) as well as Bayes Factor hypothesis testing. The PRDA R-package can be found at https://github.com/masspastore/PRDA.

Claudio Zandonella Callegher

Post-Doc Researcher

My research interests include the formalization of psychological thoeries, Bayesian methods in Behavioral Sciences, and everything related to programming in R!