Assumptions of OLS Models: The State of Statistical Practice in Psychology
Sladekova, M., Poupa, V. L, Field, A. P.
Stage: Manuscript in preparation.
Frequentist statistics are a fertile ground for generating misunderstanding (and subsequent misapplication) of commonly used statistical tools, like p-values or confidence intervals. This is not surprising - most researchers want to be able to make informative conclusions about their data and hypotheses, rather than ruminate about how wrong they are in the long run of infinite series of hypothetical replications.
Ordinary Least Squares estimation is the most common estimation methods in Psychology, used in models like ANOVA, t-test, regression, and a range of other models falling under the umbrella of the General Linear Model. All statistical models make assumptions (some more grounded in reality in others) and general linear models are no exception.
The aim of this project is to learn more about the statistical knowledge, understanding, and practice of psychology researchers, focusing on their understanding of the OLS assumptions and the extent to which violations can affect the analysis. We also want to learn about researchers’ knowledge and application of methods for detecting and addressing violations of assumptions.
The project comprises of two studies. In the first study, the we asked the researchers to complete an online survey containing a knowledge test and some questions about their practice. In the second study, the researchers complete an analysis exercise where their task is to analyse two datasets related to two research scenarios, and note down the steps of their analysis and their reasoning along the way. Both manuscripts are currently in the final stage of preparation.