Year: 2012
Type: article
Abstract: Proposal for a tutorial on Using Bayes to Interpret Non-significant Results Zoltan Dienes ([email protected]) School of Psychology, University of Sussex, Brighton, BN1 9QH Keywords: Bayesian inference; Bayes Factors; confidence intervals; likelihood intervals; non-significant results; evidence. 1. Outline the objectives and scope of the tutorial. The purpose of the tutorial is to present simple tools for dealing with non-significant results, an area which cognitive scientists have consistently found problematic. In particular, people will be taught how to apply Bayes Factors and likelihood intervals to draw meaningful inferences from non-significant data, using free easy-to-use on-line software: Software which allows one to determine whether there is strong evidence for the null and against one’s theory, or if the data are just insensitive, a distinction p_values cannot make. These tools have greater flexibility than power calculations and allow null results to be interpreted over a wider range of situations. Such tools should allow the publication of null results to become easier. The online software for Bayes Factors (with instructions) is here: http://www.lifesci.sussex.ac.uk/home/Zoltan_Dienes/inferen ce/Bayes.htm And the online software for likelihood intervals here: http://www.lifesci.sussex.ac.uk/home/Zoltan_Dienes/inferen ce/Likelihood.htm 2. Explain how the tutorial will be delivered giving a detailed description of the material that will be covered. The tutorial will consist of lectures by me; after the second hour people will be able to use their internetted laptops to work through examples on the software, and thereby interactively follow the points I make, and also explore the tools for themselves. Schedule: 9:30 – 10:30 Basics: The different aims of significance testing and Bayesian inference (including the three moral and inferential paradoxes of significance testing and their solution) 10:45 – 11:45 Bayes Factors 12- 1 Examples with Bayes Factors, to illustrate appropriate and inappropriate use, and robustness checks (Bayesian analysis can of course be misused in ways we will clarify) 2-3 Confidence intervals, likelihood intervals, credibility intervals with examples, including the (little discussed) four principles for using intervals inferentially in theory testing 3:30 – 4:30 Examples showing the complementary strengths and weaknesses of Bayes factors and interval methods for interpreting null results 4:45 – 5:30 Discussion of e.g. any particular data people wish to bring, and free questions The tutorial will emphasize how statistics, both Bayes factors and interval estimates, can be brought into more intimate contact with theory than has typically been the case, and appropriate ways of doing this. (Interpreting null results requires making contact with theory.) My emphasis will be practical rather than ideological, though conceptual arguments will be important. 3. Justify why it is important to have a tutorial in the proposed area at the conference. Users of statistics have been criticised for decades for their interpretation of non-significant results. Users have either used null results to count against a theory that predicted a difference (without establishing that the results actually counted against the theory) or ignored the results as uninformative (without establishing that they were). One only need pick up any recent issue of almost any journal to see this. (I don’t exclude many of my own papers from this criticism!) In that sense the topic has been important to clarify for a long time. Recently, however there was been a resurgence of interest in Bayesian and likelihood methods, and the recent developments are particularly useful for users of statistics. Little can be more important than that we as a community draw appropriate inferences from data, and get the most from our data. The issues are applicable to the whole community of cognitive scientists, and hence appropriate for a meeting of the Cognitive Science Society. Several strategies for dealing with null results will be taught, as well as reasons why the most common strategies, orthodox as well as Bayesian, can be problematic. 4. Specify how relevant the topic is for the conference (i.e., does it focus on an emerging or cross-disciplinary research topic?) Bayes has been making a resurgence for getting on 10 years now in cognitive science. Part of the interest has been in Bayes as a model for how the mind works. While the workshop has nothing to say on what the best theory of the mind is, theories of how we should analyse data are clearly relevant to theories of how the mind works. More importantly, part of the recent interest in Bayes is precisely on the topic of the workshop – principles and methods for drawing statistical inferences. Indeed, at the meeting of the Society last year Kruschke held a very successful workshop on Bayesian inference based on his book. I will be teaching a slightly different philosophy and different methods (but which complement Kruschke’s approach). Krushcke
Author: Zoltán Dienes
Language: English
Subfield: Statistics and Probability
Field: Mathematics
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W2395259841