On verbal categories for the interpretation of Bayes factors

As Bayesian analysis is becoming more popular, adopters of Bayesian statistics have had to consider new issues that they did not before. What is makes “good” prior? How do I interpret a posterior? What Bayes factor is “big enough”? Although the theoretical arguments for the use of Bayesian statistics are very strong, new and unfamiliar ideas can cause uncertainty in new adopters. Compared to the cozy certainty of \(p<.05\), Bayesian statistics requires more care and attention. In theory, this is no problem at all. But as Yogi Berra said, "In theory there is no difference between theory and practice. In practice there is."

In this post, I discuss the the use of verbal labels for magnitudes of Bayes factors. In short, I don't like them, and think they are unnecessary.

Multiple Comparisons with BayesFactor, Part 2 - order restrictions

In my previous post, I described how to do multiple comparisons using the BayesFactor package. Part 1 concentrated on testing equality constraints among effects: for instance, that the the effects of two factor levels are equal, while leaving the third free to be different. In this second part, I will describe how to test order restrictions on factor level effects. This post will be a little more involved than the previous one, because BayesFactor does not currently do order restrictions automatically.

Again, I will note that these methods are only meant to be used for pre-planned comparisons. They should not be used for post hoc comparisons.

Multiple Comparisons with BayesFactor, Part 1

One of the most frequently-asked questions about the BayesFactor package is how to do multiple comparisons; that is, given that some effect exists across factor levels or means, how can we test whether two specific effects are unequal. In the next two posts, I'll explain how this can be done in two cases: in Part 1, I'll cover tests for equality, and in Part 2 I'll cover tests for specific order-restrictions.

Before we start, I will note that these methods are only meant to be used for pre-planned comparisons. They should not be used for post hoc comparisons.

A parable on confidence intervals: why "confidence" is misleading

Null hypothesis significance testing (NHST) is increasingly falling out of style with methodologically-minded behavioral and social scientists. Many diverse critiques have been leveled against significance testing; the debate is increasingly what should replace it. Building on work with my colleagues (see here and here), I discuss and critique one replacement option that has been persistently suggested over the years: confidence procedures. We begin with a parable.

New BayesFactor version 0.9.9 released to CRAN

Today I submitted a new release of BayesFactor, version 0.9.9, to CRAN. Among the new features are support for contingency table analyses, via the function contingencyTableBF, and analysis of a single proportion, via the function proportionBF. Other features and fixes include:

• Added "simple" argument to ttest.tstat, oneWayAOV.Fstat, and linearReg.R2stat; when TRUE, return only the Bayes factor (not the log BF and error)
• When sampling Bayes factors, recompute() now increases the precision of BayesFactor objects, rather than simply recomputing them. Precision from new samples is added
• Added Hraba and Grant (1970) data set; see ?raceDolls
• Added model.matrix method for BayesFactor objects; allows for extracting the design matrix used for an analysis
• recompute() now has multicore and callback support, as intended
• Moved many backend functions to Rcpp from R C API
• t test samplers now sample from interval null hypotheses and point null hypotheses where appropriate
• fixed bug in in meta t test sampler which wouldn't allow sampling small numbers of MCMC samples

Embedding RData files in Rmarkdown files for more reproducible analyses

For those of us interested in reproducible analysis, Rmarkdown is a great way of communicating our code to other researchers. Rstudio, in particular, makes it very easy to create attractive HTML document containing text, code, and figures, which can then be sent to colleagues or put on the internet for anyone to see. If you aren't using Rmarkdown for your statistical analyses, I recommend you start; you'll never go back to simple script files again (and your colleagues won't want you to).

In this post, I describe how to improve your Rmarkdown by embedding data that can be downloaded by anyone viewing the document in a modern browser with javascript enabled. For a quick look, see the example Rmd file and resulting HTML file.

BayesFactor version 0.9.8 released to CRAN

BayesFactor version 0.9.8 has been released on CRAN! This is a both a bug fix and feature update. From the NEWS:

•  Fixed bugs in model enumeration code
•  Fixed bug leading to wrong computation of number of covariate when interactions between continuous variables were included
•  Corrected typos/old information in the documentation 
•  Fixed a memory allocation bug that affected computing Bayes factors with lots of data 
•  Added meta-analytic Bayes factor for t tests (see meta.ttestBF)
•  Fixed bug in ttestBF that yielded Bayes factor of NaN for very extreme posterior interval probabilities
•  Fixed several bugs causing infinite integrals; generally improved integration
•  Added check to ensure no missing data before analyses
•  Added callbacks for access by third-party interfaces

See also the new entry in the manual for meta-analytic t tests. In addition to these changes, most of the code for contingency table analysis has been added; these functions will be released in the next update.