Numerical pitfalls in computing variance

One of the most common tasks in statistical computing is computation of sample variance. This would seem to be straightforward; there are a number of algebraically equivalent ways of representing the sum of squares \(S\), such as \[ S = \sum_{k=1}^n ( x_k - \bar{x})^2 \] or \[ S = \sum_{k=1}^n x_k^2 + \frac{1}{n}\bar{x}^2 \] and the sample variance is simply \(S/(n-1)\).

What is straightforward algebraically, however, is sometimes not so straightforward in the floating-point arithmetic used by computers. Computers cannot represent numbers to infinite precision, and arithmetic operations can affect the precision of floating-point numbers in unexpected ways.

How to check Likert scale summaries for plausibility

Suppose you are reading a paper that uses Likert scale responses. The paper reports the mean, standard deviation, and number of responses. If we are -- for some reason -- suspicious of a paper, we might ask, "Are these summary statistics possible for this number of responses, for this Likert scale?" Someone asked me this recently, so I wrote some simple code to help check. In this blog post, I outline how the code works.

BayesFactor version 0.9.12-2 released to CRAN

I've released BayesFactor 0.9.12-2 to CRAN; it should be available on all platforms now. The changes include:

• Added feature allowing fine-tuning of priors on a per-effect basis: see new argument rscaleEffects of lmBF, anovaBF, and generalTestBF

• Fixed bug that disallowed logical indexing of probability objects

• Fixed minor typos in documentation

• Fixed bug causing regression Bayes factors to fail for very small R^2

• Fixed bug disallowing expansion of dot (.) in generalTestBF model specifications

• Fixed bug preventing cancelling of all analyses with interrupt

• Restricted contingency prior to values >=1

• All BFmodel objects have additional "analysis" slot giving details of analysis

On radical manuscript openness

One of my papers that has attracted a lot of attention lately is "The Fallacy of Placing Confidence in Confidence Intervals," in which we describe some of the fallacies held by the proponents and users of confidence intervals. This paper has been discussed on twitter, reddit, on blogs (eg, here and here), and via email with people who found the paper in various places.  A person unknown to me has used the article as the basis for edits to the Wikipedia article on confidence intervals. I have been told that several papers currently under review cite it. Perhaps this is a small sign that traditional publishers should be worried: this paper has not been "officially" published yet.

BayesFactor updated to version 0.9.11-1

The BayesFactor package has been updated to version 0.9.11-1. The changes are:

  CHANGES IN BayesFactor VERSION 0.9.11-1

CHANGES
  * Fixed memory bug causing importance sampling to fail.

  CHANGES IN BayesFactor VERSION 0.9.11

CHANGES
  * Added support for prior/posterior odds and probabilities. See the new vignette for details.
  * Added approximation for t test in case of large t
  * Made some error messages clearer
  * Use callbacks at least once in all cases
  * Fix bug preventing continuous interactions from showing in regression Gibbs sampler
  * Removed unexported function oneWayAOV.Gibbs(), and related C functions, due to redundancy
  * gMap from model.matrix is now 0-indexed vector (for compatibility with C functions)
  * substantial changes to backend, to Rcpp and RcppEigen for speed
  * removed redundant struc argument from nWayAOV (use gMap instead)

The frequentist case against the significance test, part 2

The significance test is perhaps the most used statistical procedure in the world, though has never been without its detractors. This is the second of two posts exploring Neyman's frequentist arguments against the significance test; if you have not read Part 1, you should do so before continuing (“The frequentist case against the significance test, part 1”).

How to shoot yourself in the foot with various statistical philosophies

I've long been a fan of "How to shoot yourself in the foot" jokes. Having shot myself in the foot with different programming languages -- particularly with C -- I was thinking about how one might shoot oneself in the foot with various statistical approaches. So, here we go...

At the APS Observer: a profile of JASP

The APS Observer has just published a profile of JASP, a graphical user interface designed to make statistics easier. It includes Bayesian procedures by means of the R and the BayesFactor package. From the article:

 JASP distinguishes itself from SPSS by being as simple, intuitive, and approachable as possible, and by making accessible some of the latest developments in Bayesian analyses. At time of writing, JASP version 0.6 implements the following analysis tools in both their classical and Bayesian manifestations:

• Descriptive statistics
• t tests
• Independent samples ANOVA
• Repeated measures ANOVA
• Correlation
• Linear regression
• Contingency tables

Read more at the APS observer.

BayesFactorExtras: a sneak preview

Felix Schönbrodt and I have been working on an R package called BayesFactorExtras. This package is designed to work with the BayesFactor package, providing features beyond the core BayesFactor functionality. Currently in the package are:

1. Sequential Bayes factor plots for visualization of how the Bayes factor changes as data come in: seqBFplot()
2. Ability to embed R objects directly into HTML reports for reproducible, sharable science:  createDownloadURI()
3. Interactive BayesFactor objects in HTML reports;  just print the object in a knitr document.
4. Interactive MCMC objects in HTML reports; just print the object in a knitr document.

All of these are pretty neat, but I thought I'd give a sneak preview of #4. To see how it works, click here to play with the document on Rpubs!

I anticipate releasing this to CRAN soon.

On making a Bayesian omelet

My colleagues Eric-Jan Wagenmakers and Jeff Rouder and I have a new manuscript in which we respond to Hoijtink, van Kooten, and Hulsker's in press manuscript Why Bayesian Psychologists Should Change the Way they Use the Bayes Factor. They suggest a method for "calibrating" Bayes factor using error rates. We show that this method is fatally flawed, but also along the way we describe how we think about the subjective properties of the priors we use in our Bayes factors:

"...a particular researcher's subjective prior is of limited use in the context of a public scientific discussion. Statistical analysis is often used as part of an argument. Wielding a fully personal, subjective prior and concluding 'If you were me, you would believe this' might be useful in some contexts, but in others it is less useful. In the context of a scientific argument, it is much more useful to have priors that approximate what a reasonable, but somewhat-removed researcher would have in the situation. One could call this a 'consensus prior' approach. The need for broadly applicable arguments is not a unique property of statistics; it applies to all scientific arguments. We do not argue to convince ourselves; we should therefore make use of statistical arguments that are not pegged to our own beliefs...

It should now be obvious how we make our 'Bayesian omelet'; we break the eggs and cook the omelet for others in the hopes that it is something like what they would choose for themselves. With the right choice of ingredients, we think our Bayesian omelet can satisfy most people; others are free to make their own, and we would be happy to help them if we can. "

Our completely open, reproducible manuscript --- “Calibrated” Bayes factors should not be used: a reply to Hoijtink, van Kooten, and Hulsker --- along with a supplement and R code, is available on github (with DOI!).

BayesFactor version 0.9.10 released to CRAN

If you're running Solaris (yes, all zero of you) you'll have to wait for version 0.9.10-1, due to a small issue preventing the Solaris package from building on CRAN. The rest of you can pick up the updated version on CRAN today.

See below the fold for changes.

Multiple Comparisons with BayesFactor, Part 1

One of the most frequently-asked questions about the BayesFactor package is how to do multiple comparisons; that is, given that some effect exists across factor levels or means, how can we test whether two specific effects are unequal. In the next two posts, I'll explain how this can be done in two cases: in Part 1, I'll cover tests for equality, and in Part 2 I'll cover tests for specific order-restrictions.

Before we start, I will note that these methods are only meant to be used for pre-planned comparisons. They should not be used for post hoc comparisons.

New BayesFactor version 0.9.9 released to CRAN

Today I submitted a new release of BayesFactor, version 0.9.9, to CRAN. Among the new features are support for contingency table analyses, via the function contingencyTableBF, and analysis of a single proportion, via the function proportionBF. Other features and fixes include:

• Added "simple" argument to ttest.tstat, oneWayAOV.Fstat, and linearReg.R2stat; when TRUE, return only the Bayes factor (not the log BF and error)
• When sampling Bayes factors, recompute() now increases the precision of BayesFactor objects, rather than simply recomputing them. Precision from new samples is added
• Added Hraba and Grant (1970) data set; see ?raceDolls
• Added model.matrix method for BayesFactor objects; allows for extracting the design matrix used for an analysis
• recompute() now has multicore and callback support, as intended
• Moved many backend functions to Rcpp from R C API
• t test samplers now sample from interval null hypotheses and point null hypotheses where appropriate
• fixed bug in in meta t test sampler which wouldn't allow sampling small numbers of MCMC samples

Embedding RData files in Rmarkdown files for more reproducible analyses

For those of us interested in reproducible analysis, Rmarkdown is a great way of communicating our code to other researchers. Rstudio, in particular, makes it very easy to create attractive HTML document containing text, code, and figures, which can then be sent to colleagues or put on the internet for anyone to see. If you aren't using Rmarkdown for your statistical analyses, I recommend you start; you'll never go back to simple script files again (and your colleagues won't want you to).

In this post, I describe how to improve your Rmarkdown by embedding data that can be downloaded by anyone viewing the document in a modern browser with javascript enabled. For a quick look, see the example Rmd file and resulting HTML file.

BayesFactor version 0.9.8 released to CRAN

BayesFactor version 0.9.8 has been released on CRAN! This is a both a bug fix and feature update. From the NEWS:

•  Fixed bugs in model enumeration code
•  Fixed bug leading to wrong computation of number of covariate when interactions between continuous variables were included
•  Corrected typos/old information in the documentation 
•  Fixed a memory allocation bug that affected computing Bayes factors with lots of data 
•  Added meta-analytic Bayes factor for t tests (see meta.ttestBF)
•  Fixed bug in ttestBF that yielded Bayes factor of NaN for very extreme posterior interval probabilities
•  Fixed several bugs causing infinite integrals; generally improved integration
•  Added check to ensure no missing data before analyses
•  Added callbacks for access by third-party interfaces

See also the new entry in the manual for meta-analytic t tests. In addition to these changes, most of the code for contingency table analysis has been added; these functions will be released in the next update.

BayesFactor update: 0.9.7.

BayesFactor version 0.9.7 has been released to CRAN. It does have a few bug fixes, so update soon. A list of changes can be found in the NEWS file.

Bayes factor t tests, part 2: Two-sample tests

In the previous post, I introduced the logic of Bayes factors for one-sample designs by means of a simple example. In this post, I will give more detail about the models and assumptions used by the BayesFactor package, and also how to do simple analyses of two- sample designs.
See the previous posts for background:

• What is a Bayes factor?
• Bayes factor t tests, part 1

This article will cover two-sample t tests.

Bayes factor t tests, part 1

In my first post, I described the general logic of Bayes factors. I will continue discussing the general logic of Bayes factor, while introducing some of the basic functionality of the BayesFactor package.

What is a Bayes factor?

The BayesFactor package

This blog is a companion to the BayesFactor package in R (website), which supports inference by Bayes factors in common research designs. Bayes factors have been proposed as more principled replacements for common classical statistical procedures such as \(p\) values; this blog will offer tutorials in using the package for data analysis.
In this first post, I describe the general logic of Bayes factors using a very simple research example. In the coming posts, I will show how to do a more complete Bayesian data analysis using the R package.