### The BayesFactor package

This blog is a companion to the`BayesFactor`

package in R (website), which supports inference by Bayes factors in common research designs. Bayes factors have been proposed as more principled replacements for common classical statistical procedures such as \(p\) values; this blog will offer tutorials in using the package for data analysis.In this first post, I describe the general logic of Bayes factors using a very simple research example. In the coming posts, I will show how to do a more complete Bayesian data analysis using the R package.

### What is a Bayes factor?

Suppose that two researchers are interested in public opinion about public smoking bans. Paul believes that 70% of the public support such bans; Carole believes that the support is less, at 60%. Paul and Carole decide ask 100 randomly selected people whether they support public smoking bans.Because Paul and Carole have very specific beliefs about the true support for the public smoking bans, they also have predictions about how the sample will turn out. Of course, Paul's best guess is that 70 out of 100 will support smoking bans, and likewise Carole's best guess is 60. However, Paul and Carole can be more specific because the Binomial distribution is appropriate for model the kind of random sample. The figure below shows the predictions of Carole (blue) and Paul (red).

Having made predictions, Paul and Carole collect their random sample. Of the 100 people in the sample, 62 are supportive. This seems to support Carole because the observation is closer to Carole's average prediction; however, Paul points out that he also predicted that 62 was possible, and so his hypothesis is not ruled out. It does seem obvious that the observation supports Carole, but by how much?

One way of answering this question is to ask about the relative weight of the evidence: that is, how convincing is the observation with respect to the two hypothesis in question? In order to answer this question, we need a way of understanding what it means for evidence to change what we believe. We start with the idea of

*prior odds*, which describe the degree to which we favor one hypothesis over another before we see the data. These can be written as \[ \frac{P(\cal H_c)}{P(\cal H_p)} \] where \(P\) here represents plausibility and \(\cal H_c\) and \(\cal H_p\) are hypotheses of Carole and Paul. We can also describe the

*posterior odds*, which describe the degree to which we favor one hypothesis over another after observing the data. \[ \frac{P(\cal H_c\mid y)}{P(\cal H_p\mid y)} \] where \(y\) is what we observed. The question now is how we move from the prior odds to the posterior odds in the right way. Obviously, we cannot simply do this in any way we like. There must be some ways that are better than others. Bayes' rule gives a way to answer this question. Bayes' rule says that the relative plausibility of two hypotheses must be changed by the data in a particular way. \[ \frac{P(\cal H_c\mid y)}{P(\cal H_p\mid y)} = \frac{P(y\mid \cal H_c)}{P(y\mid\cal H_p)}\times\frac{P(\cal H_c)}{P(\cal H_p)} \] The term in the middle is a factor that we multiply the prior odds by to become the posterior odds. It represents how our relative beliefs should change in light of the data, and is called the Bayes factor. Luckily, the logic is quite simple:

The

We can now easily compute the evidence in the data favoring Carole over Paul. Carole specified that the observation \(y=62\) had a probability of 0.0754. Paul specified that the observation \(y=62\) had a probability of 0.0191. Under Carole's hypothesis, the observed data is much more likely than under Paul's, as shown in the figure below:**Bayes factor**is the relative evidence in the data. The evidence in the data favors one hypothesis, relative to another, exactly to the degree that the hypothesis predicts the observed data better than the other.The Bayes factor — the evidence in the data — is precisely factor by which Carole's line is taller than Paul's, which is:

dbinom(y, N, carole)/dbinom(y, N, paul)

```
## [1] 3.953
```

or about 4. The observed data favors Carole by a factor of 4.The figure below shows how much the data favors Carole for many possible observations. The data favor Carole when the Bayes factor is greater than 1, which happens for all observations 65 or less; the data favor Paul when the Bayes factor is less than 1, which happens when the observation greater than 65. For very large or small observations, the evidence is strong; for observations around 65, the evidence is slight, making it difficult to distinguish between the two hypotheses.

### Bayes factors when the parameter is uncertain

The above example was adequate for a simple demonstration, but lacks an important feature common to research: uncertainty about the true parameter value. Carole and Paul both had very specific hypotheses. Typically, this is not the case. Unless a hypothesis is very specific about a particular parameter (for instance, that an experimental manipulation has no effect), hypotheses are more diffuse.Carole and Paul might have hypotheses that look something like the figure below:

Note that on the \(x\)-axis is the

*true*proportion of the population who support the smoking bans. Suppose we call the true proportion of supporters \(\theta\). Instead of believing that the true proportion is exactly 60%, Carole now believes that it is “around” 60%. The blue distribution shows how her belief drops off as the proportion is further away from 0.6. Likewise, Paul has adopted a distribution around 0.7. We refer to these curves as \(p_c(\theta)\) for Carole and \(p_p(\theta)\) for Paul.

Since the Bayes factor is determined by the probability that each hypothesis assigns to the observed data, we must determine what Paul and Carole would now predict. Carole's hypothesis \(\cal H_c\) is a distribution of values of \(\theta\), each weighted differently. Some values, like 0.6, are quite plausible; other values, like 0.7, are implausible. In order to determine the plausibility of a particular observation, we can determine the probability that it would occur

*given a particular value for \(\theta\)*(that is, \(P(y\mid\theta)\)). We then weight this probability by the plausibility given to that particular \(\theta\) value, \(p(\theta)\). For each hypothesis, this can be represented by an integral: \[ P(y\mid{\cal H}_c) = \int P(y\mid \theta)p_c(\theta)\,d\theta \] where \(p_c(\theta)\) is the function representing Carole's hypothesis. The probability of an observation is thus a weighted average over all the different possible values for \(\theta\). The figure below shows how the more diffuse hypothesis affects the predictions for data. In gray are the predictions from the old, single point hypothesis. In color are the new predictions from the diffuse hypothesis. The diffuse hypotheses have made the predictions for the data more diffuse. The bottom row shows the new predictions made by Carole and Paul side-by-side.

We can now compare the two hypotheses in light of the observation of 62 supporters out of 100, as the figure below shows. The Bayes factor has been reduced to 2.8267 in favor of Carole. This makes sense; both Carole and Paul have hedged their predictions for the data, making them more spread out and in the process less distinct. The reduction in the Bayes factor reflects this by becoming attenuated.

Uncertain hypotheses are not a problem for the Bayes factor to handle, as long as they can be specified as probability distributions, but hedging one's bets with more uncertain predictions will limit reduce the amount of evidence that can be accumulated in favor of one's hypothesis. This means Bayes factors penalizes flexible hypotheses, yielding a natural Occam's razor.

### Strength of the evidence

One might ask how much evidence is provided by the Bayes factors of 3.9534 and 2.8267 found above. The simple answer is that the number itself is directly interpretable, since it arises as the shift of relative odds. In this, the Bayes factor is different from other proposed (but ultimately flawed) evidential statistics, such as the \(p\) value, because such statistics have no direct evidential interpretation.Nevertheless, various labels for the “strength” of a Bayes factor have been proposed. Kass and Raftery (1995), for instance, propose that Bayes factors between 1/3.2 and 3.2 are “not worth more than a bare mention”. See the Wikipedia entry for Bayes factors for more details.

### Summary

*Bayes factors represent the weight of evidence in the data for competing hypotheses.*Bayes factors are the degree to which the data shift the relative odds between two hypotheses. There are principled reasons why we should interpret the Bayes factor as a measure of the strength of the relative evidence.*The Bayes factor is intimately linked to the predictions of a hypothesis.*Because the evidence for a hypothesis given by the degree to which it has predicted the observed data, hypotheses that do not have predictions for data cannot accumulate evidence. Hypotheses with no predictions, such as \(\mu\neq0\) are not allowed.*The Bayes factor can be directly interpreted, without recourse to labels.*The strength of the Bayes factor is reflected by the fact that it is a multiplicative change in odds. However, some authors provide labels to help interpret evidence.

`BayesFactor`

package.
Thanks so much for these great posts.

ReplyDeleteIs there an example of calculating a Bayes Factor for the difference of proportions as described in the example above? Can this be done in the BayesFactor package?

Thanks!

Hi Max, I don't have a blog post about that yet, but Bayes factors of the sort you want can be calculated using the contingencyTestBF() function. See the manual here for an example: http://bayesfactorpcl.r-forge.r-project.org/#ctables

DeleteHi Richard, this post is genial. The Bayes factor is common tool in phylogenetic research to decide between two evolutionary hypotheses (models).

ReplyDeleteThis is calculated as BF= 2(harmonic mean 1(complex model) - harmonic mean2 (simple model).

Do you know this procedure?

Best regards

Oops, missed the reply. See response below.

DeleteHi Violeta, yes, I am aware of the harmonic mean estimator. BayesFactor does not use that estimator; instead, it uses importance sampling, simple Monte Carlo sampling, Gaussian quadrature, and a Laplace approximation (depending on the situation). They all estimate the same quantity, but the harmonic mean estimate is notoriously unstable (see Radford Neal's comment here).

ReplyDeleteHi,

ReplyDeleteThank you for this post. Do you have the R code for this example? I downloaded the package and I didn't see an example like this one.

Thank you so much for posting this article. It helped me a lot.

ReplyDeleteIn the last paragraph, you mentioned that "The Bayes factor is intimately linked to the predictions of a hypothesis."

Then how can I get the predictive density of x given a specific hypothesis?

I guess p(x|H) is not a predictive probability density, but just a probability that gives us information about the plausibility of observed data x under H...

Thank-you for your nice post Richard!

ReplyDeleteCould you give further detail about how you obtained the 2.8267 result for the more diffuse hypotheses? In particular, what type of distributions are those shown in the first figure of the section "Bayes factors when the parameter is uncertain"? If I am not mistaken those are the priors, right?

Thanks

So far the best explanation (to me) that I came across, thanks very much Richard!

ReplyDeleteThis comment has been removed by the author.

ReplyDeleteThank you so much for posting this article. It helped me a lot.

ReplyDeleteHadoop-Big-Data-Administration

Great article Glad to find your blog. Thanks for sharing.

ReplyDeleteibm-message-broker training in chennai

• Very good effort in collecting information.........

ReplyDeletewebsphere training in chennai

• great article!!!!!This is very importent information for us.I like all content and information.I have read it.You know more about this please visit again.

ReplyDeleteoracle training in chennai

ReplyDeletehttps://www.quester.pk/question/does-telenor-have-3g-coverage-in-gagra/https://www.quester.pk/question/does-telenor-have-3g-coverage-in-ferozewala/https://www.quester.pk/question/does-telenor-have-3g-coverage-in-fateh-jang/https://www.quester.pk/question/does-telenor-have-3g-coverage-in-faiz-ganj/https://www.quester.pk/question/does-telenor-have-3g-coverage-in-faisalabad/Hello Mr Richard,

ReplyDeleteThank you for your posts. I don't really understand much since i'm not good at maths.

Can you help me simplify the calculation of bayes factor from the following problem?

I run Beast using HIV dataset testing two clock model Relax Clock and Strict Clock. So from result of log file from the Beast which can views in tracer, how can i calculate bayes factor to compare these two model?

Thank you in advance for your help.

Sophearot (from Cambodia)

Also there should be high amount of transparency as to where the software stands in terms of quality, field service software

ReplyDeleteTesting makes your customer to get full satisfcation on your service since it found out all the bugs and errors

ReplyDeleteand rectify it. Selenium is the best tool ever to test web based applications. Thank you for your information.

Regards:

Selenium Training in chennai |

Selenium Training

Can I just say what a relief to discover somebody that really knows what they're discussing online. You definitely understand how to bring a problem to light and make it important. You can review my latest post about ms office More people should check this out and understand this side of your story. I can't believe you are not more popular given that you definitely possess the gift.

ReplyDeleteVery useful content

ReplyDeleteHadoop Training In Chennai | Sap MM Training In Chennai | ETL Testing Training In Chennai

Gta 5 Mobile Download is best site for download gta 5 apk

ReplyDeleteGet intopc for best pc software you can get here

ReplyDelete

ReplyDeleteGta 5 Mobile Dwgamez for best pc software you can get here

ReplyDeleteMinecraft pe Apk

This comment has been removed by the author.

ReplyDeleteThis comment has been removed by the author.

ReplyDeleteHey, Thanks for such a nice share. You can download windows 7 lite

ReplyDeleteThanks for sharing this details.

ReplyDeleteOnline ERP SoftwareThe very next time I read a blog, Hopefully it does not fail me just as much as this one. I mean, Yes, it was my choice to read, nonetheless I really believed you would probably have something helpful to talk about. All I hear is a bunch of moaning about something you could possibly fix if you weren't too busy seeking attention.

ReplyDeleteGet the Windows 10 Gamer Edition

What are the features of the kindle? This article talks about top six features of the wireless reading device, Kindle Fire. Kindle adbfire download enables user to surf the internet, listen to music, watch movie, read books and etc. Here are the top six features of the Kindle Fire that I am going to share with you.

ReplyDeleteThis is most informative and also this post most user friendly and super navigation to all posts... Thank you so much for giving this information to me..

ReplyDeleterpa online training |

rpa course in bangalore |

rpa training in bangalore |

rpa training institute in bangalore

Read all the information that i've given in above article. It'll give u the whole idea about it.

ReplyDeletepython course in pune | python course in chennai | python course in Bangalore

I found your blog while searching for the updates, I am happy to be here. Very useful content and also easily understandable providing.. Believe me I did wrote an post about tutorials for beginners with reference of your blog.

ReplyDeleteexcel advanced excel training in bangalore | Devops Training in Chennai

Good Post! Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge as updated one, keep blogging.

ReplyDeleteexcel advanced excel training in bangalore | Devops Training in Chennai

I found this informative and interesting blog so i think so its very useful and knowledge able.I would like to thank you for the efforts you have made in writing this article.

ReplyDeleteangularjs Training in marathahalli

angularjs interview questions and answers

angularjs Training in bangalore

angularjs Training in bangalore

angularjs Training in chennai

automation anywhere online Training

Good Post! Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge as updated one, keep blogging.

ReplyDeletepython course in pune

python course in chennai

python course in Bangalore

I am sure this post has helped me save many hours of browsing other related posts just to find what I was looking for. Many thanks!

ReplyDeleteCore Java interview questions and answers

Java training in Chennai | Java training in Tambaram

Java training in Chennai | Java training in Velachery

Java training in Chennai | Java training in Omr

Well you use a hard way for publishing, you could find much easier one!

ReplyDeleteData Science course in Indira nagar

Data Science course in marathahalli

Data Science Interview questions and answers

Data science training in tambaram | Data Science Course in Chennai

Data Science course in btm layout | Data Science training in Bangalore

Data science course in kalyan nagar | Data Science Course in Bangalore

All the points you described so beautiful. Every time i read your i blog and i am so surprised that how you can write so well.

ReplyDeleteJava training in Chennai | Java training institute in Chennai | Java course in Chennai

Java training in Bangalore | Java training in Electronic city

Java training in Bangalore | Java training in Marathahalli

Java training in Bangalore | Java training in Btm layout

Inspiring writings and I greatly admired what you have to say , I hope you continue to provide new ideas for us all and greetings success always for you..Keep update more information.

ReplyDeleterpa training in chennai

rpa training in bangalore

rpa course in bangalore

best rpa training in bangalore

rpa online training

I am really happy with your blog because your article is very unique and powerful for new reader.

ReplyDeleteClick here:

selenium training in chennai | selenium course in chennai

selenium training in bangalore | selenium course in bangalore

selenium training in Pune | selenium course in pune | selenium class in pune

selenium training in Pune | selenium course in pune | selenium class in pune

selenium online training | selenium training online | online training on selenium

Nice tutorial. Thanks for sharing the valuable information. it’s really helpful. Who want to learn this blog most helpful. Keep sharing on updated tutorials…

ReplyDeletepython training institute in marathahalli

python training institute in btm

Python training course in Chennai

We are a group of volunteers and starting a new initiative in a community. Your blog provided us valuable information to work on.You have done a marvellous job!

ReplyDeleteData Science course in Indira nagar

Data Science course in marathahalli

Data Science Interview questions and answers

Data science training in tambaram | Data Science Course in Chennai

Data Science course in btm layout | Data Science training in Bangalore

Data science course in kalyan nagar | Data Science Course in Bangalore

It is amazing and wonderful to visit your site.Thanks for sharing this information,this is useful to me...

ReplyDeleteBest Devops training in sholinganallur

Devops training in velachery

Devops training in annanagar

Devops training in tambaram

I am really happy with your blog because your article is very unique and powerful for new reader.

ReplyDeleteClick here:

selenium training in chennai | selenium course in chennai

selenium training in bangalore | selenium course in bangalore

selenium training in Pune | selenium course in pune | selenium class in pune

selenium training in Pune | selenium course in pune | selenium class in pune

selenium online training | selenium training online | online training on selenium

That was a great message in my carrier, and It's wonderful commands like mind relaxes with understand words of knowledge by information's.

ReplyDeleteData Science Training in Chennai | Best Data science Training in Chennai

Data Science training in anna nagar | Data science training in Chennai

Data Science training in chennai | Best Data Science training in chennai

Data science training in Bangalore | Data Science training institute in Bangalore

Data Science training in marathahalli | Data Science training in Bangalore

Data Science interview questions and answers

I likable the posts and offbeat format you've got here! I’d wish many thanks for sharing your expertise and also the time it took to post!!

ReplyDeleteangularjs Training in bangalore

angularjs Training in bangalore

angularjs online Training

angularjs Training in marathahalli

angularjs interview questions and answers

ReplyDeleteGreetings. I know this is somewhat off-topic, but I was wondering if you knew where I could get a captcha plugin for my comment form? I’m using the same blog platform like yours, and I’m having difficulty finding one? Thanks a lot.

AWS Training in Pune | Best Amazon Web Services Training in Pune

Amazon Web Services Training in OMR , Chennai | Best AWS Training in OMR,Chennai

AWS Training in Chennai |Best Amazon Web Services Training in Chennai