The Interpreter; Bayes Theorem; Nephites and Mayans

[_SnapDragon]

Philo Sofee,

Richard Carrier's use of Bayes’ Theorem has some of the same problems as the Dale’s. He has no idea how likely any of the scenarios he describes really are, so he pulls numbers out of his ass. You cannot use rigorous math with pretend numbers and end up with anything meaningful.

[Anonymous]

The interesting thing is, every tie I explained Bayes Theorem to people from RIchard Carrier's book Proving History, people were saying even he didn't know what he was talking about. His book on Jesus, one of the most fascinating I have ever read, whether you agree with it or not, it is utterly incredible to read, the use of Bayes in it has also been said to be just outright wrong. Soooooo, I literally do not know whether Bayes has ever been used correctly or not. And anyone who writes about it will be told the same thing, they are doing it wrong. I wonder if anyone actually knows the right way to apply Bayes to any other subject than medical.

Hi Philo,

If this is helpful or not, here is my interpretation of this. Bayes first began with a simple identity about conditional probabilities:

P(A|B) P(B) = P(B|A) P(A)

Then, he divided both sides by P(B). Simple algebra. Yawn. The creative genius comes in the way the terms were then interpreted. There are multiple ways of doing this, but a general, colloquial way of interpreting the terms is:

B: the sum total of the evidence
P(B): the probability we'd see all of the evidence we see, across all models
A: a specific model of the problem
P(A): the prior belief that model "A" is true
P(A|B): the probability model A is true, given that we did in fact see evidence B
P(B|A): the probability we'd see basket of evidence B given that model A is true

The key insight in this is that if you want to answer the question "What is the probability model A is true given that we have all this evidence B?" You need to first have an a priori belief about what the probability of A is. That is a really disturbing result, but if you understand the math, it is unavoidable. People have made very detailed studies of it. For example, people have thrown complicated, detailed formulas at this where the space of all possible "models" is a function with an infinite domain, and where the probability of A is given by a Gamma distribution rather than a Boolean distribution. You can make Bayesian reasoning very esoteric in a completely strict way.

In general there are different ways of interpreting probabilities, and really understanding what it "means" in the real world is tough. And that's just talking about probabilities. What is the probability the Dow will go up today? What does that question even mean? Those are tough questions. When you add Bayes' formula to this, including different ways Bayes' formula can be interpreted, it gets even tougher. Personally, I don't think using Bayes' theorem "correctly" is really a thing. Bayes' theorem is a model. It is a framework for addressing questions about what the probability of A is given that we have evidence B.

If we are talking about abstract mathematical problems, there are right and wrong answers. But if we are applying the math to the real world, what we are really doing is trying to gain insights about reality by modeling it with math. Models can be misleading. They can confuse the issue. They can be used to facilitate confirmation bias. That is all true about models in general, and exponentially truer with statistical models. And at best, they can be insightful. But in all cases, the model is based on a series of assumptions. The mathematical implications of the assumptions needs to be understood, and how reasonable they are given the real world problem also needs to be understood.

Personally, I wouldn't say that the Dales used Bayes' incorrectly. Rather, I would say that they didn't seem to understand all of the assumptions they were implicitly making, and were incredibly biased and sloppy in how they assigned probabilities.

My opinion is that Carrier understood the math and that his reasoning process was valid. I doubt people who say his use of Bayes is "just outright wrong" know what they are talking about.

To triangulate the philosophical implications of Bayes and how it can (and whether it should) be used when we are informally looking at evidence and trying to figure out what is true, the following books have chapters on Bayes that I would highly recommend:

The Signal and the Noise by Nate Silver
The Big Picture by Sean Carroll

If you read what those two say about Bayes, you should be comfortable that Carrier is on the right track.

[_Analytics]

Philo Sofee,

Richard Carrier's use of Bayes’ Theorem has some of the same problems as the Dale’s. He has no idea how likely any of the scenarios he describes really are, so he pulls numbers out of his ass. You cannot use rigorous math with pretend numbers and end up with anything meaningful.

The difference between Carrier and the Dales is that Carrier framed the evidence in a more careful way, spent a lot of time discussing it and justifying his numbers, was very careful about the implications of covariance, and then provided a wide range to the probabilities. In the end, he said the probability that Jesus was historical was probably somewhere in the range of very small to 33%.

If you want to address the question, "what is the probability that the Book of Mormon is historical" or "what is the probability that Jesus was a historical figure", you have to pull numbers out of your ass. The difference between Carrier and the Dales is in the quality of the analysis supporting their conclusions. The Dales were sloppy and arrogant. Carrier was meticulous. Carrier is arrogant in his own way, of course, but by comparison to the Dales, he is downright humble.

p.s. Welcome to the board!

[_Res Ipsa]

I think a good case can be made that they didn’t apply Bayes at all. Nowhere did they do what Bayes requires: assessing the likelihood of each piece of evidence under each hypothesis. If you do that for the evidence in the appendix, it’s easy to see that, whatever they were doing, it wasn’t Bayesian.

[_Analytics]

I think a good case can be made that they didn’t apply Bayes at all. Nowhere did they do what Bayes requires: assessing the likelihood of each piece of evidence under each hypothesis. If you do that for the evidence in the appendix, it’s easy to see that, whatever they were doing, it wasn’t Bayesian.

That is sort-of what they were doing with the likelihood ratios. Granted, they consider the question, "what is the probability Joseph Smith would guess X" a lot more than they seriously considered the question, "what is the probability somebody from ancient Mesoamerica would write X." I would agree they didn't seem to understand what the likelihood ratios really meant in the context in which they were using them.

To me at least, saying that they "didn't apply Bayes at all" is like saying England's famous Reliant Robin wasn't "really a car at all." It was a ill-conceived, poorly designed car. But it was still a car.

https://www.youtube.com/watch?time_cont ... Qh56geU0X8

[_Water Dog]

I think a good case can be made that they didn’t apply Bayes at all. Nowhere did they do what Bayes requires: assessing the likelihood of each piece of evidence under each hypothesis. If you do that for the evidence in the appendix, it’s easy to see that, whatever they were doing, it wasn’t Bayesian.

Analytics long post above is spot on. Res, the problem with your quibble is it's like debating whether Mormons are Christian or not. Depends on what we're talking about. Mormons cite Christ. Mormons follow a reworked Christian theology. But if you talk to other Christians they are pagans that worship false gods. Bayes is a misunderstood term that has become a buzzword, similar to others like "machine learning" or "big data." People toss these terms out without knowing what they mean to add smart sounding legitimacy to their argument. In truth they are broad terms and don't mean a whole lot. Bayesian simply means that the formula created by Thomas Bayes was used in some manner. And they are, so it's Bayesian.

[_Res Ipsa]

I think a good case can be made that they didn’t apply Bayes at all. Nowhere did they do what Bayes requires: assessing the likelihood of each piece of evidence under each hypothesis. If you do that for the evidence in the appendix, it’s easy to see that, whatever they were doing, it wasn’t Bayesian.

Analytics long post above is spot on. Res, the problem with your quibble is it's like debating whether Mormons are Christian or not. Depends on what we're talking about. Mormons cite Christ. Mormons follow a reworked Christian theology. But if you talk to other Christians they are pagans that worship false gods. Bayes is a misunderstood term that has become a buzzword, similar to others like "machine learning" or "big data." People toss these terms out without knowing what they mean to add smart sounding legitimacy to their argument. In truth they are broad terms and don't mean a whole lot. Bayesian simply means that the formula created by Thomas Bayes was used in some manner. And they are, so it's Bayesian.

Thanks, WD. You and Analytics have persuaded me.

[_Lemmie]

I think a good case can be made that they didn’t apply Bayes at all. Nowhere did they do what Bayes requires: assessing the likelihood of each piece of evidence under each hypothesis. If you do that for the evidence in the appendix, it’s easy to see that, whatever they were doing, it wasn’t Bayesian.

Analytics long post above is spot on. Res, the problem with your quibble is it's like debating whether Mormons are Christian or not. Depends on what we're talking about. Mormons cite Christ. Mormons follow a reworked Christian theology. But if you talk to other Christians they are pagans that worship false gods. Bayes is a misunderstood term that has become a buzzword, similar to others like "machine learning" or "big data." People toss these terms out without knowing what they mean to add smart sounding legitimacy to their argument. In truth they are broad terms and don't mean a whole lot. Bayesian simply means that the formula created by Thomas Bayes was used in some manner. And they are, so it's Bayesian.

Thanks, WD. You and Analytics have persuaded me.

Now, slow down, RI! Your assessment was this:

I think a good case can be made that they didn’t apply Bayes at all.

I took that to mean a correct application of the principles underlying the Bayesian factor model they appropriated from the medical diagnostic literature.

In that sense, i agree that they did not apply the corresponding principles at all. For example, they set up a test where, by THEIR definition, false positives approach a probability of 1 and true negatives approach a probability of zero, then rather than run repeated incidences of the test to determine actual numbers, they simply assign three LR ratios and their inverses. These are numbers which would NOT be realistic results given their assumptions about false positives. The various ratios most definitely would not be limited to inverses of themselves given 1 denominator approaching 1 and the other approaching zero.

Then they multiply 131 + 18 test results; no medical analysis would EVER do that, and in fact the literature cautions that too many tests actually can weaken the observed effects of the true evidence. etc, etc.

So i agree with you, Res Ipsa, but I do like Analytics example of calling a car a car, so I'll say that if medical diagnostic likelihood ratio use can be represented by a Lamborghini, then the "Guessing and the Book of Mormon" likelihood ratio use is like one of those tiny cars you might win in a gumball machine, one where the wheels don't turn and it's made of plastic. Still a car, yes, but realistically no comparison.

[_Res Ipsa]

Thanks, Lemmie. I'm thinking about effective arguments to make in response to the paper. I'm persuaded that, even if that argument were technically correct, it's a weak argument that would distract from several very strong arguments. When I have several strong arguments, it's generally a mistake to throw in a couple of weaker ones.

[_Lemmie]

Thanks, Lemmie. I'm thinking about effective arguments to make in response to the paper. I'm persuaded that, even if that argument were technically correct, it's a weak argument that would distract from several very strong arguments. When I have several strong arguments, it's generally a mistake to throw in a couple of weaker ones.

To make sure I understand, are you suggesting that saying "the paper is incorrectly using a Bayesian factors model by making up numbers instead of presenting actual results, and that the made-up results are therefore invalidated" is one of the weaker arguments?

We will have to agree to disagree on that.