Well, the Warriors lost last night, the series is tied up, and the next game is Wednesday!
I wanted to expand on the issue with the use of Bayesian factors I mentioned earlier. The paper notes that the method they use can also be used diagnostically to evaluate medical test results, and in fact, the factors they choose (2, 10, 50, and 1/2, 1/10, 1/50) seem to come from the medical literature where those factors are used as shortcuts to quickly approximate changes in posterior odds after tests.
( For example, see
Simplifiying Likelihood Ratios )
As an example, consider getting a negative test (T-), when you could have the disease (D+) or not have the disease (D-). The likelihood ratio would be as follows:
p(T- | D+) / p(T- | D-)
The numerator is the probability that the test comes back negative when you have the disease.
The denominator is the probability the test comes back negative when you don't have the disease.
Note also that both probabilities, by definition, range from 0 to 1, inclusive.
In order for that ratio to have a value that is greater than 1, (like the 2, 10, or 50 from the article),
the denominator would have to be less than 1. In other words, it must be possible that not having the disease could result in a negative test, or it could result in a positive test, such that testing negative without having the disease does not happen with a probability of 1.
Suppose the probability of getting a negative test when you have the disease is 60%. If the denominator was less than that, say 40%, then the ratio would be:
.6 / .4 = 1.5
If the denominator was equal to 1, then there is no value of the numerator (since it has to be between 0 and 1 inclusive), that would give a ratio equaling greater than 1.
Now let's apply this to the paper. Here is their definition of the likelihood ratio:
This likelihood ratio is the strength of each individual statement of fact as a piece of evidence. It is calculated as the probability that the statement is true if whoever wrote the Book of Mormon was guessing divided by the probability that the statement is true if instead the Book of Mormon is fact-based and essentially historical.
Therefore the ratio can be written as:
P(B|A) / P(B|~A),
Where A is defined in the paper as the hypothesis that the Book of Mormon is fictional;
~A is the hypothesis that the Book of Mormon is not fictional.
Also from the paper:
First, the Bayes factor specifically accounts for the possibility that the evidence may have occurred under the other hypotheses. This is accomplished in the denominator of the Bayes factor.
This is like the diagnostic ratio, in that the authors are saying that elements B (like T-) could occur if the Book of Mormon is fictional (like D+), or elements B could occur if it is historical (like D-)
Now we come to the problem.
For the authors to use this likelihood ratio, they must consider elements B that have the same properties as the T-, or negative test. In other words they need to test elements B that are both factual and nonfactual.
For example, elements B could be every statement in the Book of Mormon, both those known as factual, those known as nonfactual, those not known if they are either, etc. Every element needs to be part of the experiment. Instead, the authors limit their testing to only 131 pieces of evidence, which they define as true statements of fact. From the paper:
If the Book of Mormon is fiction, then its author was guessing every time he wrote as fact something about the ancient inhabitants of the Americas.
This means we can compare reasonably these “guesses” in the Book of Mormon with the facts presented by Dr. Coe in The Maya.
[bolding added]
If we compare this back to the medical testing, that would be like saying they are only considering negative tests results that occur when a person actually did not have the disease.
It would be like not considering the possibility that negative tests occur when the person has the disease, such that
p(T- | D-)
always equals 1.
Or in the case of the paper, the authors are picking out only elements B that correspond to factual elements B in The Maya. This means that the denominator of their likelihood ratio;
P(B | ~A)
is the probability that a factual statement is a fact, given that Book of Mormon factual statements are facts.
It will always be 1, by virtue of limiting the analysis to elements B that will always be facts that would correspond to facts if the Book of Mormon were factual.
Therefore their testing will always favor their hypothesis (~A) that the Book of Mormon is NOT fiction. Even in the few cases where they assign a ratio value of 2, 10, or 50, they are doing so by violating their own definitions.
Throughout the thread, several people have commented about issues such as limiting the scope to the 131 facts is biased, or that additional elements from the Book of Mormon should have been considered, or that the assigning of ratio values is skewed. These issues are all related to this mis-use of Bayesian factors.
In my opinion, this paper has severely mis-applied the concept of using likelihood ratios based upon Bayesian principles to update probabilities.
I would have to agree, stak, it is definitely their own thing.