The Interpreter; Bayes Theorem; Nephites and Mayans

[_Res Ipsa]

PG, I don’t think it’s that subtle. The Maya was not written as a rebuttal to Book of Mormon apologists and their parallelomania. There is no reason that we should expect a large percentage of “facts” in a fictional Book of Mormon to be addressed in The Maya. So, by restricting their analysis to instances where the assertion is mentioned in both books, they’ve likely excluded a substantial amount of evidence that the book is fiction.

But it’s even worse than that: the comments state that the went through the Book of Mormon, found all factual claims, then looked to see if the subject of the claim is discussed. But they failed to do the opposite, which also excluded an entire category of evidence against a historical Book of Mormon — facts about the Maya that we would expect to find in a historical Book of Mormon.

The paper tacitly recognizes how methodologically wrong this restriction was by violating to include statements by Coe that were made in contexts other than The Maya. Had they not done so, their analysis would simply have ignored many significant by Coe himself. That should have been a huge red flag to the Dales that their methodology was flawed. But they just made an ad hoc special exception to their own stated methodology, which covered up a glaring problem with it.

The paper’s methodology is a rigged game, and in more ways than this.

[_Res Ipsa]

PG, get a copy of the Maya. Start listing facts from The Maya that are not found in the Book of Mormon. Stop when you get to 131 and note the percentage of The Maya that is left.

[_Gadianton]

An independence aside, as I’m trying to understand how they understand independence: they count silver and gold together, suggesting dependence because they are mined together; but yet they count iron and steel as separate misses when steel depends on iron, if you have steel you have iron.

[_Water Dog]

The apparent source for the Dales' particular set of allowed likelihood ratios is an interesting detail that might be worth adding.

I see it as relevant because, if I'm right that this is what they did, then it means their input probabilities aren't merely arbitrary but the situation is much worse than that. Using their dice rolling example, it would be like stating from the onset that you want "snake eyes" and then manually placing the dice as needed to reach the goal. In doing so, regardless of whatever independence may legitimately exist between the defined events, they create a new mathematical dependency by deriving their inputs in such a fashion.

Event Probability Model: [Subjective Feelings About Evidence's Significance] ==> [Jeffreys K Table In Reverse] ==> [Competing Hypothesis Probabilities]

Say you have an equation, A + B = C. A and B "could" be independent variables. C, however, by definition, depends on A and B. If you then go backwards, have C as your input, from which you model A and B, those variables are no longer independent.

The probabilities are derived from a model. And therefore are themselves dependent on said model. They create a new form of codependency that is not in any way accounted for later.

[_Res Ipsa]

Way upthread I asked whether the Dales were doing a Bayesian analysis at all. I think the answer is clear: not at all.

Yesterday I carefully went through the comments to the paper (taking screenshots of the ones the shed more light on the Dales' methodology) and read the blog entry. I also read the parts of Carrier's "Proving History" where he makes the case that Bayes is a useful tool for answering historical questions. This morning I realized that there was a step in the Dales' analysis that I hadn't noticed before -- a step that shows quite clearly that the Dales paper does not perform a Bayesian analysis.

The paper says:

To perform our analysis, we assign one of three likelihood ratios to testable facts or “correspondences” between the Book of Mormon and Dr. Coe’s book. The facts, taken from Dr. Coe’s book, are compared with statements of fact in the Book of Mormon. Recall that the hypothesis we are testing is that the Book of Mormon is false, and we assume a billion to one prior odds in favor of the hypothesis that the Book of Mormon is indeed false.

Pieces of evidence in favor of the hypothesis, that is, that the Book of Mormon is false, are weighted by their “likelihood ratio,” which is a positive value greater than one (either 50, 10 or 2). This likelihood ratio is multiplied by the skeptical prior of a billion to one to increase the weight of the evidence against the Book of Mormon.
Points of evidence in favor of the essentially factual nature of the Book of Mormon (called the converse hypothesis) are weighted by their likelihood ratio, a positive decimal fraction (0.5, 0.1 or 0.02). These fractions are multiplied by the skeptical prior of a billion to one to decrease the weight of the evidence against the Book of Mormon, in other words, to provide evidence for the factual nature of the Book of Mormon.

Do you see the missing step? Where do they describe the methodology for determining whether a specific correspondence is a "hit" or a "miss." They don't because they don't use any methodology. It's a completely subjective decision made in a black box.

Why is this important. In Proving History, Carrier describes two major benefits of using Bayes Theorem. One is the explicit consideration of the prior probability. The other is the process of forcing the author to carefully choose the right questions, to keep assumptions within the realm of reasonable, and to make assumptions explicit. Where does that happen? In the Dales' analysis, it should happen in formulating the questions to be asked in determining the numerator and denominator of the LR.

Given that there were no horses in mesoamerica at the relevant time:

Assuming the Book of Mormon is fiction, what is the likelihood that it would mention horses in the way that it does?
Assuming the Book of Mormon is history, what is the likelihood that it would mention horses in the way that it does?

Going through this process helps avoid a couple of traps identified by Carrier that present themselves in historical debates: (1) treating possibility as probability; and (2) interpreting the evidence to fit the theory you think is right.

The second one is extremely important because it is the original sin of Apologetics.

If you do the analysis correctly, there is no need to sort the correspondences into hits and misses as the first step in the analysis. You simply answer the two questions, perform the division, and the LR tells you whether the correspondence is a hit or a miss. But the Dales do this as a hidden step that is never explained.

Now, the Dales might respond that the difference between a hit and a miss is obvious, but let's revisit one of their pieces of evidence:

Fundamental level of political organization is the independent city-state

Coe’s standard: “Sylvanus Morley had thought that there was once a single great political entity, which he called the ‘Old Empire,’ but once the full significance of Emblem Glyphs had been recognized, it was clear that there had never been any such thing. In its stead, Mayanists proposed a more Balkanized model, in which each ‘city state’ was essentially independent of all the others; the political power of even large entities like Tikal would have been confined to a relatively small area, the distance from the capital to the polity’s borders seldom exceeding a day’s march” (p. 274).

Book of Mormon correspondence: Throughout the Book of Mormon itself there is never a reference to “Nephite nation” or to a “Lamanite nation.” Interestingly, the word nation is used in reference to the Jaredites (Ether 1:43), a very different people culturally than the Lehites. The Book of Mormon uses this phrase: “nations, kindreds, tongues and people.” The Nephites and Lamanites were clearly kindreds. In contrast, the word nation is used frequently in terms of the “nations of the Gentiles.” The noncanonical Guide to the Scriptures has eight references to “Nephite nation,” showing how deeply engrained this idea of nationhood is in modern readers. But the Book of Mormon never puts those two words together for Nephite/Lamanite societies. The nation-state is not a political structure found anywhere in the Book of Mormon. Instead, the Book of Mormon peoples were organized politically in city-states. Often one city-state would dominate a group of other city-states. This dominance is the subject of the next correspondence

Analysis of correspondence: The correspondence is specific and detailed. There is not a single reference in the text of the Book of Mormon to “Nephite nation” or “Lamanite nation.” It is also unusual. Joseph Smith was growing up in the new nation of America, with a great deal of pride and self-identity as an independent nation. How did he avoid identifying the Lamanite or Nephite peoples as “nations”? But he did avoid it. What a lucky “guess” — over and over again during the course of the Book of Mormon history.

Likelihood = 0.02

Let's apply the methodology: Coe says the Mayans were organized in independent City States. The Book of Mormon does not mention independent City States. Under the Dale's methodology, this should not have been included. Using the proper methodology, it would be at least slightly more likely that a historical Book of Mormon would mention City States than would a fictional one. So, performing an actual Bayesian analysis, this correspondence would be at least weak evidence in favor of a fictional Book of Mormon, and the Dales should have assigned it a value of 2 under their methodology.

How did the Dales manage to transform weak evidence in favor of fiction to weak evidence in favor of history? By falling into Carrier's traps. They started out by trying to explain how a failure to mention the correct Mayan form of government could be interpreted to mean that the Book of Mormon was saying that its people had the same form of government. In other words, they started by explaining the evidence to make it fit their preferred theory, treated that possibility as a probability, and decided the correspondence was evidence of history, skipping the actual analysis required under a Bayesian method.

Every step in the Dales' analysis is wrong, and lots of it is backwards (hat tip to Water Dog). And all of the error favor what they want to be true.

[_Res Ipsa]

The apparent source for the Dales' particular set of allowed likelihood ratios is an interesting detail that might be worth adding.

I see it as relevant because, if I'm right that this is what they did, then it means their input probabilities aren't merely arbitrary but the situation is much worse than that. Using their dice rolling example, it would be like stating from the onset that you want "snake eyes" and then manually placing the dice as needed to reach the goal. In doing so, regardless of whatever independence may legitimately exist between the defined events, they create a new mathematical dependency by deriving their inputs in such a fashion.

Event Probability Model: [Subjective Feelings About Evidence's Significance] ==> [Jeffreys K Table In Reverse] ==> [Competing Hypothesis Probabilities]

Say you have an equation, A + B = C. A and B "could" be independent variables. C, however, by definition, depends on A and B. If you then go backwards, have C as your input, from which you model A and B, those variables are no longer independent.

The probabilities are derived from a model. And therefore are themselves dependent on said model. They create a new form of codependency that is not in any way accounted for later.

Water Dog, I didn't really appreciate the impact of this until I found the two papers and read them. I knew that limiting the LR's two those three values was wrong, but their rationale for using those numbers is just crazy.

[_Physics Guy]

I don’t think it’s that subtle.

I wanted to count it as subtle because I didn't think of it. So I figured I could just assign it a subtlety ratio of 50.

[_Analytics]

One more insight.

The Dales repeatedly ask, "What are the chances Joseph Smith could have guessed this about the Mayans?"

Here is a first attempt to answer that question.

The computer game Q20 is a 20-questions game. The player thinks of anything specific thing they want. The game then asks a series of yes/no questions. Within 20 questions, the computer guesses what you were thinking of. It is exactly right about 85% of the time. It's pretty amazing how good it is.

The 20Q game has a database of 10,000 "things" that it will guess you are thinking of. Keeping that in mind...

I created a model where each book describes 2 "things" per page. Each "thing" that the theoretical book describes is drawn from any one of 10,000 specific things in the universe of things the book could be about. The Book of Mormon has 600 pages, The Maya has 320 pages, and the View of the Hebrew has only 222 pages.

With that as a base, by pure coincidence the Book of Mormon would be expected to correctly describe about 69 "things" that are also in The Maya. The View of the Hebrews, being a much smaller book, would only be expected to describe about 30 things.

Note the following:

First, the Dales found 131 hits--significantly more than the 69 hits I would have expected. This can be explained by a couple of things. First, The Maya was selected because it is the best hit for the Book of Mormon--not because it was a random culture from the past. Second, the authors had a pretty strong bias on what they considered a hit.

Second, according to the Dales, the View of the Hebrews had 17 hits--fewer than the 30 I would have expected. This can be explained by the authors bias against View of the Hebrews.

Third, this shows that their "control" of the View of the Hebrews is invalid--since View of the Hebrews is a shorter book, we would expect a guesser to make fewer correct guesses. This has no bearing on whether or not the Book of Mormon is historical.

Finally, if the author was in fact guessing, he would make some correct guesses over so many pages. This simplistic analysis illustrates that the number of guesses Joseph Smith allegedly got right is at least on the same order of magnitude of how many we would expect him to get right.

[_Res Ipsa]

One more insight.

The Dales repeatedly ask, "What are the chances Joseph Smith could have guessed this about the Mayans?"

Here is a first attempt to answer that question.

The computer game Q20 is a 20-questions game. The player thinks of anything specific thing they want. The game then asks a series of yes/no questions. Within 20 questions, the computer guesses what you were thinking of. It is exactly right about 85% of the time. It's pretty amazing how good it is.

The 20Q game has a database of 10,000 "things" that it will guess you are thinking of. Keeping that in mind...

I created a model where each book describes 2 "things" per page. Each "thing" that the theoretical book describes is drawn from any one of 10,000 specific things in the universe of things the book could be about. The Book of Mormon has 600 pages, The Maya has 320 pages, and the View of the Hebrew has only 222 pages.

With that as a base, by pure coincidence the Book of Mormon would be expected to correctly describe about 69 "things" that are also in The Maya. The View of the Hebrews, being a much smaller book, would only be expected to describe about 30 things.

Note the following:

First, the Dales found 131 hits--significantly more than the 69 hits I would have expected. This can be explained by a couple of things. First, The Maya was selected <I>because</I> it is the best hit for the Book of Mormon--not because it was a random culture from the past. Second, the authors had a pretty strong bias on what they considered a hit.

Second, according to the Dales, the View of the Hebrews had 17 hits--fewer than the 30 I would have expected. This can be explained by the authors bias against View of the Hebrews.

Third, this shows that their "control" of the View of the Hebrews is invalid--since View of the Hebrews is a shorter book, we would expect a guesser to make fewer correct guesses. This has no bearing on whether or not the Book of Mormon is historical.

Finally, if the author was in fact guessing, he would make some correct guesses over so many pages. This simplistic analysis illustrates that the number of guesses Joseph Smith allegedly got right is at least on the same order of magnitude of how many we would expect him to get right.

Analytics, I was just about to post this howler when I saw your entry:

We believe that facts with a 2% likelihood (one in 50 chance) are essentially impossible to guess correctly, given any amount of knowledge or study reasonably available to the writer of the Book of Mormon.

How can anyone with even a rudimentary knowledge of probability make this kind of statement? Pop quiz: If I start making guesses, each of which have a 2% chance of being correct, how many guesses must I make before there is a 50% chance that I've guessed one correctly? And the Dales characterize this as impossible? And this got past a statistician who peer reviewed the work?

[_Water Dog]

How can anyone with even a rudimentary knowledge of probability make this kind of statement? Pop quiz: If I start making guesses, each of which have a 2% chance of being correct, how many guesses must I make before there is a 50% chance that I've guessed correctly? And the Dales characterize this as impossible? And this got past a statistician who peer reviewed the work?

Someone should create a GoFundMe to send them a copy of Freakonomics.