The Interpreter; Bayes Theorem; Nephites and Mayans

[_honorentheos]

I posted this to The Interpreter which is awaiting moderation. It was a bit of effort so I don't want to lose it incase it doesn't get posted. Some editing is included from the original.

Hi Dr. Dale,

First, thank you for taking the time to answer concerns and questions regarding your paper. Whatever else one may think, it certainly opened up a discussion that appears to have been wanting given the attention it has drawn. And unrelated to the paper, your professional background and accomplishments are highly commendable, and a service to future generations. In all cases, you deserve respect.

That said, I don’t wish to be seen hiding a dagger in my cloak, so to speak. So I’ll go quickly to my concerns with the paper, it’s methodologies and results. And attempt to be brief.

Concern 1: The skeptical prior is overcome automatically by simply finding and adding correspondences. The approach taken inevitably overcomes an arbitrarily determined likelihood that the Book of Mormon is fiction.

Given your methodology and the assigned likelihood ratios, if you were to assign all 131 of the correspondences the weakest probability it was based on knowledge rather than a guess (0.5 or a 1 in 2 likelihood), one only needs to propose a small handful of weak correspondences to overcome what you present as a strong skeptical prior. As you pointed out in your section on sensitivity analysis, that number appeared to be 17.

Concern 2: The correspondences selected to achieve the results did not need to demonstrate actual correspondence to be included.

The paper does little if anything to demonstrate the methods for identifying the criteria derived from The Maya for each correspondence and stating them in a way that could be used to determine if the Book of Mormon contains an objectively mapped corollary to be evaluated. I’ve noted this elsewhere in these comments, but since 1.1 is chronologically first in your paper it’s convenient to start with it when pointing to examples. In that example, the quote from Coe described Maya society as being formed into city-states without a centralized government over the whole of the Mayan people. It describes the approximate geography of the city-state polities as approximately the distance a person could travel in a day. These criteria, which seem objectively uncontentious if stated as such, aren’t what you used to compare the Book of Mormon with The Maya. Instead, you chose to focus on the absence of a single word being used in reference to the Nephites, “Nation”. Your methodology did not require the demonstration of actual correspondence between the source material and the Book of Mormon that maps in a objective manner. I’d argue it appears to fail on the terms you proposed, where the use of, “Nephites”, “Lamanites”, “the people of the Nephites” and other language used in the Book of Mormon serve the same purpose as would the use of, “nation.”

And that was determined to have a 1 in 50 chance Joseph Smith could have guessed it correctly, while it seems stepping back shows the Book of Mormon got it wrong rather than included an improbable guess.

Concern 3: The methodology constrains what Coe described as characteristics that applied to the Maya to things you believe serve as hits or misses.

Using your chosen example above, 3.12 Existence of opposites, could be discussed under concern 2 above, noting that dualism generically is found in most cultural creation myths and used to explain the universe for obvious reasons. Night/day, darkness/light, sun/moon, birth/death, growth/decay, summer/winter, planting/harvest, action/reaction, Yin/Yang, inhale/exhale, creation/destruction - human societies find paired opposites inherent in creation and have created narrative mythologies to explain them across continents and millennia. Eastern religions have these cycles deeply embedded in them. And its part of the Hebrew creation mythology that God the creator ordered the heavens and the earth, with a greater light ruling the day and a lesser light ruling the night, male and female created He his living creations. Cosmic dualism, or the idea that there is a war between good and evil, is also embedded in post-exilic teachings and was behind Cyrus the Great liberating the captive Hebrews when the Persians concurred the Babylonians. As a Zoroastrian, his concern was with good combating evil at cosmic scales. Yet what the Maya tz’ak describes is more of a "two-sides required to have a coin" concept. Of the items listed in 2 Nephi by the speaker, Lehi (a pre-exilic Hebrew if one accepts the book as history), the examples are philosophical concepts. Not natural pairings as listed in the excerpt from The Maya.

This raises multiple questions, not least of which is if it really deserves to be considered a “hit”? It’s inclusion as such is entirely contingent on your interpretation of it being one. And as noted in concern 1, accumulating only a handful of supposed hits no matter how tenuous would overcome the skeptical prior. If the methodology for determining something is a hit is essentially subjective, where does that leave the paper itself?

There are other concerns, such as the arbitrary limits placed on the likelihood ratios or missed correspondences (for example, why is something that simply didn’t happen in the New World such as metal armor used to decimate the Lamanites which should have left an archeological and cultural imprint on the western hemisphere not included? And if it were, why is it constrained to only having a 1 in 50 likelihood that it would be included in a potentially historical Book of Mormon? Why wouldn’t the likelihood that the Book of Mormon was both historical and mistakenly describes metal armor used and adopted being included not much, much lower bordering on improbable?) But there’s no need to overwhelm the discussion with too many points.

I do appreciate the discussion and your engagement. It's a topic that is close to many people for varying reasons, including those who may be skeptical or critical of the LDS Church's claims. Opening it up and making it possible is meaningful in its own right.

[_honorentheos]

So, inoculate against the intellectual dishonesty of cherry-picking by posting an article in The Mormon Interpreter that is based on 1) cherry-picked, dependent data that is improperly presented as independent, fully representative data, and 2) improper use of probabilities that come from medical testing models with properties that his models do not have.

Thanks for finding that, honor, it gives an interesting perspective to how this article developed.

http://www.ohthatcleverkidjosephsmith.com/?p=414

Great find. This really settles it I think. My original post reads as if it was written directly in response to this blog. Ironically, he even later uses the example of the odds that someone who received a positive cancer result actually has cancer.

The bottom line is these are newbs that don't know what they're doing. They drew off a knowledge of medical screening which was beginner level in terms of Bayes, without understanding the underlying math and concepts. They then attempted to shoehorn the Book of Mormon into the same setup/model, which is an out of context misapplication. Garbage in, garbage out.

A lot in here to unpack.

<lots of unpacking>

What Dale did is subjectively concoct a list of "evidence." He then subjectively assigned a list of K/LR values to each item in the list depending on whether he, subjectively, considered it "strong" or "weak" etc. And then he multiplied all that shiz together and got a very big faith promoting number. That's it. So, that being the case, the MI paper tells us nothing about the objective world. It tells us the subjective results of what's going on in Dale's brain and nothing more. So what he's come up with is a methodology for how other people can delude themselves into subjectively believing things which are objectively idiotic. 1) Create a list of stuff you think is cool. 2) Assign arbitrary K values to them based on scale of nerdiness. 3) Multiply together. 4) If number is greater than 50/50, keep paying money to the Mormon church. 5) If number is <50/50, evaluate social costs, return to #1 and add more cool things to list to get the number you want. 6) Repeat. 7) Bayes proves church is true!

Thanks for taking the time to unpack this more, Lemmie and Water Dog. Clearly the two of you picked up on the underlying issues immediately and have done wonderfully in helping more of us see what you could from the outset. I agree with DrW that it would be nice to provide a succinct rebuttal that lays out the issues and why they matter to the results though my impression from looking around online at where the paper is getting attention is that it's reception is largely predetermined. It looked like Dr. Coe responded to John Dehlin bringing it to his attention by saying he is too old for that crap, that he stands by his position, and anyone who wants to dig into it more should refer themselves to Mormon Stories. Not sure what they'll find beyond the interviews between John and Dr. Coe, but I guess that's a response. The faithful response seems to be any criticism is just sour grapes on the part of those who can't accept the Book of Mormon is true and the maths prove it. It's a reflection of the times we live in I guess.

[_aussieguy55]

May I ask you what your academic background is?Your grasp of all this is impressive. I just need to persevere to understand having only ever done stats in a psychology subject.

[Guest]

According to the Dales, they assume that Coe is 100% right about Mesoamerican history, so they're assuming that the chance of horses in Book of Mormon times is 0%. The 2% chance that appears in the analysis is: IF we grant that the Book of Mormon is a genuine ancient history of Mesoamerica, then there's still a 2% chance (according to the Dales) that it might describe horses

I agree with the first sentence. For the second sentence, the two scenarios you describe 1) language 2) moroni wrong, I agree in principle, except 2 is paradoxical because if Moroni wrong Book of Mormon is true history, so it comes down to language. In any of the hits against Book of Mormon, the Dales don't seem to allow for language flexibility "definitely says horses". so they contradict themselves. In the hits in favor of Book of Mormon, language is indeterminate, not inadequate, "specificity", and so they can maintain their assumption (and have every incentive to) that coe is 100% right without a contradiction. (their assumption of "unusual" also in play for the hits) For the misses, they assume a contradiction. However, if you read all of their examples of misses, (ill double check later for an example), they seem to relax the assumption that Coe is 100% right. Giving the benefit of the doubt, i went down the path of what they might have been thinking given small chance Coe might be wrong. However, we can throw everything I did out the window, I don't mind: but please, complete the picture you were painting: write down a ratio that equals 50, and explain what top and bottom means in a way consistent with Dale's verbal analysis.

The probabilities of horses in the two different true-Book and fake-Book universes are unrelated to each other

I totally agree, but it feels like they are linking those worlds, beginning with using Coe's book as a filter.

Any combination of numbers between 0 and 1 could have been used by the Dales as long as the ratio is 50.

My suggestion way back is to take this in two parts. part 1) forget about what the words on the page mean, show a ratio, or several ratios, that are numerical candidates for the number 50 that the dales provided. part 2) explain what the numbers might mean, consistent with the dale's verbal analysis.

Analytics did a great job showing what he intended to show, and at this point we've fulfilled part 1. Here are some example's of potential LR's a) .98/.02; b) .098/.002, c) .5/.01.

now comes part 2. show how the Dale's reasoning is consistent with ratios suggested so far. It should be the case that perhaps steel is c), horses are a) and brass is b). If the Dales describe a world consistent with how medical testing works, then we'd expect a mix of ratios that come up 50.

Edited: I haven't changed my post except to color one line above. I later concluded that it was wrong. In thinking about medical examples, i couldn't come up with a way to do what i imagined the use of Coe's book was doing.

[_Lemmie]

Analytics did a great job showing what he intended to show, and at this point we've fulfilled part 1. Here are some example's of potential LR's a) .98/.02; b) .098/.002, c) .5/.01.

Maybe I'm missing your point Gad, but the ratios you are using from analytics don't all equal 50, so why are you considering a and b?

From my earlier post here are three, including one you used

[for the LR = 50, the inverse ratios work,

1 / 0.02, or .5 / .01, or 0.05 / 0.001.

I'm pretty sure they did not calculate any of those, but are just picking easy levels of numbers that correspond to the medical literature, as they said this in a comment:

[_Res Ipsa]

The Dales' arbitrary limit on likelihood not only diminishes the power of strong evidence, it also increases the power of weak evidence.

Let's take the following correspondence between the books: both say that the people ate food. The likelihood ratio should be 1. This correspondence is not evidence at all. But under the Dales' methodology, this correspondence is assigned a value of .5. Something that has no evidential value at all is transformed into a piece of evidence that the Book of Mormon is twice as likely to be fact as fiction. In other words, any piece of evidence that is actually weaker than .5 is arbitrarily promoted to .5. Likewise with evidence that is weaker than 2.

[_Res Ipsa]

now comes part 2. show how the Dale's reasoning is consistent with ratios suggested so far. It should be the case that perhaps steel is c), horses are a) and brass is b). If the Dales describe a world consistent with how medical testing works, then we'd expect a mix of ratios that come up 50.

I don't think we can show how the Dales' reasoning is consistent with any given numerator and denominator. The paper doesn't show a single example of them assigning values to the two components of the LR. They simply limit the resulting number to one of 6 values and make ad hoc arguments for which of the values should apply, using ill-defined criteria that aren't applied consistently from example to example. Nothing in the paper, the comments or the blog allows us to reconstruct the values for the numerator or the denominator they used because they didn't attempt to place a value on them.

[_Gadianton]

Great question for a rest stop, Lemmie. I would prefer to use your numbers. Let’s begin there. I Can’t for the life of me understand how they could have come up with the words they wrote and get B and C in your example, can you explain how either of those would make sense as candidates given their words? A comes close, but A would be the same analysis for every “miss”, suggesting the data was contrived rather than calculated. If it were real data, we’d expect to see a mix of numerator and denominator data that ended up as 50, making for an interesting analysis that you actually needed a test for, and wasn’t obvious where it leads from the beginning. If you can explain your b and c in terms that make sense given the explanations of the authors such that they are candidates for an analysis that would be reasonable to assume dales performed, then my first point is falsified and I’m on the wrong path.

My next point would be to hold the one candidate that seems reasonable, your a), to further scrutiny. But how about I go there on the next rest stop; and it’s there I’ll answer your direct question. But that might not matter if you think what I wrote above is way off, so let’s see what you say about the above first.

[_Res Ipsa]

Thanks for posting the blog excerpts. To me, they highlight the fact that the Dale's have not given serious thought to how evidence actually works. They are convinced that the number of "evidences" is what matters. That's just flat out wrong.

Evidence can be defined something like this: A is evidence of B if and only if A increases the probability that B is true. That increase can be minuscule or it can be determinative. When analyzing evidence, the side with the most instances of evidence doesn't necessarily win. One piece of determinative evidence can outweigh many instances of contrary evidence. Probably the best practical example is the freeing of death row prisoners by the innocence project. All of these prisoners were convicted under an evidence standard of beyond reasonable doubt. But one single piece of evidence, DNA, overcomes all of the prosecution's evidence at trial and results in the prisoners being freed.

...

Stepping back, remember that the Dales put the prior probability of a fictional Book of Mormon at a billion to one. The valuation of the horse issue is 50 to 1. How did they conclude that the horse evidence is 8 orders of magnitude weaker than the totality of everything else we know about the Book of Mormon? Why do they reach that conclusion? They don't explain. They just take extremely strong evidence against the Book of Mormon and dramatically weaken it by putting an artificial limit of 50 on the likelihood ratios. The result is an overwhelming bias that overemphasizes the importance of weak evidence and underemphasizes the importance of strong evidence. That is, in part, what allows them to overwhelm a few, very strong instances of evidence that the Book of Mormon is fiction with many instances of weak evidence that the Book of Mormon is historical.

Res,

You should post this on The Interpreter's comments section. Dr. Bruce Dale did come back today to engage with some of the comments. I think your example above or something like it would go a long ways towards moving the conversation beyond the intrenched view that critics are intimidated by the scope of the results and "kicking against the pricks". Not that I'd know the difference, but I really thought you put this quite clearly and understandably.

Thanks, Honor. I think the comments section of the Interpreter website is wholly inadequate to lay out just how terrible this paper is. When individuals post serious criticisms one at a time, the authors don't even think through the impact of the criticisms before responding in a fairly rote fashion. They literally don't understand what the best of the critics are telling them. I think laying all the problems out in a single document would be the most effect rebuttal. One approach might be to describe what a proper approach to their study should look like, point out all the ways the study deviates from what should have been done, and illustrate the impact on the
Dales' conclusions.

[_Res Ipsa]

Here’s an illustration of the problem with the limits placed on the likelihood values:

Assign evidence E1 the strongest possible LR of 50.

How many instances of the weakest evidence would it take to offset this single instance of the strongest evidence?

Just 6. 50 * .5 *.5 *.5 * .5 * .5 * .5 = .78

So, 6 instances of the weakest possible evidence is more powerful than one instance of the strongest possible evidence?

That is what this paper boils down to.