The Interpreter; Bayes Theorem; Nephites and Mayans

[_Lemmie]

Hi Lemmie, I'm curious if you have thoughts on the issue brought up in the last couple of comments between me and Brant? In particular, that the Book of Mormon can be used to support contradictory positions in relation to specific points taken from Coe but the papers methodology determined the correlation warranted their lowest probability Smith was guessing. It seems this shows rather conclusively the paper failed to use Bayes theorem properly wouldn't it? My understanding of Bayes is it is intended for working with conflicting positions to work out the probability of competing hypothesis being true. When the evidence in the Book of Mormon forces a supporter to present qualifying statements saying there are alternative interpretations available to justify the paper placing a item as a hit in spite of conflicting evidence being present and justifiably read in the Book of Mormon yet the authors assigned it as so detailed, specific and unusual it is unlike[ly] due to guesswork, it seems the act of doing so is essentially conceding the paper failed at its stated purpose.

Sorry I didn't see this sooner, i have read yours but let me look back at his posts before answering.

Hi Honor, I went back and looked at Brant's comments in the context of your question. It was a little disappointing, because I really got the sense even stronger this time through his posts that he really doesn't agree with this paper but that he is obligated to be supportive, hence the oblique responses to your comments.

He made this comment below, and I think it is how he separates the issue:

Point 1: Internal consistency and external correspondence are fundamentally different. Fantasy authors work hard to create an internally consistent story, and some use mythological material from known cultures to give a recognizable flavor or context to the story. Attempting to correlate any of those to external sources (i.e. history/archaeology) is impossible beyond random connections. The question here isn’t that every possible scenario is chosen, but whether or not a specific region fits. Random connections are cherry-picking. The larger number of interrelated connections is beyond random. The issue of whether the article establishes those connections can be debated, but then that wasn’t the design of the question. It is hard to complain that the article didn’t do what it didn’t attempt.

It seems that in his mind, statement by statement, he can isolate the question of whether Joseph Smith could have guessed, while avoiding the big picture issue that the likelihood of these guesses being related is very strong because the statements themselves are related.

Of course, the fact that the authors simply assume that all these statements are independent, while not allowing commenters to discuss the issue of possible dependency, begs the question of independence rather egregiously.

[_Lemmie]

Great question for a rest stop, Lemmie. I would prefer to use your numbers. Let’s begin there. I Can’t for the life of me understand how they could have come up with the words they wrote and get B and C in your example, can you explain how either of those would make sense as candidates given their words? A comes close, but A would be the same analysis for every “miss”, suggesting the data was contrived rather than calculated. If it were real data, we’d expect to see a mix of numerator and denominator data that ended up as 50, making for an interesting analysis that you actually needed a test for, and wasn’t obvious where it leads from the beginning. If you can explain your b and c in terms that make sense given the explanations of the authors such that they are candidates for an analysis that would be reasonable to assume dales performed, then my first point is falsified and I’m on the wrong path.

My next point would be to hold the one candidate that seems reasonable, your a), to further scrutiny. But how about I go there on the next rest stop; and it’s there I’ll answer your direct question. But that might not matter if you think what I wrote above is way off, so let’s see what you say about the above first.

Ok. so here are my 3 possibles,

[for the LR = 50, the inverse ratios work,

1 / 0.02, or .5 / .01, or 0.05 / 0.001.

And three examples:

Example 1:

Using 0.5 / .01 in the author's likelihood ratio, the numerator would be :

P(statement not true, given Book of Mormon fiction) = 50%

And the denominator:

P(statement not true, given Book of Mormon non-fiction) = 1%


so authors are saying there is a 50% chance Joseph Smith would guess the false statement horses in a Book of Mormon that is fiction,

divided by a 1 % chance the wrong statement horses is in a Book of Mormon that is non-fiction.


Example 2:

If i use my first number, 1/ 0.02, then

P(statement not true, given Book of Mormon fiction) = 100%

And the denominator:

P(statement not true, given Book of Mormon non-fiction) = 2%


so authors are saying there is a 100% chance Joseph Smith would guess the false statement horses in a Book of Mormon that is fiction,

divided by a 2 % chance the wrong statement horses is in a Book of Mormon that is non-fiction.


Example 3:

And my third number, 0.05 / 0.001, then

P(statement not true, given Book of Mormon fiction) = 5%

And the denominator:

P(statement not true, given Book of Mormon non-fiction) = one-tenth of 1%


so authors are saying there is a 5% chance Joseph Smith would guess the false statement horses in a Book of Mormon that is fiction,

divided by a one-tenth of 1 % chance the wrong statement horses is in a Book of Mormon that is non-fiction.

That's what the authors are saying.

Now, my opinion:

I do not think it is reasonable, nor do I think they worked out the numbers. I think they arbitrarily assigned a scale of numbers to subjective statements. That's like saying that their subjective opinions of "good, better, and best" equal "2, 10, and 50."

Also, the denominator should be arbitrarily close to zero, given the authors assume, when talking about true statements, that all statements in Coe are fact true. They specifically state this in their article.

Dividing by a number that approaches zero gives a ratio that approaches infinity. Arbitrarily limiting the possibilities to 2, 10, and 50 does not allow for this.

I think the positive Likelihood ratios should be much much larger.

[_honorentheos]

Thanks, Lemmie. I appreciate your perspective and think you hit on Brant's position pretty much dead on.

Hi Honor, I went back and looked at Brant's comments in the context of your question. It was a little disappointing, because I really got the sense even stronger this time through his posts that he really doesn't agree with this paper but that he is obligated to be supportive, hence the oblique responses to your comments.

He made this comment below, and I think it is how he separates the issue:

Point 1: Internal consistency and external correspondence are fundamentally different. Fantasy authors work hard to create an internally consistent story, and some use mythological material from known cultures to give a recognizable flavor or context to the story. Attempting to correlate any of those to external sources (i.e. history/archaeology) is impossible beyond random connections. The question here isn’t that every possible scenario is chosen, but whether or not a specific region fits. Random connections are cherry-picking. The larger number of interrelated connections is beyond random. The issue of whether the article establishes those connections can be debated, but then that wasn’t the design of the question. It is hard to complain that the article didn’t do what it didn’t attempt.

It seems that in his mind, statement by statement, he can isolate the question of whether Joseph Smith could have guessed, while avoiding the big picture issue that the likelihood of these guesses being related is very strong because the statements themselves are related.

Of course, the fact that the authors simply assume that all these statements are independent, while not allowing commenters to discuss the issue of possible dependency, begs the question of independence rather egregiously.

[_Lemmie]

Re the authors multiplying numerous small LRs to get their astronomical result in favor of a true Book of Mormon, I looked into use of this technique and from a health Statistics lecture noted this:

How many outcome variables should I have? If we have many outcome variables:

.... the risk of false positives, finding an apparent effect when there is none in reality, would be increased.

The risk of false positives comes from the problem of multiple testing. Suppose that the treatment actually has no effect. If we carry out a single test of significance the probability of getting a significant difference would be 0.05, one in 20. If we carry out many tests of significance, the probability of getting a significant difference is much higher.

https://www-users.york.ac.uk/~mb55/msc/ ... sampsz.htm

in other words, stringing together coincidences and multiplying each small outcome to get a much larger overall result is considered meaningless.

[_Dr Exiled]

Drs. Dale need to admit to how they assigned the probabilities. They are purely arbitrary, without any basis other than hope and desire for an outcome. Dr. Sr. freely admits on his blog that he is acting like a biased defense counsel, that he cherry picks his inputs, that anyone can assign any value they want to the evidence and that he has a faithful, hopeful perspective. But in his article and his defense of it, he won't admit his conclusion based reasoning. Come on buddy. You make a more honest admission of your bias on your blog. Time to fess up. Your exercise was just hopeful, wishful views of the text in comparison to Dr. Coe and reality. Granted, Mormonism is forever possible in some universe, yet the probabilities weigh heavily against it.

What is the probability Drs. Dale made wishful thinking errors given Sr.'s bias admissions on his blog? Given disappearing DNA? What is the probability of misuse of Bayes, given the thorough thrashing that we've witnessed of this weak paper?

Sir, take down your gaslighting paper. There is no excuse in believing nonsense when it is passed on as scientific proof to the population at large.

[Guest]

thanks Lemmie.

so authors are saying there is a 5% chance Joseph Smith would guess the false statement horses in a Book of Mormon that is fiction

That part makes perfect sense. What I can't imagine is that any wording the authors actually use could possibly be construed as consistent with this sentence you wrote. It seems that every high LR miss is of the same kind, a "really high possibility that Joseph Smith would clearly guess this obvious, every day thing in a Book of Mormon that is fiction". For horses, brass, chariots, donkeys etc. it's the same analysis. In the cases of sheep and pigs, they lower to 10, because it isn't so clear as Joseph Smith only mentioned it once. So sure, perhaps they could mean there is a 5% chance Joseph Smith would guess the false statement of sheep, but not a 5% chance he'd guess a false statement of brass, and certainly I can't fathom a 5% chance he'd guess brass while a 100% chance he'd guess a false statement of horses.

All their variety for the high LRs is explained in the numerator. Nothing they say suggests to me that they could possibly mean: you know, there is a 100% chance he'd guess horses and a 5% chance he'd guess brass, but there is a much greater chance that coe could be wrong about horses and so the LRs are 50 in both cases. You imply so much also when you point out their denominator has no good explanation.

To answer your initial question, unfortunately for me, I convinced myself that I was wrong about a significant point I was making since the last rest stop when I posted.

The super short answer is i decided to allow any ratios that come close to 50 as valid for the sake of getting anyone to explain how you can have a variety of ratios in the high LRs given the dales description; that part I still stand by (for now).

Longer answer: But the detour that got me contemplating things like .98/.02, was in imagining ways that the dales were putting constraints on what they could mean by using Coe as a filter. What I meant in my response to PG was that 1/.02 is inconsistent with the dales reasoning. You pointed out the .02 gives them a freebie (as does PG), but I was taking it further, believing they were further constrained to lower the 1. After contemplating my cancer medical example a few pages back, I convinced myself that the dales use of Coe doesn't actually have the effect I believed. I think PG probably proved me wrong there but it wasn't obvious until I tried to constrain the cancer example in a similar way and it didn't make sense.

[_honorentheos]

Thanks, Honor. I think the comments section of the Interpreter website is wholly inadequate to lay out just how terrible this paper is. When individuals post serious criticisms one at a time, the authors don't even think through the impact of the criticisms before responding in a fairly rote fashion. They literally don't understand what the best of the critics are telling them. I think laying all the problems out in a single document would be the most effect rebuttal. One approach might be to describe what a proper approach to their study should look like, point out all the ways the study deviates from what should have been done, and illustrate the impact on the Dales' conclusions.

You're probably right, Res, and the format there alone is against having a coherent discussion beyond a post or two in a conversation. That said, it's been interesting for me to see how a particular criticism gets met there so even if I'm an idiot in way over my head, it's not been a bad experience either. I'm curious to see what lengthy reply Dr. Bruce Dale provides to my three critiques post, and if he will take up the follow up question where I tried to express the paper is really a nice model of how his own thinking goes when it comes to the questions it is supposedly answering. I do think it managed that spectacularly well and one now has a candid look into the mind of a Book of Mormon apologist through their exercise.

[_Physics Guy]

Here is my summary of the critique so far. I'm going to have to give up on this issue now and get some actual work done ...

1) The Dales' likelihood ratios aren't calculated at all, but just chosen, by subjective assessment, from a small set of allowed values. As Shumway pointed out, the Dales often seem to assign likelihood ratios as low as 0.1 when a value as high as 0.9 might be at least as plausible. If we raise 0.1 to the 132nd power, we get the insanely low odds of 1 in 10^132, but if instead we raise 0.9 to the 132nd power, we get odds of only just under 1 in a million. That's a factor of 10^126 which is subjective because of subjectivity raised to a high power. The subjectivity does not tend to average out over the long haul of 132 subjective assessments because the subjective assessments by the Dales are not random. Their subjective assessments are consistently biased in favor of the Book of Mormon being true.

2) The Dales' likelihood ratios are all assigned from among a small set of allowed values, and this set of ratios does not go higher than 50 or lower than 1/50. So the logical impact of strong evidence is arbitrarily capped by the Dales at a very modest level. This arbitrary capping artificially prevents discrepancies like horses or steel from having the decisive effects that they should have. Since the features that the Dales consider consist of many debatable "hits" plus a few glaring discrepancies, capping the impact of the discrepancies grossly inflates the final likelihood of the Book of Mormon being authentic. The artificial inflation factor in the final odds due to arbitrary capping might plausibly be as large as 10^60 or so.

3) The Dales multiply a large number of small likelihood ratios together, but this would only be valid if the likelihood ratios were independent and they obviously aren't. A fraudulent Book of Mormon would never be composed by making independent random guesses for every feature of the fictional Nephite world. Instead that fictional world would be invented by consistently copying from a model society (such as Biblical Israelite culture) and model geography (such as New England), with only a few independent choices to be made according to the author's fancy. So even if the Dales' subjectively assessed likelihood ratios for fraud were all fair for each individual feature, it would be extremely wrong to multiply them all together as if they were independent. Ignoring their correlation is such a severe error that it can easily inflate the odds against fraudulence by a factor of 10^100, perhaps even by much more.

Conclusion
The Dales' final estimate of the odds against the Book of Mormon being fraudulent is very high, but this is only because the severe flaws in their methodology have produced very large false inflation factors. The Dales' very high odds against the Book of Mormon being fraudulent are thus entirely an artifact of bad methodology. If their calculation is redone with the subjectively assessed likelihoods being chosen more skeptically, with adverse evidence allowed its full weight instead of being artificially capped, and with many highly correlated choices not being treated as independent guesses, then the Dales' conclusion can easily be completely reversed.

The Dales have not only produced an extremely optimistic Mormon odds estimate that is very different from the estimate that skeptics could make. They have also made major objective errors in capping likelihoods and ignoring many strong correlations. Beyond even that, they have confidently presented their analysis as rigorous; their disastrously invalid assumptions have been made without any admission that the assumptions are even debatable. Without that overconfidence this paper might have been a scientific work that was just badly mistaken, but the overconfidence marks it as a crackpot production of which Interpreter should be ashamed.

[_Res Ipsa]

Great summary!

[_DrW]

Great summary!

Agreed.

Well done, Physics Guy. "Crackpot production" is an entirely apt description.

Now, if only the authors or someone on the Interpreter editorial board would show a shred of integrity and take the paper down with an apology to the readers.