The Interpreter; Bayes Theorem; Nephites and Mayans

[_Res Ipsa]

Lemmie, if we estimated the dependence of the Dales’ correspondences at 20%, would we have to multiply the denominator of their LR by .2^(number of evidences)?

Here's a helpful place to start, Res Ipsa: http://www.stat.yale.edu/Courses/1997-9 ... ndprob.htm

Thanks. It was helpful. And I wouldn’t want to calculate the probability of 131 dependent variables. At least I wouldn’t want to without a fifth of bourbon and a jumbo bottle of aspirin.

[_Physics Guy]

I think the Bayesian logic of multiplying independent LRs is really correct. It does sound funny that you can multiply a few LRs which could each individually be described reasonably as "weak", and overpower an LR which would reasonably be described as "strong". I think the problem here is not with Bayes, though, but with how humans tend to perceive probabilities subjectively.

Once a probability gets to around 80% or so we start thinking of it as a sure thing. And on the other hand we're pretty careless about overlooking chances up to 10% or so, as if they don't matter because they're too low to take seriously. Maybe we have vestigial instincts from primitive times in which you had to decide quickly where to run and there was no point in fussing over lesser chances because you were going to have a fair chance of dying no matter what you did. Anyway, I don't think our intuitions are very good at assessing probabilities that are either small or close to 1. We tend to count everything close to 1 as if it were 1, and we don't pay much attention to the difference between 2% and 0.0002%. If it's really 2% I'll act as though it will never happen—and so if you propose to me a small probability for an unlikely event, I'll agree to 2% without much thought, even if 0.0002% would be more accurate.

That kind of coarse-grained Bayesian reasoning is probably not even so bad as long as you're only looking at single issues. If you treat 2% or 0.1% as if they were 1%, you're still only going to be surprised around 1% of the time. Our clumsy intuitions about probability really fail us when we start combining multiple bits of evidence, however, because then you get exactly the kind of paradoxes you're raising. If the LRs are really correct, then the conclusion about how much weak evidence can overpower strong evidence will be valid, but our colloquial language for describing this kind of logic in words may be too crude to do it justice. The conclusion might really be bad, because the weak evidence was overweighted in a way that we could overlook in each individual issue but that blew up when we multiplied. Or the conclusion might be sound and just sound bad when you put it in words.

Bayesian inference still works—it has to work because it's nothing but logic—but the pressure to get the numbers really right rises exponentially with the number of LRs that you need to combine. A systematic 5% error in each of ten LRs makes for roughly a 60% error (crude first-order estimate) in the product. If your errors in your LR estimates are themselves independently random, then the combined error probably only scales with the square root of the number of LRs, but even when the LRs themselves are independent it is often the case that one's procedure for computing the LRs has a systematic bias of some kind. So in practice Bayes gets dramatically harder and harder to use when you apply it to larger bodies of evidence, unless the evidence is so clear that you hardly need Bayes anyway. The promise of distilling a clear and rigorous verdict out of many individually inconclusive bits of evidence is in principle real but very difficult to realize in practice.

You can get the numbers to give you clear answers but unless you're extremely careful the uncertainty is simply transferred into doubt about how good your numbers are. Bayesian inference is not a magic trick to spin straw into gold.

[_Gadianton]

Anyway, I don't think our intuitions are very good at assessing probabilities that are either small or

Great point. I didn't think of that, even though it's kind of been on my mind and I did read the explanation of the factors mimicking decibel levels. We're kind of stuck here.

it is often the case that one's procedure for computing the LRs has a systematic bias of some kind. So in practice Bayes gets dramatically harder and harder to use when you apply it to larger bodies of evidence

Another superb point I didn't think of.

those together seem like a theoretical limitation not just practical. well, maybe i should browse a little more before continuing this line.

[_aussieguy55]

Ben McGuire made this interesting comment on MAD

"The problem isn't one of whether or not history is interpreted accurately and the comparison between two historical models is valid and significant (although these are interesting questions). The two major issues for me are first, whether or not the way in which this is transferred is valid (whether or not the literary comparison between two modern texts is in fact a suitable surrogate for a comparison between two historical realities) and second, whether or not an appropriate comparison is being made between the two modern sources. For the first part, I believe that this is an inappropriate transfer. And for the second part, I believe that the comparison being made is more parallelomania than anything else. And based on these two conclusions, the statistics (and the math behind it) becomes irrelevant"

[_Arc]

For the first part, I believe that this is an inappropriate transfer. And for the second part, I believe that the comparison being made is more parallelomania than anything else. And based on these two conclusions, the statistics (and the math behind it) becomes irrelevant"

Ben McGuire is not the only one criticizing the Dales paper on the MAD board. Among the MAD Board critics, JarMan's arguments against the paper are especially effective. His call for the paper to be withdrawn is being allowed by the moderators. Worse still, the paper has no effective support among its advocates.

No. Neither option can be exhaustive on its own. If one option was exhaustive it would be true by default. The pair together has to be exhaustive, meaning that one or the other is certain to be true. "Divine or not" (as above) is exhaustive. One choice must be true. "Divine or Joseph guessed it" is not exhaustive. As I've mentioned, someone else could have made it up. The authors clearly didn't consider this in their analysis.

I would love the Book of Mormon to be an actual ancient record. I really would. As such, I thoroughly enjoy good, apologetic scholarship toward that end. But bad apologetics gives believers a false sense of security in their belief and casts a cloud over good scholarship. Apologists have done a lot of good work to help people believe when they have doubts. I'm happy for that. But I don't think it's ethical to use bad scholarship in apologetics. . . even if it helps people to believe. Ultimately that's why I believe this paper needs to be withdrawn. (Bolding mine.)

Severe criticisms of the paper, including several calls for it to be retracted, are now coming from all quarters; the Interpreter comments section, ldsphilospher.com, MDB, reddit and even the MAD board.

It has been nearly two months now since the paper was posted and the criticism just keeps coming. It appears that if Bruce Dale doesn't want to spend the rest of his time on earth defending the paper, he should just take it down.

[_Physics Guy]

JarMan was on the ball about the Dales' paper right from the start.

[_Arc]

JarMan was on the ball about the Dales' paper right from the start.

Yes he was. Better than that, he has chosen a fatal and easily understood criticism of the Dales paper to justify his call for its withdrawal.

Even those who have no clue about the mathematics needed to make reasonable Bayesian conditional probability estimates can understand the need for the hypotheses being tested to be exhaustive and mutually exclusive. Despite claims to the contrary, the hypotheses supposedly being tested in the Dales paper, no matter how may times they are modified, clarified or contorted, are never stated as exhaustive and mutually exclusive.

There is just no defense against that fatal flaw, and JarMan has played it to near perfection.

[Guest]

At one point, in responding to comments on his Interpreter paper, Bruce Dale states he is ready to go head to head with his critics using Bayesian statistics to test hypotheses that are exhaustive (e.g., either the Book of Mormon is a historical record or it is fiction). Bruce says that he has plenty of ammunition outside the Book of Mormon / Maya comparisons to defend his position.

This challenge sent me looking for the best smoking gun - the one that fires the silver bullets.

There is no shortage of falsifiable (and most would say falsified) claims in the Book of Mormon including the Jaredite transoceanic voyage in barges, and middle eastern Semite populations using steel weapons and horses in the pre-Columbian western hemisphere.

However, there are none so untenable as the building of a blue water ocean going vessel by a small band of refugees in an imaginary place called Bountiful along the southern coast of Oman.

As described several times on this board, there is unanimous agreement among mainstream scientists and engineers looking at the availability of timber needed to build a blue water ocean going vessel in all of Oman, or the UAE, or Yemen, that such materials do not now, and never have, existed in the region.

This well documented consensus, valid for any time in the natural history of the planet, is backed by data from archaeology, geology, botany, mining and petroleum engineering, and historical records. Scientists who have published peer reviewed papers to this effect include a respected LDS professor from BYU.

This undisputed data alone negates the tale of the Lehites - a central theme of the Book of Mormon narrative. Added to the utterly ridiculous story of the Jaredite crossing in barges, no amount of Bayesian probability manipulation could lead one conclude that the Book of Mormon is anything other than fiction. The two most obvious fictions in the Book of Mormon have to do with the claimed transoceanic voyages. If the old world bands of refugees described in the Book of Mormon could never have arrived in the new world, one need say nothing more.

These contrary data are absolute showstoppers - on the order of Jeffrey Holland's claim that, according to the Book of Ether, the Atlantic basin was formed within the last 10,000 years, separating the hemispheres and leaving the Americas devoid of human population until the first arrivals from the old world as described in the Book of Mormon.

[_Gadianton]

I've changed my mind. A couple of days ago I was convinced that it is very bad to multiply likelihood ratios together but boy, was I wrong. I now think not only is it okay to multiply them together, it's probably a really good idea to, and specifically for the reasons I had issues with before: in order for more generic pieces of evidence to overpower a "smoking gun". My impulse was that something has to be wrong with the assumptions that allows for victory by parallel-mania: here is a smoking gun in the case against the Book of Mormon, but then lo, here are six weak parallels that overcome it. several great points were made against that thinking and I got very close to conversion, but two additional points clicked into place when I went back to read about LRs yesterday.

Point 1: The wording of articles I'd read weeks ago using "sensitivity / specificity" language and then the Kass paper being abstract as it is, clouded the obvious that all these factors are, are literal "odds" calculations, so why wouldn't you multiply them? When I did a fresh Google yesterday, I got lucky: the article that came up was as simple as it could be put. A Bayes factor of 2.5 is a 2.5/1+2.5 = .71. So yeah, I feel kind of dumb about that but better to just come clean and start over. If I can do it, Dales, so can you.

Point 2: I targeted this one, given Physics Guy bringing up errors. Many of the introductory articles to medical test LRs either don't mention or gloss over statistical significance. So I hunted down articles with "likelihood ratio" and "confidence interval". Now it gets a little more interesting. Here's a good one:

https://pdfs.semanticscholar.org/8ae8/b ... b5203c.pdf

Words of caution about LRs:

Second, although test results at the extremes of the distribution provide the greatest diagnostic information, because they have the most extreme likelihood ratios, the estimates of likelihood ratios at these extreme val-ues are very imprecise (ie, they have wide confidence intervals) due to the sparse data at the extremes.12,13 Although the point estimate for the likelihood ratio for>40,000 WBCs is 25.0, the 95% confidence interval is 2.4 to 257.2 (Table 4).

A good calculator for those interested in LRs with sample sizes considered:

http://www.sample-size.net/sample-size- ... ood-ratio/

That was a paradigm shift to my thinking. Instead of thinking of strong evidence in terms of how much weak evidence would it take to beat the strong evidence, the real constraint is how deep can you go into the tail before your LR is statistically meaningless. Given sample sizes that I suppose are the norm in medical testing, it is what it is. Maybe sometimes you'll be lucky and have a 50 that's usable, but a couple of 10s with good confidence (if not a single 10) are generally going to be better.

So thinking about the Dale's own exemplar use case, medical testing, what does their project have in common? I'm trying to envision a medical example that works according to the Dale's numbers. In real medical testing, there seems to be an array of factors for a diagnosis, with precious few at the extremes. On the face of it, the Dale's have this weird case where they have several "smoking guns" on either side of the equation, blasting away at each other like in a Yosemite Sam cartoon and the guy who had one more bullet wins. Each piece of evidence tilts the balance to the 98th percentile, and two together brings it to three nines, and it's only getting started.

Well, first off, sure, nobody knows anything about the model and the Dales can say whatever they want. Perhaps it just so happens that this diagnosis does have several 50s. But then, given questionable confidence intervals, where does that leave their analysis? what happens if you multiply a mere 25 that could be anywhere between 2.5 and 250 (see above) by another 25 with the same problem? And then divide by the counter-evidence with a couple more ambiguous 25s?

A possibility is that the Book of Mormon is a "deep-tail" book, unlike cases in medical testing, due to an apologist's ability to imagine the model is whatever helps win an argument with a critic in that moment. Well, three nines in medical testing might be way out there, but in circuit board failures or something like that, perhaps you can go deep into a tail. But, supposing you can, expectations would be raised, and certainly, you'd raise the standard for strong evidence. If you can get a 150 with confidence then maybe that means a 10 is no longer very strong at all. And so best case scenario for the Dales, is that what they mean by "strong evidence" is really moderate evidence such that it still has a tight CI and we can feel good about multiplying together.

Obviously, that means there is stronger evidence artificially blocked off, and it does matter, as maybe there is a 74.8 out there with a pretty good CI. But I think there are a couple other points. First of all, Billy Shears helpfully did some LRs, here is just the denominator for city of Laman:

Let’s assume that this strong tendency is 10%. In other words, there is a 10% probability that the consonants of cities from Book of Mormon times would survive the way the city Laman did. If that is the case, what is the probability that only one Lehite city (Laman) exhibited this “strong tendency”? If there are 100 named Book of Mormon cities and the probability of a name sticking is 10%, then we would expect that 10 Mayan cities would have names that could be traced back to their true Book of Mormon historical roots. The probability that only 1 does is about 0.13% (this was calculated by approximating the binomial distribution with a normal distribution).

So, dividing the probability of the evidence assuming the hypothesis is true by the probability of the evidence assuming the hypothesis is false is the likelihood ratio, which for this point of evidence is .22/.0013 = 170

To get a 170, you got to go pretty deep into that tail. And sure enough, in this case, it doesn't look like the deep-tail hypothesis can be raised as we're talking about a single city with the D- result, which is going to make for a bad CI. If only the Nephites had 10,000 cities, then if 1% of those preserved ***, we could say that by falling short of 1000 -- 10% of them, that the resulting 170 in favor of a fictional Book of Mormon means something. Supposing this LR isn't a complete outlier, then I think we'll find that all the very strong evidence comes with a big question mark, and multiplying it out -- I can scarcely imagine that being valid, even if they are independent, but that's just a guess as unfortunately, the papers I've come across don't deal with weird hypothetical examples. But we do have stats people on board who might have an opinion...; )

[_Dr Exiled]

At one point in the Interpreter comments section, Bruce Dale states he is ready to go head to head with his critics using Bayesian statistics to test hypotheses that are exhaustive (e.g., either the Book of Mormon is a historical record or it is fiction). Bruce says that he has plenty of ammunition outside the Book of Mormon / Maya comparisons to defend his position.

This challenge sent me looking for the best smoking gun - the one that fires the silver bullets.

There is no shortage of falsifiable (and most would say falsified) claims in the Book of Mormon including the Jaredite transoceanic voyage in barges, and middle eastern Semite populations using steel weapons and horses in the pre-Columbian western hemisphere.

However, there are none so untenable as the building of a blue water ocean going vessel by a small band of refugees in an imaginary place called Bountiful along the southern coast of Oman.

As described several times on this board, there is unanimous agreement among mainstream scientists and engineers looking at the availability of timber needed to build a blue water ocean going vessel in all of Oman, or the UAE, or Yemen, that such materials do not now, and never have, existed in the region.

This well documented consensus, valid for any time in the natural history of the planet, is backed by data from archaeology, geology, botany, mining and petroleum engineering, and historical records. Scientists who have published peer reviewed papers to this effect include a respected LDS professor from BYU.

This undisputed data alone negates the tale of the Lehites - a central theme of the Book of Mormon narrative. Added to the utterly ridiculous story of the Jaredite crossing in barges, no amount of Bayesian probability manipulation could lead one conclude that the Book of Mormon is anything other than fiction. The two most obvious fictions in the Book of Mormon have to do with the claimed transoceanic voyages. If the old world bands of refugees described in the Book of Mormon could never have arrived in the new world, one need say nothing more.

These contrary data are absolute showstoppers - on the order of Jeffrey Holland's claim that, according to the Book of Ether, the Atlantic basin was formed within the last 10,000 years, separating the hemispheres and leaving the Americas devoid of human population until the first arrivals from the old world as described in the Book of Mormon.

This! Great points! Unfortunately, feelings Trump data and logic with some of my close relatives and friends. They have a feeling and no amount of logic will change their group determined conclusions.