$30k challenge to Interpreter’s “Team Bayes”

[Gadianton]

In terms of number of pages, the Book of Mormon is more like the Bible than a typical 19th century novel.

Don't give credit where it isn't due. He didn't compare it to a typical 19th century novel.

He let the Bible represent the length of an ancient compilation of books. He found that the Book of Mormon was similar to the length of the Bible, with no controls. Even then, he cheated, because obviously the Bible is much longer, and so instead he compared lengths of individual books and found a 53% probability that such sub-book length given the book in ancient. P(L | A)

So in the null hypothesis mode, let's say P(A) is 50-50, then P (A | L) = .5 * .53 / P (L)

What is the probability of chapter length by itself? Probably 50-50 also and so he was almost right.

Anyway, we shall never know, instead he uses a alternative hypothesis model, which is the probability that Smith, as a first book, wrote a book so long (total length, not sub-book length)

Based on this fact in isolation, it is something on the order of 1,000 times more likely that the Book of Mormon really is an authentic ancient record and not a product of the 19th Century. Do I got that right?

in Alt hypothesis mode, he claims P (modern | length) = .00055. That's the chances of a first-time author would write a (novel?? that became famous? got published?) sprawling novel so long. So it's basically, 1,000 times more likely that the Book of Mormon is ancient because it's 1,000 times less likely that a modern first-time author would write a book in total length this long.

And so if you were off you're rocker back in the 19th century, drank up a storm every night and scribbled out 800 pages of nonsense, then provided it was broken into chapters of extremely vaguely similarity in length to the Bible, then it's most likely ancient.

[Gadianton]

He had a Mahalanobis Distance of 17.5, which, when plugged into a chi-square test with three degrees of freedom (for the number of variables in the analysis), reveals a probability of p = .00055. In other words, we would expect about 55 in 100,000 first-time authors of his age and education to publish a work with the length of the Book of Mormon.

[Philo Sofee]

Lem
Kyler Rasmussen has concluded that the rareness of publishing a book of this length supports his hypothesis that it is truly an ancient document, provided by an angel who brought gold plates. What do you even say to such nonsense? Did the mathematician pass on this? Did he really think that such assumptions have credibility?

And don't forget the stone which gave him actual glowing English words from an unknown foreign ancient language...

Perhaps Kyler is wishing to wait until he has built up some probability before dropping the bomb shell of angels, gold plates, unknown languages, and English translating rocks? We shall see. But yes he absolutely HAS to provide some probabilities to all this as well, WITH controls.
It will be fascinating to see how he discusses THIS background reality with probabilities.... It's going to take his original 1x10^-41 far below that threshold.... and I am not even sure he is aware of that yet. He will probably use the witnesses to overcome such drastically low priors...

We need to remember also it was Joseph Smith himself who said some revelations are from God, some from men, and some from the devil, establishing from the get go a mere 33% probability that a revelation is validly God's. This includes the entire Book of Mormon... If KR gets his own probability above this 33% threshold, he is going to have to actually demonstrate the evidence that whatever revelation Joseph presented was from God and not the other two...

[Physics Guy]

Hey, I think I have a statistical proof that I myself am an inspired Prophet, and I'm going to show you all that proof right now. I have job security, so you can donate all your tithes to the worthy causes of your choice, but you'd better pay up, because behold my proof:

XÖILHVLN-M,SRTÄIN-YYVMHISHFDN,Y.NFAF!

To your unenlightened eyes those 37 characters may look like something anyone could produce by random scrabbling on a German Mac keyboard with caps lock. But it is the firm doctrine of my new religious movement that this precise sequence of characters is the eternally preordained message from the powers beyond. So, whipping out Bayes:

The probability of that precise sequence, given the hypothesis that my doctrine is entirely correct (and so I'm a Prophet and you have to send checks), is a big fat One. That's my doctrine, that exactly that sequence and no other has been revealed through my unworthy (but very important) self. Was there any chance at all that any one of those 37 characters could have been something else? No, there was no chance at all. That's my hypothesis, which the evidence you've just seen supports overwhelmingly.

The probability that that precise sequence was generated randomly, on the other hand, is astronomically low. I mean, I've got 48 keys on here that generate characters, not counting possibilities like undoing the caps lock or holding CTL or OPT. So the odds of that precise sequence can't be higher than 1 in 48^37, which is more than 10^62. That's a fair bit more than you can shake a stick at.

So there you go. By Bayes's theorem the chance that the above Revelation was random is over 10^62 times smaller than the chance that I really am a true Prophet. That clearly makes me a Prophet. Pay up.

And please let me know how much money you send to those charities out of devotion to my Prophetic office. I'm going to see if it can offset my taxes.

[Dr Moore]

Take away the idea of miracles, and ask this instead about any random person.

What are the odds that person has 25 specific, statistically independent personality traits and has read or heard stories about 25 specific cultural traits of the Biblical Israelite peoples?

You get to define which traits to look for.

Hint: the odds quickly exceed the total number of humans ever born.

Related question, on the topic of "outliers" and what they mean: are the odds that Hitler realized the scope of his atrocities any better or worse than the odds of Joseph Smith realizing the scope of his life?

[Physics Guy]

I think this is the same issue as my prophetic claim: how does Bayesian inference deal with the Texas Sharpshooter?

The evidence is a barn full of holes. Hypothesis 1 is that the Texan is a great marksman who aimed at each of those spots. Hypothesis 2 is simply that 1 is not true: either he isn't a great marksman, or he wasn't aiming at those spots, or neither.

On Hypothesis 1 the probability of those particular spots being hit is close to 1. On Hypothesis 2 it has to be very small. The two hypotheses would seem to be mutually exclusive and collectively exhaustive. So it looks as though Bayesian inference from the data of barn holes has to support Hypothesis 1 very strongly. This is obviously nonsense but what exactly went wrong?

Part of the problem surely is that our seemingly exhaustive set of hypotheses fails to mention what actually happened, which is that the guy fired at random and then claimed to have aimed at those particular spots. And yet I can't seem to put my finger on the mistake, here.

If he fired at random then this particular part of holes was very unlikely. Nothing he claims afterwards make them any more likely. So I don't see what rule of Bayesian inference should make me raise this low P(holes|2).

If he aimed at these spots deliberately and is a great shot, on the other hand, then it wasn't unlikely at all that he hit these holes. So again I don't see what Bayesian principle could make me reduce this high P(holes|1).

Hang on, I'm getting an idea. But maybe I'll let other people think about this, too, for a while before posting it.

[Lem]

I think this is the same issue as my prophetic claim: how does Bayesian inference deal with the Texas Sharpshooter?

The evidence is a barn full of holes. Hypothesis 1 is that the Texan is a great marksman who aimed at each of those spots. Hypothesis 2 is simply that 1 is not true: either he isn't a great marksman, or he wasn't aiming at those spots, or neither.

On Hypothesis 1 the probability of those particular spots being hit is close to 1. On Hypothesis 2 it has to be very small. The two hypotheses would seem to be mutually exclusive and collectively exhaustive. So it looks as though Bayesian inference from the data of barn holes has to support Hypothesis 1 very strongly. This is obviously nonsense but what exactly went wrong?

Part of the problem surely is that our seemingly exhaustive set of hypotheses fails to mention what actually happened, which is that the guy fired at random and then claimed to have aimed at those particular spots. And yet I can't seem to put my finger on the mistake, here.

If he fired at random then this particular part of holes was very unlikely. Nothing he claims afterwards make them any more likely. So I don't see what rule of Bayesian inference should make me raise this low P(holes|2).

If he aimed at these spots deliberately and is a great shot, on the other hand, then it wasn't unlikely at all that he hit these holes. So again I don't see what Bayesian principle could make me reduce this high P(holes|1).

Hang on, I'm getting an idea. But maybe I'll let other people think about this, too, for a while before posting it.

I have a few ideas:

1. Are the Sharpshooter’s misses counted?

A way to do this would be to check:

2. Did anyone count the total shells, comparing the number of hits in the one building to total shots fired?

The above two measurements would be the opposite of how the Dales weigh hits versus misses. (The Dales are the type to brag they won $100 gambling, without telling you they maxed out their debit card and lost $10,000 to do it.)

And last, sort of like Smoot using as proof the fact that Smith wrote a book in which he, Joseph Smith, is prophesied to be a great leader,

3. Did anyone look closely enough at the holes to determine whether the shooter was standing inside the barn, with his rifle propped up against the wall each time he pulled the trigger?

[Gadianton]

I think the claim is a twist on the sharpshooter. Because he's using the hypothesis / alt-hypothesis version of Bayes, he's created a false dilemma where denying the target hits, or affirming the sharpshooting, denies the supposed sharpshooter, and affirms a contrived, irrelevant, exclusive alternative.

"The holes on that barn door were either made by Billy the Kid and his Momaday, or by Lil' Jim tinkering around in his backyard"

hypothesis: Billy the kid came back from the dead and shot up Lil' Jim's door.
alt-hypothesis: Lil' Jim, an amateur shooter and huge Billy fan, shot the holes for target practice.

evidence:
The holes on the barn door were measured by a CSI expert, and from the supposed distance, could have been produced by the caliber of Billy the Kid's Momaday gun. The odds are 50-50 that the Momaday could produce holes like that.

There were two holes per joined plank of the door, skipping a plank in between and the shots were from 50 feet away. There is no way that Lil' Jim could have intentionally produced that pattern. We measured 100 other barn doors in the neighborhood, and nobody had two holes per plank with skips in between.

conclusion:

since Lil' Jim couldn't have done it, then Billy the Kid is our guy.

You could object that we're drawing the target around the holes, but you could also point out that Lil' Jim's neighbor is Big Tex. Nobody dared trespass to measure the holes on his barn.

[Physics Guy]

Well, I still find this old chestnut example of a fallacy a bit confusing from a Bayesian viewpoint, but I think I have this resolution.

What I think is that the conditional probabilities of the holes, given the two hypotheses, are indeed respectively close to 1 and very small. The reason we're not forced to admit that the guy is a sharpshooter, I think, is that our prior probability for hypothesis 1 is not simply our prior probability for the guy being a sharpshooter.

This being Texas, the chance of him really being an amazing shooter might plausibly be as high as 1%. But our prior for Hypothesis 1 should be enormously lower than that, because Hypothesis 1 is not just that the guy is a sharpshooter. It's also that he was aiming at each one of those holes in his barn. There's nothing wrong with defining that as our hypothesis, but we have to consider the whole hypothesis when we assign it a prior probability.

So say the odds of this guy being a true sharpshooter are 1 in 100. What are the odds that an expert sharpshooter would choose to aim a series of shots at those particular points on the barn, instead of at any other set of points on the barn?

The sharpshooter himself can swear on a stack of ammunition boxes that he totally did that, and he can define that claim to be part of Hypothesis 1 if he likes, right along with his sharpshooting skill. Nothing in Bayesian inference says we have to give his claim any prior credit, however. We are totally free to choose any prior P(1) that we like. And just because P(sharpshooter) = 0.01 was reasonable doesn't mean that P(1) has to be anywhere near 1%.

Perhaps it could be, if for example this particular set of bullet holes spells out, "Jim loves Jane" or something like that. If it just looks like a pretty random set of holes, though, we would be reasonable to choose a P(1) equal to 0.01 times the random probability of those holes being there instead of anywhere else on the barn. Because why would a sharpshooter pick those spots to aim, in particular? Even if we grant that he could hit them if he wanted, the prior chance that he would want to hit those random points should reasonably be extremely low.

And consequently the extreme lowness of P(holes|2) is exactly mirrored in the extreme lowness of the prior P(1)—unless the pattern of holes really doesn't look random, in which case, goldurnit, the guy might really be a great marksman. The Bayesian conclusion is then that a barn with a random pattern of holes in it makes us no more inclined to believe this guy is really a sharpshooter than the 1% inclination to believe that that we had without considering any barn holes.

If I'm right about this being the resolution to the paradox, then the paradox is revealed as a stupidly blatant swindle: the sharpshooter tried to get us to commit to a Bayesian prior of 1% that did not apply to his full Hypothesis 1, but only to the sharpshooter part of it. Then he used the unconsidered second part of Hypothesis 1, about aiming at those precise spots, to raise P(1|holes) close to 1. He simply tucked a huge set of odds up his sleeve during the draw and pulled it out for the showdown.

The moral for the Book of Mormon Bayesian shenanigans seems clear: don't be satisfied to accept that low prior of one in 10^{40} just because it's so outrageously low. The apologist is going to smuggle in even larger factors, as additional assumed conditions in his hypothesis. These additional conditions in the Mormon hypothesis would make our prior much lower even than 10^{-40} if we took them into account.

[Dr Moore]

I'd just add that most of the "bullet holes" used by the Dales, Gee (and I will wager, to come by Kyler R) are NOT ACTUALLY BULLET HOLES. They might be age cracks in the wood, or old tree knots fallen out. But they're identified confidently as "bullet holes" by representatives of the shooter, certain he's a sharpshooter, determined to find bullet holes.

What these reps failed to do --
1. confirm that the sharpshooter claimed to make the shots
2. clearly state what a bullet hole looks like
3. survey the whole barn door to eliminate false positives (controls)

With these forays into the stats world, Mopologists have unwittingly forced themselves into an analytical framework that exposes their laziness and fallacious analyses.