The Late War/Bayesian probability Article at Interpreter

[Guest]

The apologists move into full damage control mode by (a) contracting an outside specialist to weigh in on the Late War (b) vying for the appearance of fair-mindedness by not dismissing the Johnson's findings outright.

Let us not forget the order of events:

(i) The apologists poison the well: Multiple apologist blogs feature a Jeff Lindsay article satirizing attempts to attribute Book of Mormon authorship after they learn of the Johnson study but before releasing any information about their awareness of it.
(ii) Apologists all over the web outright deny any credibility to the Jonshon study or opinions of anyone entertaining the possibility of a Late War connection.
(iii) Publish the McGuire piece dismissing the viability of algorithmic plagiarism detection based on personal "ad hoc research" and an 11-year-old book.

The cataclysmic hit to apologist credibility by reacting to the new study as if they were "engaged in a war" rather than a scholalry dialogue needed tempering. The question is whether the scramble to appear collected is too late. It's very possible that cooler heads within Mormon Studies have looked at the eruption as confirming their reasons for shying away from apologetics.

All that being said, the Schaalje piece should be taken seriously. Here are my observations.

It would have been very difficult for Joseph Smith, even as translator, to express concepts without reference to his culture

But it would not have been difficult to avoid King James "Bible talk" as both LW and BoN make it clear that such speaking was not common and likely to be ridiculed even in Joseph Smith's day. Such manner of expression would have been a conscious effort by the Book of Mormon author, as it was for the authors of LW and BoN.

If some of Joseph Smith’s cultural milieu made its way into the translation of the Nephite record as Joseph recognized obvious lessons from antiquity for himself and his times, I see nothing disconcerting.

Except that the manner of degree could fully rule out all "tight translation" theories and decades of Book of Mormon apologetics. Ultimately, the historicity of the Book of Mormon is unfalsifiable. For many, mere unfalsifiability and a highly subjective model of "translation" that potentially validates any neural activity by Book of Mormon composer(s) will not be good enough.

describing the claim as the ‘Smoking Gun’ or the ‘Silver Bullet’ that would bring down Mormonism

Since this piece is targeted at our board and most specifically one distinguished man of the cloth who posts here -- who should feel flattered by these attempts to convert him -- I point out that this statement is false. Everyone I recall posting on this thread has insisted that the LW discovery will have little to no effect on Mormonism itself. We weren't born yesterday.

Some critics of the church acknowledged Ben’s thoughtful analysis, but dismissed it, saying that statistical results can always be debated and nothing ever proved via statistics

This is not quite true. There have been numerous reasons cited for criticizing Ben's analysis, least of which have been statistical viability. Many on this board hold advanced degrees and aren't about to comment wildly outside their expertise on what can or can't be demonstrated with statistics. The statistical part of this whole conversation has near universally been treated from cold to warm by our board, whether it's Ben's analysis, Chris and Duane's, or even my own fiddling with excel. Almost everyone on this thread is saying, "look, this black box we don't quite understand has produced this amazing book that is fascinating in thematic similarity to Book of Mormon."

Results using the Bayes rule can be counterintuitive. Here is a classic example.

As an aside, I wonder if he's ever applied Bayes to predict the probability the Book of Mormon is true based on praying and coming up with a positive answer. His discussion of the formula here is interesting, but given he's a professional and the formula is rather simple, there is no question he can see in advance what the parameters should be to generate this or that outcome. This makes it a little bit not true that he "let's the math do the work". I think his supporting points are more interesting than the calculation.

Concerning the supporting points:

a book other than LW may well be identified as more similar to the Book of Mormon for several reasons..

I'm not disputing the 5 justifications here, but I don't think it really matters for the majority of participants on this board who are more interested in the thematic similarities in conjunction with the KJV language. Consider two possibilities. (i) A new book is discovered as a better fit based on better modeling and data that lacks thematic similarity. So what? For those interested in the models and possibilities of such models revealing the plagiarism of ideas, something very hard to do if even possible, this would be important. But most on this board shy away from the technical part of the conversation. Granted all this would be of much importance to the Johnsons. (ii) another book surfaces of closer thematic fit. This would only vindicate critics, except in the very narrow cases where specific claims about Joseph Smith's borrowing are made.

More interesting to me:

Some reasons why LW might have a high similarity measure even though it had no influence on the Book of Mormon are...2. unrelated books that use deliberate scriptural language will share many phrases with the Book of Mormon,

This is a point of growing importance in my opinion. Amidst Nevo's desperate apologetics, he's offered a key point. He cited an author that has created a list of books of similar style to the Book of Mormon prior to the Johnson study, none of which are available from archive.org. It's possible that books with "Bible writing" language were not fairly represented in the sample. We really don't know how easy or difficult it is to come up with "Bible writing" in conjunction with a basic familiarity with the Bible, and produce thematic content that seems unique in its connection to the Book of Mormon. The list Nevo gives deserves serious consideration by both those interested in the statistics and also those interested in the thematic similarities; the results could seriously shuffle our perspectives.

Points 1, 3, 4, and 5 are all "Maybe-maybe not" points in my opinion for reasons I don't want to go into right now, but if anyone reading this specifically challenges me on any of those I don't mind elaborating. not trying to blow off, just condense.

6 in any corpus there will always be a ‘most similar’ book to the Book of Mormon, even if none of the books in the corpus had anything to do with the Book of Mormon.

This last one I believe can be disregarded. Unless I misunderstood the conference presentation, the LW got into territory close to Joseph Smith's own writings in similarity to Book of Mormon. My understanding is that absent LW and BoN, the next best thing would not have popped up with a higher significance to fill the spots. Would those down on the list be interesting enough for the Johnson's to create an ex-mo presentation out of? And if so, would critics on this site find enough thematic interest to take them seriously? Most critics on this site seem uninterested in the Spaulding theory. Whether this lack of interest is justified or not, don't let it be said that critics would have blindly taken whatever book happened to come in first place and run full throttle with it.

In summary, I think this was a helpful article. I hope the author is familiar with the practices of Mopologetics and didn't get suckered into this, I'd feel bad about that.

[_Spanner]

Nicely put, Gad.

As you have identified, this statement would best describe reasons for obtaining a false positive result using the Johnson's methodology:

Some reasons why LW might have a high similarity measure even though it had no influence on the Book of Mormon are...2. unrelated books that use deliberate scriptural language will share many phrases with the Book of Mormon,

The Johnsons can quantify that risk though (and the other issues identified), and if it is anywhere near 50% (as used in the analysis) then they might as well pack up and go home. The point of this discussion was to drive up the false positive rate to a point that Bayesian analysis will always show that the research has a very low impact on the probability of Late War influencing the Book of Mormon. Throwing a number at it was very premature, since the Johnsons are working on establishing accurate true/false positive rates for their method.

Likewise the statements about the true positive rate - the Johnsons have shown the method accurately ascribes the D&C and the other Mormon scriptures to the same author as the Book of Mormon. And it accurately links other books by the same author, as shown with their Jane Austin and other examples. On what grounds is that 80%? Again, it is possible to assign an actual figure but something just got plucked.

I suspect that an anchoring ploy is being used. That is to dash in first, before accuracy rates have been published, and use assumptions to put the first number out there. The first number is always the one to stick. The intention was to show that apologists, being "fair" to critics, have shown that the Big Data method only nudges the probability of Late War influence a little bit -from 50% to 60%. So with a faithful starting point of .01% it won't move bugger all.

But if they don't watch out it could backfire. As I noted on the other thread, the number that may well end up anchored in people's minds is the 60% one.

[_Everybody Wang Chung]

Some interesting comments from the MI article:

I have no problems with your math, but your assumption that P(E|H*) is at least 0.5 is speculative. In your breast cancer example, P(E|H*), that is, specificity, is calculated from data from large population studies. Your Book of Mormon calculations fail because the P(E|H*) is assigned arbitrarily through rhetorical argument. Even if your arguments are sound, this is a nonmathematical method.

I like your approach Bruce. Bayesian analysis is really a really great tool. I have a couple of questions:

1. If P(E|H*) = .5, doesn’t that mean that we should see nearly 50% of the selected sample of books to be classified as matches to the Book of Mormon on a similar level as The Late War?

2. Have you examined the probability that the Book of Mormon is a product of 19th century New England using a Bayesian framework? The probability you estimate of 0.6 for the Book of Mormon being influenced by The Late War given a positive match in the Johnsons’ analysis could then be used with the probabilities of other evidences to update the Bayesian probability.

Gary,

You are correct. I admit that my value for P(E|H*) was based on a rhetorical argument rather than data, and I was sure (~0.95) that someone would bring this up. You win the prize.

I listed reasons why I think the probability of a false positive is so high (~0.5). It would be better to have data on this. If we could identify books that for sure were not influenced by any other books in a corpus, we could empirically estimate P(E|H*). But for the time being, the Bayesian approach at least allows us to intelligently discuss our various reasons for choosing a value for P(E|H*), and to sensibly combine the guesses we make for the various quantities. Rather than a failure, I think this approach has great value in thinking clearly about the problem. It’s not the final word.

What value for P(E|H*) would you choose? Do you think my reasons for setting P(E|H*) at 0.5 are mistaken?

Isn't this a HUGE problem for the study? Why would the author assume a .5%?

[_Gadianton]

Everybody Wang Chung,

I think he's saying "for arguments sake" let's just say it's .5; can go either way, and also then a simplified version of the equation can be used. Then you see how much the other factors sway it. And his point is that even though some of the results might seem impressive, the contribution isn't that much, it gets up to .63.

I think with a Phd in stats and professor in that field and without (to my knowledge) an extended history working with the apologists, he'd not risk his reputation by going in real life with a bunch of outright bogus math.

For some the apologists, his post may be no more relevant in the details than this: "A professor of statistics has reviewed the argument and still believes the Book of Mormon is true!"

I think the article overall is a useful contribution to the discussion.

[_Gadianton]

The intention was to show that apologists, being "fair" to critics, have shown that the Big Data method only nudges the probability of Late War influence a little bit -from 50% to 60%

I don't disagree. I mean, maybe you and Everybody Wang Chung are right about the .5 starting point, it's just to me the calculation isn't as interesting as the lists of swaying factors. Duane seemed to be interested in Nevo's list. Unless the archive pool had other books that could be said to be representative of that list, I think this needs to be looked at.

[_Spanner]

The P(E|H*) is the probability of a false-positive (Late War is identified as an influence when it is not) that I mentioned above. As the commenters note, the reasons given for this being so high (.5 or 50%) are quite dodgy and like the low true-positive rate (.8 or 80%) look to be deliberately lowering the impact of the study. With these rates, they can afford to make the starting probability 50% (.5) and only have it nudged a wee bit higher by the Big Data results.

I think they have out-clevered themselves. Non stats users will just see the 60% probability figure and not realize that it is relative to the prior probability.

[_Spanner]

For some the apologists, his post may be no more relevant in the details than this: "A professor of statistics has reviewed the argument and still believes the Book of Mormon is true!".

Bingo

[_Kishkumen]

What about a Bayesian analysis of the probability that the Book of Mormon is an ancient text?

Why is it that I am not at all impressed with the notion of a Bayesian analysis?

Is it that a number of people have used it to support propositions that ultimately contradict each other in the larger picture?

I don't know, but I continue to find myself in shrug mode.

[_Always Changing]

Yes. They clearly need to add as many books written in the scriptural style as possible, since this is what the methodology is selecting. And they are working on it. This is probably the biggest problem they face. And possibly not one that they had thought of beforehand. I am very interested in what they come up with, when they plug in the proofread Snowden's American Revolution. THAT was a lot of work.



OK, Gad, I will be nicer to Dr. Schaalje in the future.

[_Doctor Scratch]

I hope the author is familiar with the practices of Mopologetics and didn't get suckered into this, I'd feel bad about that.

Oh, I wouldn't worry about that, Dr. Robbers. See here, for instance. Schaalje was also apparently recruited by the Mopologists--perhaps even by Matt Roper himself--to help counter the Jockers study.