Possible Modern Source for the Book of Mormon

[_Always Changing]

The question is, how did they come up with the words? Imagine a hypothetically neutral baseline, if they ran that and discovered Late War in the process it would be awesome. However, If they were already aware of Late War already, and if they picked phrases already knowing the contents of the late war, it will be interesting to learn what kind of controls were used to avoid cherry picking.

I'm guessing on a "brute force" search, which took the first four words in the Book of Mormon, compare against the book base total content, sequence up one word in the Book of Mormon and loop. Likely took many hours to get results.

That is what I am thinking. Very legitimate, and what I thought I would do if I could write code. Simply replicates what UD did way back when without the tedious labor. I am only to page 19--- sigh.

[_Nevo]

I wonder though, whether Hunt had read the Snowden book. He got the idea from somewhere?

Hunt may have read Snowden. Rick Grunder thinks he did. But the fact is he could have got the idea to write in "the ancient style" from any number of sources. The genre was well established when Hunt began writing.

[_canadaduane]

I'm guessing on a "brute force" search, which took the first four words in the Book of Mormon, compare against the book base total content, sequence up one word in the Book of Mormon and loop. Likely took many hours to get results.

That is what I am thinking. Very legitimate, and what I thought I would do if I could write code. Simply replicates what UD did way back when without the tedious labor. I am only to page 19--- sigh.

Is RockSlider calling us brutes?

Yes, that's exactly what was done. Downloading the texts from archive.org took about 4 days. Compiling the baseline took an additional 4 days of computation and disk I/O--getting frequency counts for all 1 through 4-grams in all 100,000+ books, then selecting 5,000 out the set for a baseline and adding those frequencies to create a "master list" or baseline of frequency counts. Once all of that was prepared, ranking the books against the Book of Mormon and using the baseline took about 2 days.

Update: We go in to a lot of detail on the algorithm at http://askreality.com/hidden-in-plain-sight/

[_Gadianton]

Hi Tim,

I did a little snooping around the Good Book looking for the words "trench", "ditch", "parapet", and "picket" and I didn't find much of interest. Again, we have some real scripture scholars around here who could easily point out deficiencies in my research, but I'm optimistic for now. I believe much of the war described in earlier times is of the Israelites attacking the cities of other nations rather than bunkering up and protecting themselves. I looked for descriptions of Jerusalem being attacked with similar results and came up empty. So the attack on Fort Stephenson is very similar to Alma 49 and 50, and tentatively, I'm going to say the similarity isn't shared with the Bible. Hunt alludes to David and Goliath, and I'm thinking that if there had been a more direct connection to a similar Bible war, Hunt would have surely used it.

I'm Wondering if the "102 score" (exaggerated) lost by the King of England and the 7 wounded Americans kept the numbers reported of 1000 lost by the Lamanite King and 50 wounded of the Nephites more honest than the epic battles of the Jaradites. ;)

[_Fence Sitter]

I thought this exchange over at MAD was worth re-posting here. I hope that is okay with both Ben M. and Canadaduane.

There are several major flaws to the statistical model methods I have read about here and in the linked sites...

I expect I will have a blog post about it ready by the end of the week that has a more complete discussion on my problems with the statistical part of this issue. If Duane can find a way to give me access to his baseline data collection, I would be happy to offer up some suggestions and look for ways to make it more effective (but I am not sure he will be terribly interested in my perspective).

Ben M.

This is a fantastic critique, Ben! I appreciate the thoroughness of your comments. I have some counterpoints, but there are certainly some great issues raised that will help further the work to make this and other comparisons more accurate in the future. I hope it's clear from what we do and how we're incorporating new information that we are primarily interested in truth and understanding. This is the first time my brother and I have been involved in a study like this, so we are naturally learning as we go. Nevertheless, to my knowledge, it's the first attempt at a large-baseline-enhanced algorithmic search for books that could plausibly have been in the mind of Joseph Smith while he was authoring/translating/"translating by the power of God" the Book of Mormon.

Apart from the fact that you have already eliminated the best evidence for comparison (in getting rid of all the rare items occurring in fewer than a frequency of four), your rarest phrases (given the most weight - one of them would equal the equivalent of 600 phrases at the frequency of the copyright statements) all occur in only four instances. So we can do a search. Like, for example, "with-his-army-against". So we dig up an archive (like Google books), run a search - follow your dating criteria (from 1500 to 1830 - easy enough to program in), do a search on the exact quote, and voila

29 texts. This phrase apparently isn't so rare.

This is a good point--the limitation of our 5,000 book sample and our tossing the rarest of 4-grams is not as strong as if we include the entire corpus of texts and do not toss rare 4-grams. Part of the problem is that we have noisy data--OCR is still, at this time in history, a somewhat inaccurate business, which given our current computer hardware capabilities is why we had to toss 4-grams with frequency less than 5. I expect in 1 to 5 years our historical texts will have been scanned (or re-scanned) with a much lower error rate. That should help us in our quest for more knowledge, because we will not have to work as hard (and lose some signal) to get around the noise.

That said, I do think there is still a weightiness--a red flag, if you will--to finding a collection of relatively rare 4-grams between two texts. I won't say the case is closed that this shows a primary source, but in a quest to understand how things came to be (and I find shared common ground with you in seeking to get to the bottom of things), consider this hypothetical situation: let's say that Ellen White (Prophet of the Seventh Day Adventists) read Pride and Prejudice in her teen years and then went on to write Conflict of the Ages. What would the evidence look like if that were the case? Would we expect to see direct quotations? Possibly, but she wouldn't want to make it a dead give away that she'd copied something else. Would we expect to see influence? Yes, because when the creativity of the human mind is employed in generating a new work, it cannot help but remix things that it has encountered before (every English word we speak is a remix of someone else's using it--perhaps your parents, siblings, friends, books you've read, etc. have caused certain word sequence probabilities in your mind to outweigh other word sequence probabilities when expressing an idea).

The moment we find a parallel in only three of four sources - that is the moment that phrase no longer has any value at all for authorship attribution. Significant rarity isn't (as this statistical model suggest) found in relative rarity, but rather in absolute rarity.

If I understand your argument about authorship correctly, you're saying that finding two texts with a specific phrase that is shared by no other texts is a way we can be certain that a book has the same author. Are you also applying this authorship attribution test to this study, i.e. that we can only identify the texts that influence an author if we find matches that are in the two texts and no other?

I think the strength of the "signal" is fairly strong in the case of The Late War--a high frequency of relatively unusual terms indicates a close relationship between books.

I wonder how many of the almost 200 phrases you provide really aren't that rare.

I would expect that all of the 159 phrases with significance score 0.25 are equally rare (within certain error bounds). I mean, that's how we found them after all--in a simple random sample of 5,000 books (from a population of ~130,000 books), these phrases are "very rare" relative to other 4-grams in the English language at that time. As mentioned before, one of the things we can do to improve our results is to expand the size of our baseline.

and yes, I looked at a number of other phrases and came up with a relatively high frequency of phrases that were simply not as rare as your data collection suggested

I think this is a really neat thing--you were able to use Google to search to see a small window into even more data than we have access to on Archive.org. As information becomes more and more freely available, everyone will be able to do better and better searches and analyses. Exciting times!

If we want to look at a snapshot of the language in 1830, we don't start with international texts in 1500. We look at maybe +/- 50 years (and both directions is important - the language doesn't simply stop with the publication of the Book of Mormon) and focus where possible on American texts preferably in New York.

Thank-you, this is invaluable. I will see what we can do to get some results on this. Part of the problem in adding books after 1830 is that we start to get results that are Book of Mormon derivatives. For example, Orson Pratt's tract showed up (in 1839 I believe?) and became the top result when we expanded forward in time. A second issue is that there are far far more books published *after* 1830 than before (the number of books published each year seems to be on a power curve of some kind--it's amazing how much more data the human family is creating year in and year out).

I imagine that the significance rating of a book published in 1819 in New York is going to plummet like a stone in water.

Remember, our algorithm at this time is a search algorithm, not a proof of significance (although I think we have some more interesting work coming on that front). So by narrowing the search to New York, the "rating" is probably still going to point to the same book since the expanded search (a search of 100,000+ books from all areas of the world) already surfaced this book published in New York as the top scoring result.

Just as much to the point, that copyright statement is itself a pastiche of quotes. And with the quote marks (and other punctuation) removed, we have no idea looking at the various fragments where a text by one author ends and another begins.

Good point. I had hoped that we would be able to just leave each book, untouched, with its source text intact. But you raise an interesting case where a handful of n-grams will appear simply by combining the end of the copyright with the beginning of the Book of Mormon.

Of the list of parallels presented on the blog, 75 of them are taken from that copyright statement (that's a little more than 13%). Yes, it has little impact on the final scoring. But then, it has a huge impact on the overall number of parallels found in that list.

Looking back, I see how removing these from our published data could have been helpful for some people coming in for the first time to see our results. We had decided to publish the entire list as a matter of transparency, however, and I still feel that is the best way to go. We aren't trying to inflate things by making the list look longer--as you've correctly identified, the size of the list "has little impact on the final scoring".

And then I have issues with the removal of the King James Language...

One of the interesting features is that in trying to copy the language of the KJV, Hunt manages to get in 2,341 shared four word phrases with the KJV. This amounts to roughly 4.6% of all of the four word phrases in Hunt's book (there is a total of 51,221 unique four word locutions in the book - not a lot of repetition actually). That's quite a few. The Book of Mormon on the other hand shares 25,020 four word phrases with the KJV. that's 12.3%. The Book of Mormon doesn't so much imitate the KJV as it uses it (which we already knew). What makes this interesting is that if we don't exclude the biblical phrasing (that is, we include all of it) then the comparison between the Book of Mormon and Hunt's work yields 1,478 shared four word phrases. That amounts to 0.73% of the Book of Mormon. Of that 1,478 phrases, 847 of them are shared but come from the KJV (that's just about 60%). Leaving me with my estimate of 631 shared phrases between Hunt and the Book of Mormon that are not found in the KJV. That comes to 0.31% of the total unique four word phrases in the Book of Mormon. This is not a large footprint by any stretch of the imagination. By comparison, in the link above, we can see that Jules Verne's Around the World in 80 Days translated into English in 1873 has a larger footprint, since 0.8% of the phrases found in that book also can be found in the Book of Mormon.

In our work, what we've found is that the total number of matches without a baseline divisor is meaningless. For example, when we find matches between the Book of Mormon and Pride and Prejudice, it looks at first blush like "hey! there are hundreds of matches!" but when you then divide by a baseline frequency (as we did in our study) it quickly becomes apparent that all of those matches are noise. In fact, only two or three 4-grams shared between the Book of Mormon and Pride and Prejudice can be considered "rare" by our "baseline-weighted" measure. In other words, there is (almost) no overlap, and certainly not a close relatedness between these two books, even though it looks like it when you count up matched 4-grams.

The Late War and The Book of Mormon, however, is a significant find because many of these matches *are* rare. I think that is the major contribution of our work to date. We've identified a (perhaps novel?) way of being more precise about which overlapping 4-grams have meaning and which are simply background noise.

So in my estimation, while the creation of the baseline data is an interesting idea, it fails in the fact that apparently 5,000 may not be a large enough random sample (in that it apparently doesn't extrapolate well - and yes, I looked at a number of other phrases and came up with a relatively high frequency of phrases that were simply not as rare as your data collection suggested).

Criticism accepted. We'll improve our results at some point in the future, but we will have to set up the hardware infrastructure to support it.

And without the weighting that you get, Hunt's book isn't actually very statistically close to the Book of Mormon.

I think this is perhaps the primary flaw in the argument presented. All of our relatedness studies show that it's only the baseline-weighted matches that matter.

Thank you again for your consciencious and instructive post. I feel the gears of progress turning in my head..

[_Gadianton]

In my considered opinion, Canadaduane is being very generous with Ben in this exchange.

[_Always Changing]

In my considered opinion, Canadaduane is being very generous with Ben in this exchange.

He is just gleaning ideas from Ben on what loose ends need to be tied down. Isn't that what the last sentence means?

After all, Ben is just about the brightest bulb in the chandelier.

[_Fifth Columnist]

Apologists have a big problem and it is this: unlike almost all other word print studies, the similarity between the late war and the Book of Mormon is readily apparent to anyone who reads it. Mopologists can complain all they want about how they are different, but anyone who actually reads the late war will be struck at how much it sounds like the Book of Mormon.

[Guest]

Fascinating. So instead of justifying this clear case of selection bias here and unprofessional behavior, you just say you'll ignore the apparent issues. I seriously doubt you even know what intellectual honesty means.

It pains me to respond to this since there is obviously nothing substantive in this post either. I guess I should have completely dispelled all of your assertions once again in my last post.

If you don't understand what I'm talking about, you are welcome to ask some questions. What I've stated is rather clear, and for you to fail repeatedly to address it is remarkable.

First of all, a collection of random subsets based on one common correlation such as the "day of the month" would be expected to be random as a whole in the EXACT same manner that each individual subset is random. This expectation of randomness would neither INCREASE nor DECREASE when these subsets are combined. Are you following me?

No, that is completely untrue. There are many events that are NOT random. For example, people often eat at regular intervals or they may attend religious services every week.

This is a very simple premise and if you can't understand this then we're at an impasse.

Well, I'll explain it to you. You've made no attempts to understand my position, have been dismissive toward each of the points I bring up, and as a result have made no attempt to address what I've said.

The ONLY argument that you have in this case is to prove that the event in any of these subsets would NOT be expected to be random in a historical context. You can't. We can take each specific event type and look at that type in a historical context and immediately see a random pattern on how they fall in the month such as an assassination of a leader :

Lincoln Apr. 15th
Kennedy Nov. 22nd
King jr Apr. 4th
Malcom X Feb. 21st
John Lennon Dec. 8th (not exactly a political leader but I couldn't think of anyone else when compiling this list)

That is not my argument and I don't disagree with using this a dataset.

Of course you could argue selection bias

I'll stop you right there. That is not what selection bias means. Selection bias means that you are SELECTING YOUR SAMPLES for no other purpose than to FORCE YOUR CONCLUSION. Now carry on.

for this list because it is a subset of a larger list and I admittedly just looked up 5 of the most well known assasinations and I couldn't prove that it wasn't. But it can only be argued as selection bias because I've left out KNOWN data. Once all KNOWN data is included, the possibility of selection bias is ruled out.

Nope, that isn't it at all as I said.

Second of all, you have completely either misunderstood the analysis, or you are purposely misrepresenting it to make your case for selective bias. They did NOT take "many events" from "only a portion of the Book of Mormon history", they did in FACT take ALL of the events that actually had a day of month specified in the text (which is essential for this analysis) from the ENTIRE Book of Mormon history. In addition to this, every date except for 2 actually had more than 1 year separating them which increases the expectation of randomness. Do you even know what selection bias is? I would advise you to review this for your own benefit.

Actually, I don't think you know what selection bias is. See my reponse just above. And I don't misunderstand the situation at all. I'm well aware that these are likely ALL the events with dates available. I was just waiting for you to admit that in the hopes you would recognize the problems. Clearly you haven't, so I'll point it out to you again.

The intellectually honest thing to do is to simply admit there is not enough of the same kind of events (for example, assassinations) coupled with dates in the Book of Mormon to perform this type of analysis.

What you are championing and what the authors do instead is not intellectually honest. You repudiate your examples and pick a number of events simply because you don't have enough data about one particular type of event. This is what is wrong with this analysis and why I've said there is clear selection bias here.

Third of all, your're suggestion that they should have used something such as birth dates for their analysis is asinine considering that there were no birth dates described in the Book of Mormon text specifying a day of the month. I was seriously laughing when I read that because I was contemplating if you might actually be joking. Obviously not.

No, what I was stating is they should use the same criteria and methods as they use in their examples when analyzing the Book of Mormon. And the only reason you are laughing is because you failed again to understand what I was saying.

Clear case of selection bias? Ridiculous. You can argue that the Book of Mormon is in fact the 1 historical book in 2000 but you can't argue the expectation of randomness.

I think I've dealt with your response quite adequately. All that is apparent to me is you have repeatedly missed my point, been dismissive to my objections, and failed utterly to consider and respond in any meaningful way to what I've said.

[Guest]

Apologists have a big problem and it is this: unlike almost all other word print studies, the similarity between the late war and the Book of Mormon is readily apparent to anyone who reads it. Mopologists can complain all they want about how they are different, but anyone who actually reads the late war will be struck at how much it sounds like the Book of Mormon.

I read a few short excerpts from The Late War last night to my wife during commercial breaks as we watched TV and asked her to identify which part of the Book of Mormon each quote was from. She didn't get one right. When my HT or the elders comes by I am going to run a few by them to see haw they fare. I may even start posting a few on the fridge in quotations.