Craig Criddle wrote:Over the past week, I’ve been tracking some of the on-line threads written on our publication. Some of the posts I’ve seen have included speculation about my colleagues, about me, and about our motives.
In this post, I would like to explain how we came to work together, who did what, and why.
I would also like to make a request of those who cite this paper in future posts.
In the Fall of 2005, I posted an essay “Sidney Rigdon: Creating the Book of Mormon” at two locations on the internet. In that essay, I explained my background and biases with respect to the Book of Mormon, and I explained how I came to the conclusion that Sidney Rigdon was its likely architect. I based my conclusion on the evidence summarized in that essay. Among other things I had noticed word usage patterns in the Book of Mormon that seemed to me to be consistent with Rigdon's style.
While I found some intriguing patterns, I did not have the expertise to carry out a more detailed text analysis. In particular, I was not knowledgeable in computerized text analysis. I was actually manually calculating word frequencies, using the word count feature of Microsoft Word, combined with Excel tables.
Frustrated by that process, I typed the key words “computer text analysis Stanford” into Google, to see whether a colleague at Stanford might have the necessary expertise. I received hits for Matthew Jockers in the English Department. So I emailed Matt. I told him that I had a hypothesis regarding the authorship of the Book of Mormon, and that I was looking for a collaborator with expertise in computer text analysis. He was interested. So we met, and I showed him what I had. Matt knew very little about Mormonism and nothing about the Mormon scriptures, but he was familiar with authorship attribution scholarship and intrigued by my hypothesis that one or more 19th century authors potentially authored the Book of Mormon. He was also very knowledgeable in text analysis and had the computer tools and know-how needed to understand the problem and extract the word frequency data from texts. After some discussion, he agreed to do the analysis for me; at that point we had not discussed co-authoring a paper.
With the help of friends, I was able to obtain most of the texts we needed for analysis, but I was not successful in obtaining reliable text for Joseph Smith (which we are still hoping to obtain). Matt took the texts I provided, segmented and encoded them into the xml that his tools require. He did the same with two control texts that he obtained.
Initially, Matt provided me some lists of frequently used words, bigrams, and phrases in the Book of Mormon and in the other texts. I ran some tests using my amateur methods, and Matt ran some tests of his own using his methods. We had some similar outcomes, with Rigdon appearing as a likely major contributor. This led Matt to believe that the theory was worth rigorous testing using more sophisticated methodologies. He decided to organize and lead a team effort with an eye toward publication of the research. We both understood the need for a bona fide statistician, and Matt recruited Daniela Witten, a doctoral student in Statistics with expertise in machine learning and classification.
Matt organized and directed regular meetings of the three of us in which we discussed how to proceed. Matt wanted to conduct tests using the Delta method, a method commonly used for authorship attribution. Daniela proposed additional testing using the method of Nearest Shrunken Centroids (NSC), a pattern classification technique developed at Stanford.
Matt and Daniela then did the analysis, applying both Delta and NSC. I was not involved in that part of the work. We all recognized that I had a bias issue, and we agreed that we would let the chips fall as they would: if the results came back negative for Spalding-Rigdon theory then that’s what we would report. But the results came back supportive of the Spalding-Rigdon Theory.
Matt then wrote the first draft of our manuscript and sent it to me and to Daniela for our additions. I added my expertise in Mormonism and the historical context. Daniela wrote the sections on statistics and NSC. The manuscript went through more than 20 revisions thereafter. We also solicited help from a small group of informal reviewers and incorporated their suggestions.
On April 5, 2008, Matt submitted the manuscript to the Journal of Literary and Linguistic Computing. The anonymous peer review process took six months. On October 7, 2008, we finally received notification that the paper was accepted pending an adequate response to the reviewer comments. We completed our response to the reviewer comments and submitted the corrected manuscript on November 6. On November 24, we received word that the manuscript was accepted. At the same time we received page proofs. We corrected them and returned the article on November 27. It was published electronically on December 6.
I would like to make a couple of points.
First, this manuscript should be referred to as the "Jockers et al. (2008) study" or as the "Stanford authorship study," not as the “Criddle wordprint study”. Yes, I did contribute significantly, but Matt led and coordinated the team, and he is the corresponding author. That is as it should be. Without him, nothing would have resulted. Daniela’s expertise as a statistician and skills as a writer were critical to the paper, so her contribution should not be discounted either. This is important for all to understand.
One reason I am making these points is because, as many apologists have already (correctly) pointed out, I am the team member with bias. Matt and Daniela were unbiased, and had very little knowledge of Mormonism before they became involved with this project.
While I contributed expertise as a former Mormon, Matt and Daniela carried out the data analysis and the results are what they are, independent of my participation in the research.
I am hoping that by providing this background, it will become clear to all that this work should not be referred to as "the Criddle word print study”: it was a team effort led by Matthew Jockers. I was a member of that team, and, while my expertise was important, I did not carry out the analysis itself.
Craig Criddle
_