Response to Jockers, Criddle, et al., Now Available

The upper-crust forum for scholarly, polite, and respectful discussions only. Heavily moderated. Rated G.
_harmony
_Emeritus
Posts: 18195
Joined: Fri Oct 27, 2006 1:35 am

Re: Response to Jockers, Criddle, et al., Now Available

Post by _harmony »

bschaalje wrote:
Effects of KJE make the situation more difficult to pin down. We will see.

Closed set attribution will not be of any help in dealing with archaic language.


It helps if the language actually existed....
(Nevo, Jan 23) And the Melchizedek Priesthood may not have been restored until the summer of 1830, several months after the organization of the Church.
_Daniel Peterson
_Emeritus
Posts: 7173
Joined: Thu Jul 05, 2007 6:56 pm

Re: Response to Jockers, Criddle, et al., Now Available

Post by _Daniel Peterson »

harmony wrote:
bschaalje wrote:Closed set attribution will not be of any help in dealing with archaic language.

It helps if the language actually existed....

It does. I've got more copies of the text around my house than I can count.

You can even buy a copy of the Book of Mormon yourself, if you're at all interested. But, if you're not, I can still assure you that it's full of thees and thous and haths and beholds.
_bschaalje
_Emeritus
Posts: 31
Joined: Fri Jul 16, 2010 8:03 pm

Re: Response to Jockers, Criddle, et al., Now Available

Post by _bschaalje »

aussieguy55 said:
It's interesting when one is an amatuer in these matters of the S/R theory to think wow they have done a kinghit on Jockers et al. Then a response to the paper in an email by one of the authors of the first paper, many flaws in the Schaalje paper are pointed out which I as a amatuer I did not see. The case in my opinion is definitely not closed. I'll ask the author if I can put their comments up here.

Please post the email. I'm definitely interested.
_harmony
_Emeritus
Posts: 18195
Joined: Fri Oct 27, 2006 1:35 am

Re: Response to Jockers, Criddle, et al., Now Available

Post by _harmony »

Daniel Peterson wrote:It does. I've got more copies of the text around my house than I can count.

You can even buy a copy of the Book of Mormon yourself, if you're at all interested. But, if you're not, I can still assure you that it's full of thees and thous and haths and beholds.


I was referring to Reformed Egyptian, of course.

I can't believe you missed the sarcasm! Or maybe you didn't... LOL
(Nevo, Jan 23) And the Melchizedek Priesthood may not have been restored until the summer of 1830, several months after the organization of the Church.
_Daniel Peterson
_Emeritus
Posts: 7173
Joined: Thu Jul 05, 2007 6:56 pm

Re: Response to Jockers, Criddle, et al., Now Available

Post by _Daniel Peterson »

harmony wrote:I was referring to Reformed Egyptian, of course.

I can't believe you missed the sarcasm! Or maybe you didn't... LOL

I didn't.

But, of course, serious point, all of this wordprint analysis focuses, necessarily, on the English text of the Book of Mormon. It has no direct relevance to the question of antiquity or of "reformed Egyptian."

(I do think, though, that it forms an integral part of a broader cumulative case very much related to the Book of Mormon's claimed antiquity.)
_aussieguy55
_Emeritus
Posts: 2122
Joined: Sat Nov 18, 2006 9:22 pm

Re: Response to Jockers, Criddle, et al., Now Available

Post by _aussieguy55 »

Hi Noel,

Yes, I have read the article and even with no knowledge of statistics the essential problems with the essay are clear.

1. Though the authors would like readers to believe that our use of NSC was "naïve" (as they write in the article), it was not in fact naïve. (It warrants mention that the statistician, Prof. Witten, who did the statistics on our project, is Tibsharani's former grad student. Tibsharani is the one who invented NSC. ) Our use of the algorithm was perfectly legit for what we applied it to. Schaalje misrepresents our objectives in order to create a straw man argument related to the business of a closed set of candidate authors.

2. Schaalje's primary complaint is that we use a closed set of candidate authors. That is a legitimate complaint and it is a point that we acknowledge quite clearly in the paper. That said, all authorship problems are ultimately closed set problems. You cannot test for every single person in the entire history of the world. Instead, you have some candidates and wish to figure out which among them is the most likely culprit. Our objective was to test *existing* theories of authorship of the Book of Mormon. And that's exactly what we did. We took the list of suspects and tried to rank them in terms of their likelihood. We point out in the paper that it is possible that the *real* author is not in the closed set. The real author could, for example, be Moroni or Napoleon. The point of our work was to "reassess" prior theories of authorship using machine classification. Our result is only compelling if you first accept the historical evidence and then accept that the candidates we tested represent a good set of candidates. The Schaalje paper creates a straw man argument and then plays fast and lose with the facts of our paper, cherry picking little bits here and there to make it look like we did something other than what we did, or that we had a goal other than what we had. In my opinion the fundamental conclusion of our work is that it lends additional support to the spalding-rigdon theory of authorship.

3. Schaalje makes a silly point of testing for Rigdon in the Federalist papers. No one ever suspected Rigdon of writing the Federalist papers. And then to make sure that his results pack the right punch to match his bias, he uses an entirely different feature set from ours and then, voila, they say that Rigdon shows up as most likely author of some of the Federalist papers. This is just plain hogwash.

4. Schaalje tries to suggest that text length is a factor. we checked that there was indeed no correlation between text length in our study and the results we derived. I can't remember if that was included in the final version of our paper.

For what it is worth, Schaalje does offer some interesting ideas but ultimately they are constructed out of an entirely different data set and used in an entirely different approach. And all of this they did in the context of statistics and formulas that only a highly trained statistician would understand. In my opinion, they should have first sought to have the statistical methodology reviewed and published in a journal of statistics. They suggest a modification of NSC and NSC is an approach that has been peer reviewed by statisticians who have the necessary expertise to evaluate it. In my opinion, Schaalje played fast and loose with everything in our own work in order to present a critique that would be pointless to try and rebut. Their paper is full of baffling formulas and mathematical slights of hand. In my opinion, it says virtually nothing. Unfortunately, it will achieve exactly what the authors hope, namely, it will provide them with a peer reviewed paper that they can hold up and claim to be a rebuttal of our paper. Anyone who can read and understand it will see that it is nothing of the sort, but few in Schaalje's target audience will, I suspect, actually try to read it.

Matt
Hilary Clinton " I won the places that represent two-thirds of America's GDP.I won in places are optimistic diverse, dynamic, moving forward"
_harmony
_Emeritus
Posts: 18195
Joined: Fri Oct 27, 2006 1:35 am

Re: Response to Jockers, Criddle, et al., Now Available

Post by _harmony »

aussieguy55 wrote:Hi Noel,

Yes, I have read the article and even with no knowledge of statistics the essential problems with the essay are clear.

1. Though the authors would like readers to believe that our use of NSC was "naïve" (as they write in the article), it was not in fact naïve. (It warrants mention that the statistician, Prof. Witten, who did the statistics on our project, is Tibsharani's former grad student. Tibsharani is the one who invented NSC. ) Our use of the algorithm was perfectly legit for what we applied it to. Schaalje misrepresents our objectives in order to create a straw man argument related to the business of a closed set of candidate authors.

2. Schaalje's primary complaint is that we use a closed set of candidate authors. That is a legitimate complaint and it is a point that we acknowledge quite clearly in the paper. That said, all authorship problems are ultimately closed set problems. You cannot test for every single person in the entire history of the world. Instead, you have some candidates and wish to figure out which among them is the most likely culprit. Our objective was to test *existing* theories of authorship of the Book of Mormon. And that's exactly what we did. We took the list of suspects and tried to rank them in terms of their likelihood. We point out in the paper that it is possible that the *real* author is not in the closed set. The real author could, for example, be Moroni or Napoleon. The point of our work was to "reassess" prior theories of authorship using machine classification. Our result is only compelling if you first accept the historical evidence and then accept that the candidates we tested represent a good set of candidates. The Schaalje paper creates a straw man argument and then plays fast and lose with the facts of our paper, cherry picking little bits here and there to make it look like we did something other than what we did, or that we had a goal other than what we had. In my opinion the fundamental conclusion of our work is that it lends additional support to the spalding-rigdon theory of authorship.

3. Schaalje makes a silly point of testing for Rigdon in the Federalist papers. No one ever suspected Rigdon of writing the Federalist papers. And then to make sure that his results pack the right punch to match his bias, he uses an entirely different feature set from ours and then, voila, they say that Rigdon shows up as most likely author of some of the Federalist papers. This is just plain hogwash.

4. Schaalje tries to suggest that text length is a factor. we checked that there was indeed no correlation between text length in our study and the results we derived. I can't remember if that was included in the final version of our paper.

For what it is worth, Schaalje does offer some interesting ideas but ultimately they are constructed out of an entirely different data set and used in an entirely different approach. And all of this they did in the context of statistics and formulas that only a highly trained statistician would understand. In my opinion, they should have first sought to have the statistical methodology reviewed and published in a journal of statistics. They suggest a modification of NSC and NSC is an approach that has been peer reviewed by statisticians who have the necessary expertise to evaluate it. In my opinion, Schaalje played fast and loose with everything in our own work in order to present a critique that would be pointless to try and rebut. Their paper is full of baffling formulas and mathematical slights of hand. In my opinion, it says virtually nothing. Unfortunately, it will achieve exactly what the authors hope, namely, it will provide them with a peer reviewed paper that they can hold up and claim to be a rebuttal of our paper. Anyone who can read and understand it will see that it is nothing of the sort, but few in Schaalje's target audience will, I suspect, actually try to read it.

Matt


So this is the email, huh?

I wonder what Daniel thinks of this?
(Nevo, Jan 23) And the Melchizedek Priesthood may not have been restored until the summer of 1830, several months after the organization of the Church.
_Joey
_Emeritus
Posts: 717
Joined: Mon Jul 09, 2007 1:34 am

Re: Response to Jockers, Criddle, et al., Now Available

Post by _Joey »

I suspect we won't be hearing to much more from the likes of Peterson (or Gardner) when it comes to accountability here anymore!
"It's not so much that FARMS scholarship in the area Book of Mormon historicity is "rejected' by the secular academic community as it is they are "ignored". [Daniel Peterson, May, 2004]
_GlennThigpen
_Emeritus
Posts: 583
Joined: Wed Apr 09, 2008 5:53 pm

Re: Response to Jockers, Criddle, et al., Now Available

Post by _GlennThigpen »

I expect that Bruce will be more than capable of speaking to Matt's criticisms. I don't see why he was complaining about Bruce using baffling statistics formulas. After all, the whole NSC deal is based upon statistics.
Testing Rigdon against the Ferderalist papers was not silly. Matt missed the point entirely there. Rigdon was tested against the Federalist Papers known to have been authored by Alexander Hamilton to show just how far off authorship attribution using a closed set NSC can be if the actual author is not in that closed set.
Matt noted that they were aware of and stated clearly there could be a problem with using an author set where the real author is not known. But they did not employ any means to test for that scenario, or at least they did not publish it.
For the tests on the Book of Mormon, Bruce et al used the same 110 words used by the 2008 Jockers study, and when applying the "naïve" NSC alstgorithm, their results were identical, except for one chapter.
The extended NSC results were quite different. Only a few random chapters were attributed to Spalding or Rigdon.
One of the extensions that Bruce et al added to the Jockers method was an algorithm to ensure that invalid results would not be returned when the real author is not among the supplied candidate sets. This was applied to the Rigdon tests on the Federalist papers. In the "naïve" run, Rigdon (without Hamilton in the candidate set) was falsely "chosen" as the author of 28 of the 51 texts. Here is the quote from the paper:
"Early or late Rigdon was falsely chosen as the author of 28 of the 51 Hamilton texts with inflated posterior probabilities ranging as high as 0.9999 (Fig. 2). Pratt was falsely chosen as the author of 12 of the papers, and Cowdery was falsely chosen as the author of the remaining 11 papers. These results dramatically demonstrate the danger of misapplying closed-set NSC."
When Bruce used his extended method, Rigdon showed up falsely on only two of the 51 papers while the other 49 were attributed to an unobserved author.
However, when Hamilton was added to the mix, not a single false positive was attributed to Rigdon or anyone else. Hamilton was correctly identified in each instance.

Now if someone can point out the flaws in that...... I really don't think Matt has a case there.

Glenn
In order to give character to their lies, they dress them up with a great deal of piety; for a pious lie, you know, has a good deal more influence with an ignorant people than a profane one. Hence their lies came signed by the pious wife of a pious deceased priest. Sidney Rigdon QW J8-39
_CaliforniaKid
_Emeritus
Posts: 4247
Joined: Wed Jan 10, 2007 8:47 am

Re: Response to Jockers, Criddle, et al., Now Available

Post by _CaliforniaKid »

Glenn,

I suspect Bruce and Matt would both agree that the difficulty of implementing a methodological modification should be in proportion to the analytical benefit it brings.

In other words, the problem with the complicated formulas from Matt's perspective is that they aren't really worth the effort. For an outsider and unbeliever in Mormonism such as Matt, the probability that the Book of Mormon was written by someone other than Matt's candidates seems vanishingly small.

Where Bruce would disagree is that because of his faith, Bruce assesses the probability of a non-candidate author differently, and thus sees more benefit from using an open set.

In principle, Matt is right. Given Matt's assumptions, Bruce's modification shouldn't have been expected to produce substantially different results. However, Bruce's results are very different. I don't see how Matt can ignore that. Something is wrong with Matt's assumptions. Either the author is not in his candidate set, or NSC is not returning accurate results when applied to the Book of Mormon.

in my opinion, the problem is the latter. All this quibbling about an open set vs. a closed set is like arguing about whether a flat head or a philips head screwdriver is preferable for performing brain surgery. Frankly, neither of these instruments is right for the job. Before agonizing over such a minor wrinkle in Matt's NSC methodology, Bruce probably should have addressed the larger problem of its apparent inapplicability to a text such as the Book of Mormon. Unlike the Federalist Papers, the Book of Mormon differs from the training texts in genre, style, content, and language. It is very naïve to assume the method will work in such a case.

Peace,

-Chris
Post Reply