Throwing shade at Signature Books

The catch-all forum for general topics and debates. Minimal moderation. Rated PG to PG-13.
Post Reply
drumdude
God
Posts: 5215
Joined: Thu Oct 29, 2020 5:29 am

Throwing shade at Signature Books

Post by drumdude »

DP wrote:Jacob Ames: "I find it funny that ex-members are eager to say that papers from experts haven't been peer-reviewed. If they have been peer-reviewed, it doesn't matter if it wasn't peer-reviewed by a non-member. But then they all talk about non-peer reviewed works such as Hassan's BITE model, or dubious "historical" works like Vogel's. They seem to want their cake and eat it too."

Indeed.

I'm not sure that Signature Books -- which has published most if not all of the works of D. Michael Quinn, Dan Vogel, etc. -- uses anything like academic peer review.

I personally and directly know, though, that the Interpreter Foundation does use a peer review process that we've very consciously patterned after the mainstream peer review process used across academia. So did the Maxwell Institute and FARMS when I was there.

So it's amusing to see some folks deny that Interpreter and FARMS/Maxwell are peer reviewed and then turn around to treat Quinn, Vogel, and the like as if they had been published by Oxford University Press.

Personally, I think that peer review can be helpful but that it shouldn't be fetishized.

Interpreter's Mission statement 2012, with no details of the peer review process
Interpreter Foundation is a nonprofit educational organization focused on the scriptures of The Church of Jesus Christ of Latter-day Saints: the Book of Mormon, the Pearl of Great Price, the Bible, the Doctrine and Covenants, early LDS history, and related subjects. All publications in its journal, Interpreter: A Journal of Mormon Scripture, are peer-reviewed and are made available as free internet downloads or through at-cost print-on-demand services. Other posts on the website are not necessarily peer-reviewed, but are approved by Interpreter’s Executive Board.
Around 2017 Interpreter expanded their explanation of their single-blind peer review process:
Interpreter utilizes a single-blind peer-review system. This means that the author does not know the names of those who review the paper, but the reviewer may know the name of the author. We chose to not use a double-blind system (where both author and reviewer are unaware of the name of the other). Mormon studies is a very, very small portion of the overall academic ocean. Because of this, double-blind is impractical because most reviewers would recognize authors (at least those published previously in Interpreter or in other academic venues) based on their writing style, subject areas, and other internal clues regardless of whether the author’s name was removed from the paper or not.
Around 2019 Interpreter reversed their decision to use single-blind, and claimed to use a double-blind peer review process:
Interpreter utilizes a double-blind peer-review system in as many instances as possible. This means that the author does not know the names of those who review the paper and the reviewers do not know the name of the paper’s author. At an editor’s discretion, and on a case-by-case basis, an article may only undergo a single-blind peer review. (This is a peer review where a reviewer may know the name of the author, but the author still is unaware of who the reviewer is.) For instance, it makes very little sense to do a double-blind peer review on a paper that is expanded from a previously published article or is based on a public presentation by the author.
Signature Book's process:
All of Signature’s books go through a rigorous evaluation process that includes several readers and may take up to six months. We are not able to respond personally to all queries, especially to unsolicited submissions. Sending an unsolicited submission and not receiving a response within a month or two thereafter means that we have decided to pass on your submission.
Marcus
God
Posts: 5036
Joined: Mon Oct 25, 2021 10:44 pm

Re: Throwing shade at Signature Books

Post by Marcus »

I heard the statistician who peer reviewed the Dale fiasco retired in shame, and has vowed to never speak or write a number again.
User avatar
Physics Guy
God
Posts: 1557
Joined: Tue Oct 27, 2020 7:40 am
Location: on the battlefield of life

Re: Throwing shade at Signature Books

Post by Physics Guy »

Most physics journals use single-blind review, and the only reason I’ve heard is the same, that subjects are so specialized that everyone who might be a good reviewer will recognize the authors from the paper’s content. At least in many cases I’m sure that’s true.

The problem is that Mormon studies is a small field, in the sense that few people study in it, but also broad, in that the range of relevant subjects is large. Too few—if any—in the list of faithful Mormon scholars who review for Interpreter know much about statistics or linguistics, for example. Yet papers are submitted to Interpreter that involve such poorly known subjects crucially.

The result is that Interpreter reviewers must often be under-qualified in their roles. So it doesn’t matter how closely the peer review process resembles that of reputable journals. The best reviewers in Interpreter’s limited pool aren’t always up to the job.

Peer review is not a magic spell that guarantees sound publications if only the process is followed correctly. It’s only as good as its pool of reviewers, and the reviewers are only as good as their expertise in the subjects that are important in a given paper. When too many reviewers lack that relevant expertise, even if they are otherwise smart and knowledgeable, peer review doesn’t mean much.

I’m not sure how much Interpreter can really do about this. One option is not to publish papers that seem to hinge on technical subjects that no reviewers know well, even if the papers look good to a lay person. Another is to call in expert reviewers for those papers from outside the Mormon studies community—if necessary by paying them for their reviews.

The only other option that I can see at the moment is for Interpreter to remain a somewhat amateur journal that frequently publishes nonsense. That’s too bad for the authors that do publish solid work in Interpreter, and for the reviewers and editors that put thought into papers for which they are competent critics. A thin but steady stream of Dale or Carmack garbage makes every Interpreter paper look dubious.
I was a teenager before it was cool.
drumdude
God
Posts: 5215
Joined: Thu Oct 29, 2020 5:29 am

Re: Throwing shade at Signature Books

Post by drumdude »

Physics Guy wrote:
Thu Jul 21, 2022 3:30 am
The only other option that I can see at the moment is for Interpreter to remain a somewhat amateur journal that frequently publishes nonsense. That’s too bad for the authors that do publish solid work in Interpreter, and for the reviewers and editors that put thought into papers for which they are competent critics. A thin but steady stream of Dale or Carmack garbage makes every Interpreter paper look dubious.
This is the path they have taken in my opinion. And the token switching from single to double blind review doesn't magically make Interpreter more respected or trustworthy. The fact that they switched at all, given its uselessness, seems to indicate they know the journal has an integrity problem and they want to be seen as doing something to solve it.

Compare the meticulous work of Michael Quinn and Dan Vogel to the flood of nonsense that spews out of Interpreter every week. Any objective eye can tell which is of superior quality.
User avatar
Moksha
God
Posts: 5810
Joined: Wed Oct 28, 2020 3:13 am
Location: Koloburbia

Re: Throwing shade at Signature Books

Post by Moksha »

Giving a review of opinions about fictional works seems very important. To be considered for such a peerage, one must first be a freshman BYU student who has volunteered for this assigned reading for extra credit. They must add their initials to show they've read the material. Initials must be returned to the assistant editor by Friday at noon for the extra credit. No further discussion is required or desired.
Cry Heaven and let loose the Penguins of Peace
User avatar
Gadianton
God
Posts: 3842
Joined: Sun Oct 25, 2020 11:56 pm
Location: Elsewhere

Re: Throwing shade at Signature Books

Post by Gadianton »

This is odd. How many hours did FARMs spend peer-reviewing Hugh Nibley's works before publishing them?

Signature Books isn't an academic journal.

Interpreter is pretending to be an academic journal, but it never will succeed and it will never have peer review in any meaningful sense. That it published the Dales and KR are the most epic examples of failure. Everyone at Interpreter is an apologist set out to lie not only to the audience, but themselves. Apologist outlets would to well to really get peer review going for the most outrageous claims and work on cleaning up the self-deception, as that will help them deceive others more effectively.
User avatar
Dr Moore
Endowed Chair of Historical Innovation
Posts: 1812
Joined: Mon Oct 26, 2020 2:16 pm
Location: Cassius University

Re: Throwing shade at Signature Books

Post by Dr Moore »

Gadianton wrote:
Thu Jul 21, 2022 3:34 pm
Interpreter is pretending to be an academic journal, but it never will succeed and it will never have peer review in any meaningful sense. That it published the Dales and KR are the most epic examples of failure. Everyone at Interpreter is an apologist set out to lie not only to the audience, but themselves. Apologist outlets would to well to really get peer review going for the most outrageous claims and work on cleaning up the self-deception, as that will help them deceive others more effectively.
That last sentence, oof. So true, but unlocking that potential energy, likely not an input investment anyone would be willing make.
Post Reply