Peer Review as Shadow Cancelling
If you think academics can avoid abuses by keeping out of politics, think again.
Fake reviews, vindictive editors, ignorant reviewers, “moderation” without reading, rejections for want of “a critical theory lens,” retention of submissions for a year without review, and defamation. If you think I gathered these abuses of peer review in only “woke” fields, think again. They’re problems in the hardest of sciences. And if you think academics can avoid these abuses by keeping out of politics, think again. Submissions are being rejected for their subjects or conclusions.
I received more stories of abuses than were released for publication, because of fears of professional retaliation. I will publish here only stories from academics prepared to go on the record, including myself.
Economic incentives
Both traditional subscription journals and open-access journals have a business incentive to protect the consensus. Traditional subscription journals are selling to potential readers. Open-access journals are selling to potential authors. In both cases, the consensus is the largest segment. Journals of both types also rely on volunteers, which adds a different economic incentive. Why would you review for free? One motivation might be to cancel whatever you don’t like. And even if a reviewer is not censorious, a volunteer is not economically motivated to review attentively.
All this helps to explain rising evidence for inattention at best, bias at worst. A hilarious test of the process found that psychology journals only realized that three out of 12 resubmitted papers had already been published! Of the nine that were reviewed, eight were rejected. Given that the two researchers had changed the papers’ authors and institutions (with such silly names as Tri-Valley Centre for Human Potential), they offered the results as evidence of bias against less prestigious institutions.
That was back in 1982. Have things improved? From 1993 to 2014, the frequency of errors corrected after publication increased steadily in three of the highest-ranked scientific journals (Nature; Science; Proceedings of the National Academy of Sciences). The British Medical Journal tested its own reviewers: none spotted all eight major errors in a mock submission. The median reviewer spotted two. Other medical professionals “observed that the quality of peer reviews has declined over the past decade, and their focus is often misguided, emphasizing grammar and style over science and methodology.”
Bias
David Martin Jones is a political scientist, Director of Research at the Danube Institute, and Visiting Professor of War Studies at London. Although he still publishes peer-reviewed books, he tells me that he has given up submitting to journals given their increasing bias.
Around 2010, Jones tells me, a new editor at Review of International Studies, in which he had previously published, “sent me a dismissive note to say the journal was not interested in articles with our viewpoint [on counterinsurgency] and wouldn’t even send it out to reviewers.” Around the same time, the Australian Journal of International Affairs redacted a co-authored essay, for “self-plagiarism” of an article in the Journal of Cold War Studies, and banned the two authors for seven years. However, Jones observed the same journal publishing articles that “have appeared as book chapters in edited books or reports or been abstracted from a monograph. … We were clearly selected for special treatment because we took a sceptical view of the Australian Labour Party’s premier [1972–5] Edward Gough Whitlam.”
M.L.R. Smith is another of Jones’s co-authors. “David and I joked that we were cancelled before it became fashionable to be cancelled” he tells me, “because we regularly submitted work that ran counter to the usual orthodoxies that pervaded academia. … Twenty years [ago] … David and I, along with another academic from the University of New South Wales, submitted an essay to Statecraft & Diplomacy. During the review process, one of the reviewers wrote to the academic at UNSW—who was the most junior amongst us—to tell him that he would be damaging his reputation if he proceeded with the submission, and advised him to withdraw his name from the article. … I wrote a letter of complaint to the editor exposing this, but (a) never received any response, (b) never received any review.”
Smith is careful to point out that many journals reject his submissions but show fairness, transparency, and timeliness. However, such journals are not the rule. “Getting rejected,” he tells me, “and taking the rough with the smooth, is part and parcel of academic life. That said, ever since I began submitting articles to journals three decades ago, I have got used to being on the receiving end of inept, partisan, cavilling, and unconstructive reviews, or weak and unresponsive editors. Literally, they are too numerous to mention or to recall. What I might add is that poor reviewing and unprofessional practice in my experience tend to constitute the norm rather than the exception. In particular, one might single out the abuse of the process of anonymous reviewing where reviewers can say more or less anything without any form of accountability. … I would add that what contributes to the problems identified are: (a) often longstanding editors who run their publications like little fiefdoms, and (b) poor oversight from journal publishers.”
Smith was awarded the chair of the War Studies Department in 2016. A competitor for the chair was one of the editors at the Journal of Strategic Studies. Coincidentally or not, the journal dropped Smith from the editorial board shortly thereafter. “Articles that I have submitted have been instantly desk rejected,” he tells me, “even though the submissions are explicitly orientated around strategy. I came to regard the predictability as almost comical, to the point where I would intentionally submit manuscripts to the journal only to see how quickly the desk rejection would come back.”
In 2022, I submitted an article to the Journal of Strategic Studies challenging the conventional presentations of “indirect approach,” a term coined in the 1920s by Basil Liddell Hart (1895–1970), a full-time journalist, part-time military historian and strategist. Liddell Hart never formally defined the term, and after the Second World War, he would misrepresent everything he wrote as Blitzkrieg. Nevertheless, Liddell Hart remains a sacred cow in military history, and particularly in War Studies, the founders of which he mentored.
In September, an editor rejected my submission, citing two anonymous reviews. But these reviews contain nine sentences between them. Both direct me to a particular historian (Azar Gat), thus proving that they had not properly read my article or that historian. My submission complains that Gat had repeatedly praised the “indirect approach” without reviewing it.
I let it go and submitted to the Journal of Military History instead. In February, its editor (another stranger) cited “two experts … that say that your argument is nothing new; moreover, they indicated to us that they felt your work bordered on intellectual and academic dishonesty by not giving due credit to those who have made the argument before.” He concluded: “we advise you in future to be more careful.” I didn’t let this one go. Within minutes of receiving the email, I challenged him to provide evidence. “Where are the reviews? Who are the authors I supposedly haven’t credited?” His backtracking is revealing:
No one accused you of anything, if you read the note. What it says is that the reviewers, who are anonymous and will remain that way, felt that the work bordered on academic dishonesty. I am confident of them, but they will remain anonymous; that’s how the process works. I do not feel it is appropriate, in this instance, to share the reviews as they would compromise that anonymity.
I refuted his mischaracterization of the peer-review process. He never replied. With the help of the president of the Society of Military History, I got hold of two reviews, and learned of a third. One reviewer states, “I am not an expert on British military thought or historiography,” and recommends sending the submission to another reviewer. The second writes: “I read an earlier version of this manuscript for [Journal of Strategic Studies], and it was not suitable for publication then. There is nothing in this version that changes my mind.”
So, two reviewers don’t know the literature enough to judge my submission. Neither accuses me of dishonesty. The president says that the third review is so personal and unsubstantive that it would be neither “useful” nor “anonymous.” That’s three useless reviews. The editor waited on the third before defaming me, without the confidence to share any evidence, and without keeping straight the number of reviews he had commissioned.
Selectivity
Academic journals and repositories are starting to look like capricious social media. A good example is arXiv, which describes itself as a “free distribution service and an open-access archive for 2,222,050 scholarly articles in the fields of physics, mathematics, computer science, quantitative biology, quantitative finance, statistics, electrical engineering and systems science, and economics.” But arXiv is not simply an online repository, it’s also a publisher of preprints that have not been published elsewhere.
Submissions are not peer-reviewed, so “moderators” decide what gets in. The arXiv webpage on moderation contains many words but doesn’t really get around to specifying the process—the criteria are mostly stylistic (such as “formatting”). The most substantive criterion is “scholarly interest,” which is subjective at best, and smacks of consensus-chasing at worst.
In 2004, Nobel Laureate physicist Brian Josephson of Cavendish Laboratory, Cambridge, blogged about being “blacklisted” by arXiv—which is to say that the website wouldn’t process his attempts to lodge papers. After his permissions were restored, his papers were barred from the areas in which he specialized. In 2009, the physicist Philip Gibbs set up an unmoderated competitor (viXra), arguing that open access promotes the best moderation.
In 2015, Nicolas Gisin of University of Geneva, who has reposed more than 300 papers under arXiv’s section on quantum physics, blogged about arXiv’s refusal of a paper by two students. The paper was later published in Physics Letters. By then, arXiv reposed more than 1.1 million papers. Gisin blogged that any failure to post on arXiv damages careers, but when he asked arXiv for an explanation, he received only the online criteria. The journal Nature asked the then chair of arXiv’s Physics Advisory Committee (Daniel Gottesman) to respond. Gottesman told Nature that arXiv could not comment on individual cases, and added: “there is no arXiv blacklist.” Gisin tells me: “I believe there was no change, hence no improvement, since my contribution to the blog and the Nature article.”
In July 2022, dozens of astronomers and physicists published a petition against arXiv’s censorship of challenges to Big Bang Theory. One of the joiners is Mike McCulloch of Plymouth University. His main subject is Quantized Inertia (QI), which he is applying as a new form of propulsion.
The preprint arXiv … is the main method by which physicists read each other’s work. It was originally, pre-2012 or so, very open and published everything that looked at least coherent. It was useful to me at that time as well, since I had access to every paper being published and I used the astronomical data there to help develop QI. … A pattern has emerged that affects my papers and those of others challenging the dark matter/big bang status quo. Our papers, even those peer-reviewed and published in good journals, have been rejected by the anonymous and unaccountable editors at the arXiv, and, most worryingly, for no stated reason. To be banned from the arXiv is tantamount to academic oblivion in physics.
Another critic is Professor Espen Gaarder Haug, of the Norwegian University of Life Sciences, who publishes in physics, mathematics, and finance. For the first two disciplines, he tells me, arXiv is the primary repository. Haug has complained of submissions being held for months before rejection, which leaves time for others to take credit for discoveries. Haug notes that arXiv has rejected papers already published in highly ranked journals, such as European Physics Letters, and Astrophysics & Space Science. Haug tells me:
ArXiv has rejected a series of already peer-reviewed papers that already have been published in good journals. And they have been made fully aware of it. … My experience is they block in particular papers questioning the foundation of ‘established’ standard theories. In addition, I think it is also partly personal. … Still, even Nobel prize winners in physics have been censored by arXiv.
Haug shared this unhelpfully vague rejection with me:
We regret to inform you that arXiv’s moderators have determined that your submission will not be accepted and made public on [arXiv]. While we acknowledge that this article has been published, our moderators determined it is not of plausible interest for inclusion within arXiv. As a result, this submission has been declined.
Since 2021, Haug has raised his concerns with arXiv and Cornell University, which hosts arXiv. Jim Entwood is employed by Cornell as Head of Content & User Support at arXiv. In response to Haug’s question of how the moderation works, Entwood wrote:
ArXiv is a private collection of content curated by a network of volunteer moderators. While the moderators try to ensure the articles are selected based on the interest of each specific research community, being as inclusive as possible while maintaining scientific standards, arXiv does not offer peer review, which is an expensive process to manage.
I contacted Entwood to ask if anything has changed since 2021. He copied to me some webpages, before deferring to Professor Ralph A.M.J. Wijers, who has “chaired” physics at arXiv since 2021. Wijers tells me that:
[The Subject Advisory Committee] select[s] moderately senior and experienced scientist[s], both for their broad overview and for not being too vulnerable to attack should their judgments be disputed. … If authors disagree with a rejection or reclassification, they can appeal. Appeals are heard by special appeals moderators, typically with a broader view and experience (e.g., physics has about 70 moderators and 4 appeals moderators). Normally, this is the last level of appeal, except if the authors feel matters of principle or due process have arisen, in which case they may further appeal to the chair of the SAC. ArXiv does not review. It only moderates, and does so quickly, usually not on the basis of the full text. To do so, author info is important so we do not do it anonymously.
Conclusion
Is this the new model of peer review? No anonymity; decisions reached in haste and “not on the basis of the full text”; unaccountable reviewers; consensus disguised as “scholarly interest”? I hope not. If peer review reduces to personality and consensus, confidence in academia would collapse generally, not just in the fields that censored certain theories on the origins and treatments of COVID-19. That would be bad for everybody.
