Education, recent, Top Stories

Academia’s Case of Stockholm Syndrome

Earlier this year, we launched Researchers.One, a scholarly publication platform open to all researchers in all fields of study. Founded on the principles of academic freedom, researcher autonomy, and scholarly quality, Researchers.One features an innovative author-driven peer review model, which ensures the quality of published work through a self-organized process of public and non-anonymous pre- and post-publication peer review. Believing firmly that researchers can and do uphold the principles of good scholarship on their own, Researchers.One has no editorial boards, gatekeepers, or other barriers to interfere with scholarly discourse.

In its first few months, Researchers.One has garnered an overwhelmingly positive reception, both for its emphasis on core principles and its ability to attract high quality publications from a wide range of disciplines, including mathematics, physics, philosophy, probability, and statistics. Despite its promise, many academics worry that leaving peer review up to authors will grind the academic juggernaut to a halt. With nothing to stop authors from recruiting their friends as peer reviewers or from publishing a bunch of nonsense just to pad their CV, how should academic researchers be judged for hiring, tenure, or promotion? Without the signal of impact factor or journal prestige, how should readers assess the quality of published research? On their own, such questions are quite revealing of the predominant attitude toward academic publishing, which treats peer review as a means to an administrative end rather than an integral part of truth-seeking.

When done right, peer review is a rigorous process that fosters honest critique, lively discussion, and continual refinement of ideas for the mutual benefit of researchers and society. When done wrong, peer review plays to the worst instincts of human nature, devolving the pursuit of knowledge into a spectator sport in which the credibility of individual researchers, prestige of institutions, and legitimacy of scholarship as a whole are staked on the appearance of quality, objectivity, and novelty that the “peer review” label brings. As the above questions indicate, the prevailing mindset focuses on all that is wrong, and very little of what is right, with peer review.

A far cry from its ostensible scholarly mission, academia today resembles “a priesthood or a guild” or even a “cult“, with peer review serving an essential administrative need in a system of promotion, tenure, funding, and accolades designed to maintain the established order. Under this model, political posturing and bogus theorizing have become indistinguishable from scholarship in some fields. Identity politics and demagoguery have even infected mathematics, as seen in the recent saga of Ted Hill’s twice accepted, once published, and twice rescinded article on the Greater Male Variability Hypothesis. In health-related disciplines, with big money at stake, conflicts of interest dictate which results are reported, and what stays in the file drawer. In our own field, small cliques monopolize the flagship journals, as nearly half of all articles published in The Annals of Statistics are authored by a member of its own editorial board.

Even ignoring the subversiveness and pettiness that current norms enable, this system fails in its major promise to ensure the quality of published literature. It is widely believed that most published scientific findings are false: a recent analysis successfully replicated only 61 percent of social science findings published in Science and Nature, and an earlier study replicated just 39 percent of findings from top psychology journals. Scientists who were once celebrated have become maligned amidst allegations of fraud or shoddy scholarship. Meanwhile, thoughtful, potentially transformative ideas struggle to see the light of day, as chronicled in Francis Perey’s four-decade quest to publish foundational work on probability theory. On top of these well-publicized incidents, individual researchers privately trade stories about their own experiences with discrimination, incompetence, and obfuscation in their respective disciplines.

Paralyzed by a severe case of Stockholm syndrome, career academics persistently complain about these problems while simultaneously insisting that substantive reform to current practices “would do more harm than good.” Fittingly, recognizing the ills of this system and vowing to do something about them has itself become a signal of belonging, as leading new initiatives within the boundaries of the current paradigm is now a surefire way to gain recognition and advance one’s career. Such initiatives are also guaranteed to leave the status quo in place, modulo cosmetic changes. It is common to appoint committees, assemble task forces, propose best practices, host panel discussions, and raise awareness, all on the false premise that—with just a few tweaks—the very same administrative process founded on filtering, suppression, and signaling will reform itself into a bastion of freedom, autonomy, and truth. Notably absent are solutions that will elevate peer review above bureaucracy and administrivia, restore science to its scholarly, truth-seeking purpose, and entrust scholars with control over their own scholarship.

Standard bureaucratic remedies, which enact more draconian measures and impose greater oversight, only reinforce academia’s toxic “publish or perish” culture, embolden editorial boards, drive a wedge between researchers, and worsen the replication crisis. By “raising the bar” for publication, these new policies bestow even more credibility to, and thus increase the impact of, the (inevitable) fraction of unreliable results that makes it through peer review’s filter. Even supporters of the current model, such as Aaron Carroll, recognize the folly of trusting in peer review: “Too often, we think that once a paper gets through peer review, it’s ‘truth.’ We’d do better to accept that everything, even published research, needs to be reconsidered as new evidence comes to light, and subjected to more thorough post-publication review.”

Science thrives by fostering what Feynman called a culture of doubt, not of consensus or signaling. In line with Feynman’s view, Researchers.One eliminates restrictions and removes barriers to publication, and with them also the credibility and prestige conferred by the “peer-reviewed” label. In turn, the platform offers a number of innovative features—and is developing several more—to facilitate pre- and post-publication peer review for the purpose of improving quality and facilitating discussion. Though most scientists and journal editors agree that replication studies and negative results are important to maintain checks and balances, most journals, concerned with their impact factor and prestige, are unwilling to publish them. Researchers.One, by contrast, welcomes and encourages positive or negative findings, original studies or replications, expository or research articles, and whatever else an author wishes to disseminate.

Those initiated into the conventional mindset may struggle to appreciate the virtues of autonomous peer review, wondering how making it easier to publish could possibly improve the state of affairs. Understand this: the point of Researchers.One isn’t to make publishing easy; it is to make publishing trivial, so that the worth of an idea is judged on its merits alone. Because publishing on Researchers.One comes with no stamp of approval, conferral of credibility, or associated prestige, readers should approach ideas published on the platform with a healthy dose of skepticism. Aware of the reader’s skepticism, authors have every reason to solicit rigorous and critical peer review feedback to ensure that their work stands up to scrutiny. The ease with which low-quality work could be published on Researchers.One thus fuels the natural skepticism necessary to keep readers alert and researchers in check, to mitigate the impact of errant findings, and to keep science on track, all without the interference of academic oligarchs. We call this process, by which quality improves by eliminating existing quality control mechanisms, scholarly mithridatism, after Mithridates VI, who is said to have purposely ingested small doses of poison as a counter-measure to an assassination attempt.

We don’t propose to cure academia’s Stockholm syndrome by changing its captor: the kind of reform we have in mind with Researchers.One is not one in which a small group of elites instantiates its own version of the current top-down system. We insist on a more organic and dynamic shift. Researchers.One is not intended as a replacement or an improvement to the current system. It is instead an alternative that offers all the scholarly benefits of peer review without the administrative overhead, and which empowers individual researchers to take control of their own destiny. As such, the success of this unconventional platform can’t be measured by conventional metrics such as impact factor, rank, or prestige. Rather, the success of Researchers.One lies in its very existence, as a forum open to anyone who yearns for true academic freedom, open access, viewpoint diversity, and for something more fulfilling than the next rung on the academic ladder.

 

Harry Crane is Associate Professor of Statistics at Rutgers University. You can follow him on Twitter @HarryDCrane or visit his website for further information.

Ryan Martin is Associate Professor Statistics at North Carolina State University. You can follow him on Twitter @statsmartin or visit his website for further information.

34 Comments

    • You know something isn’t right when a hoax and scare tactic like global warming becomes a consensus of opinion among Western academics while those outside the west give climatology the credibility of the ancient science of astrology. Being a Left vs. right issue pretty much tells you the AGW hypothesis isn’t science.

  1. Qualia says

    “the point of Researchers.One isn’t to make publishing easy; it is to make publishing trivial, so that the worth of an idea is judged on its merits alone. Because publishing on Researchers.One comes with no stamp of approval, conferral of credibility, or associated prestige, readers should approach ideas published on the platform with a healthy dose of skepticism. Aware of the reader’s skepticism, authors have every reason to solicit rigorous and critical peer review feedback to ensure that their work stands up to scrutiny.”

    What a wonderfully novel and creative incentive structure. I think this offers a fantastic alternative to the many current problems in academic publishing. Hope it is successful.

    • Optional says

      Physics and Math have had arXiv.org for at least 20 years. Same idea.
      It is well loved and well respected. But is has also not remotely fixed the problem.

      • arXiv has had a major impact in physics and math, which rely much less on journals now. But arXiv and Researchers.One are not at all the same idea.

        arXiv is a repository for pre-prints, doesn’t offer peer review (pre- or post-publication), and in most fields, journal publication is still expected after posting to arXiv.

        Researchers.One offers much more than arXiv. One could use Researchers.One as a repository — still choosing to publish elsewhere — but that is a very limited way to use the platform. There are a number of other features on Researchers.One that are either currently available or in the process of being developed.

  2. Farris says

    Researchers.One appears to be an excellent idea and forum to allow research and ideas to escape bureaucratic entanglements and see the light of day. In other words it contributes to a free and open debate. Poor ideas and shoddy research will rightfully die on the vine. Can any half baked idea be published? Sure but does one really want to risk the humiliation of having shoddy work revealed?
    One area the author did not address is the platform. What happens when an article steps upon the wrong toes? What happens when pressure is brought to bear on the platform provider? Any research or conclusions that do not pay homage to the proper myths, totems or deities of the Left will be deemed “Hate speech”.

    • E. Olson says

      The recent “grievance studies” scandal illustrates that poor ideas and shoddy research can apparently never humiliate the authors or the discipline as long as the work does pay proper homage to the “proper myths, totems or deities of the Left”.

      • Alan D White says

        Social studies lends itself to left/right slanting. This should not represent a problem since the nations left/right political representation is about 50/50……
        I don’t see STEM research having the left/right problem (climate change excluded).

  3. A fellow statistician here. I’ve been saying for a while that we are leading the way in making the computer-implementation of academic research available to anyone, through R, but we are lagging in making the research itself available.

    Like you I’ve been in many discussions about how the current academic publishing model is total crap and just drives bureaucracy, game-playing and rent-seeking. Good to know that people with more energy than me are doing something about it.

    There is of course the risk that it falls into an abyss of nonsense in the way that most “social” media have, but best of luck with the project.

  4. Orion Buttigieg says

    Described in this article is how we got AGW (man made global warming). See Climategate emails where political pressure from various angles was applied to the top editors of key science journals to NOT publish dissenting science. We now have people believing like a religion that a trace gas that has some IR qualities which makes up many ~3% of the GHG’s and man’s contribution of maybe ~3% of that is the dial on some perceived global thermostat. Not one repeatable experiment to prove it and their own damn models aren’t just wrong but about as wrong as wrong can be. Enter politicians inventing ways to spin this pseudo science into new means of taxation and viola.

    • Farris says

      “If AGW is true, then stopping or preventing it requires higher taxes, more income redistribution, more wilderness preservation, more regulations on
      corporations, “smart growth,” subsidies for renewable energy, and on and on. In other words, many of the policies already on the liberal political agenda. Liberals have no reason to “look under the hood” of the global warming scare, to see what the real science says. They believe in global warming because they feel it justifies their ideological convictions.” (Hulme, 2009)

    • Event Horizon says

      When I was a young scientist (physics), I used to default to the scientific consensus in areas where I was not an expert. I simply didn’t have the time to really look at issues I was not directly working on. I used to think “Surely, on average, every scientific community is honest and does good work”. Then at some point with all the hoopla about climate change I decided to look at the hard data. I was absolutely horrified. NASA’s claim that one can measure the average temperature of the entire planet year round to within a fraction of a degree when enormous swaths of land and sea are not even probed is absolutely bonkers. Greenland hosts 6 stations, all on the shore, none in the interior. No station at the North Pole. Two stations in the South Atlantic and one in the North Atlantic… for an ocean that covers 20% of the planet, and extends from 70 deg North to Antarctica in the South.

      Look for yourself:
      https://data.giss.nasa.gov/gistemp/stdata/

      The red dots are where there all the weather stations are. Anyone that thinks this is a valid measurement is crazy. I challenge all those claiming otherwise to measure the average annual temperature of a closed, non air conditioned ten story building with 8 thermometers (in other words, no temperature data from at least two floors). I bet you 10 different climatologists will come up with ten different results varying by far more than one degree.

      And I didn’t even go into how NASA massages the station data… which is crazier still.

      The US has the best long term data. If you look at rural weather stations in the US (urban sprawl does influence the measured temperature locally – this is a well known urban effect) there is no significant warming that I can see.

      • Stephanie says

        @ Event horizon

        I had a similar experience. In college I bought into the hype because like most around me I was successfully indoctrinated by leftism. When I started my undergrad in geology, I started to have some serious doubts.

        The difficulty in determining the current average temperature is a serious problem, but it’s even worse if you look at how they reconstruct past temperatures. One ice core-derived temperature here, one shell-derived temperature there… It’s a coarse patchwork of model temperatures, each with significant age and temperature uncertainties. If you propagated all these uncertainties properly, I suspect the picture would be very different. That’s why climate change papers make very little effort to do so. Since everyone in the field is aware the true uncertainties eclipse their narrative, no one holds anyone accountable for it.

        There’s hard geological evidence the world has been warming for 12 000 years, but skepticism that it’s sped up by humans is appropriate given all these unaccounted uncertainties. I tend to think it is in some (probably minor) proportion, but the predictions of future climate doom I think deserve little concern. As my supervisor says “models are always wrong,” and models where each parameter has a massive uncertainty are likely to be very wrong.

  5. Stephanie says

    Great idea, I wish you guys success! I’ll check it out and talk to my supervisor about publishing there.

    I’m another one of those who’s always complaining about the mainstream publishing system. That publishing houses make 50% profit while using free labour to do all the hard work reviewing manuscripts, and then have the gall to charge researchers access to their own work is obscene. It is a misallocation of government funds to hide the work produced with those funds from the public who pays for it. A complete gutting of the system is required, hopefully this new journal will be another step in that direction.

    However, as a PhD student in a saturated field, my future job prospects are still highly dependent on which journals I publish in. There’s no use putting all my work in the kind of journals who’s business model I approve of, if it means I lose out permanent positions to those who go the traditional route.

    Then again, I met someone at a conference who publishes one a year in alternative journals, and when I asked him why he doesn’t send them all there, he said there’s never a time in a researcher’s career when you don’t need papers in those reputable journals to get funding. Hopefully time and the important work done by people such as the authors will change that.

  6. Stephanie says

    I’d also like to add that science has a replication issue as well, at least in geology. Not the same way as the social sciences, but there’s little interest in working on a something that’s already been worked on. What ends up happening is a PhD student works on a suite, comes up with a model, and then no one revisits it again. So instead of building a larger database and verifying and refining results, people fracture off to do glitzy isotope work that they can pitch as “novel.” There’s a risk working on a suite that’s previously been published on that you won’t find anything new, and that fear scares off students and would-be supervisors alike. This becomes a problem when the original work was poorly done or insufficient samples were analysed to capture the processes at work.

  7. Andrew Leonard says

    A problem I perceive with the peer review model is that original, pre-peer review papers are never published. There is no means of determining the quality of the peer reviewing itself. Therefore, we cannot determine what thoughts, criticisms, alterations or agendas of any kind (career, scientific, political, etc), were a part of the peer review process for any paper, nor how careful and thorough was the peer review process.

    I propose a ‘publish everything’ model.

    • Original papers should be published in parallel with the completed, peer reviewed version
    • Peer review notes should be published in parallel with the original and final paper
    • A mechanism for reviewing peer reviews should be established – who watches the watchers in the current model?
    • Rejected papers should be available online, along with rejection notes, so we can see the what, who and why of rejection. The domain RejectedPapers.net appears to be available

    I would also suggest moving the emphasis from quality control, which gives too much power to the scientific incumbents in any field, to quality ascertainment. A strong example of a quality ascertainment mechanism is the Amazon book review and rating process.

    • X. Citoyen says

      Mostly good ideas. Might as well allow (moderated) comments too. A comment is basically a letter response sent to the journal that can be submitted in real time.

  8. E. Olson says

    Will publications in Researchers.one count towards tenure, promotion, and salary decisions at research oriented universities and departments?

    Will non-PC publications in Researchers.one prevent their authors from getting mobbed or fired for having “impure” thoughts?

    Will publications in Researchers.one that cast doubt on existing theory/paradigms with a long literature trail authored by the the discipline’s top scholars and published in its top journals be viewed as “valid” since they did not pass traditional peer review?

    If the answer is “NO” to these 3 questions, I have strong doubts that the Researchers.one platform
    (or similar) will ever become a significant new publishing model for scientific thought and research.

  9. The Amazon process also has problems. There is the obvious isue that books with only a few reviews are easily skewed by the authors friends but a less obvious effect because narrow groups of peope with similar tastes and interests review books within those zones. That might not seem an issue but my experience is that extremely formulaic poorly written, predictable books that are within a well defined genre generally get good reviews but all thd review really says is that it fits into the genre. This makes it impossible to compare across genres and devalues the rating system as a whole. The effect if applied to scientific papers could be even more pressure to conform and bias aganist novelty or innovation.

    • Andrew Leonard says

      I see your points, but the question is to what extent those issues would replicate for scientific research.

      When a friend writes a glowing review of a book, the world probably doesn’t realize there is a connection between author and reviewer. Even if this was revealed, the reviewer at least doesn’t suffer any loss of professional reputation, because they are not reviewing in a professional capacity. Would scientists and researchers risk compromising their reputations in the same manner? Probably not, and besides, who can design a social process that does not leave something up to professional attitudes?

      Also, I doubt papers written for specialist audiences would be as subject to box-ticking type reviews as are formulaically written books.

  10. ga gamba says

    I commend you for taking the bull by the horns and wish you well in your endeavour.

    • Fascinating subject, way outside mainstream media’s focus, but deserves a modicum of attention since the side effects of these subjects often land on the evening news and are treated as gospel. This is particularly true if the so-called science affirms liberal/left agenda – as someone here pointed out in an earlier comment.

      The settled science of the self-proclaimed experts actively suppresses new information often in the form of preventing publication. I follow the Plasma/Electric Universe theory of cosmology which is treated with absolute derision even after a number of predictions have been validated by modern observational tools – but are in opposition to 100+ year old mathematical theories. The mainstream scientists “surprised” or see “unexpected behavior” will rename descriptions and effects predicted by P/EU scientists and slowly integrate the science into the mainstream.

      From the stories I’ve read there is a kind of protection racket going on for the status quo in the pursuit of scientific knowledge which is so unfortunate for progress.

      • Paul Ellis says

        There always is a protection racket going on for the status quo, because those who gain net benefit from the status quo seek to preserve it. This simple observation explains most workings of societies. It’s a synonym of ‘follow the money’.

        • @Paul Ellis
          Right, but this is science – the pursuit of knowledge or some such higher purpose. It shouldn’t be tolerated. thanks, now I’m back to reality

  11. markbul says

    ““Too often, we think that once a paper gets through peer review, it’s ‘truth.’ \

    When I was in grad school in the late 1990s, our lab spent an hour each week going over a paper we had read together. Usually, we were picking it apart, or wondering how it ever got published. That was REAL science. Peer review was only ever meant to get you in the discussion – not to establish truth. It’s only been since the eruption of the ‘but it’s peer reviewed, so it must be right!’ global warming meme that anyone thought published papers were revealed truth.

  12. X. Citoyen says

    I like this idea. It might not be the solution to everything, but experimentation is necessary. The current system has outlived it’s usefulness.

  13. Bubblecar says

    “…this imbalance can dramatically bias the campus social and educational agendas in favor of progressive viewpoints.”

    It’s all a little confusing, since traditionally, committed conservatives are expected to be hostile to new knowledge and new ideas, which the universities are ostensibly there to explore. So one would expect universities to be inherently progressive in an intellectual sense and one would expect conservatives to be the wrong people to put in charge of them.

    In reality, much of the criticism of the “progressives” is that they’re not doing a progressive job at all – most are in fact conservative defenders of increasingly static and dogmatic ideology.

    So we have self-identified conservatives criticising self-identified progressives for being increasingly conservative.

    More progress is likely to be made as the number of dissenting (and genuine) progressives grows, and they complain more widely and loudly about shoddy and dogmatic academic standards and the increasing stifling of progressive debate.

  14. Alan D White says

    “progressive” as in progressing to a higher state of political being such as exists in Venezuela at the present time.

  15. Jezza says

    I am never going to publish in an academic journal of any kind ever. Instead I shall communicate my thoughts and insights to my friends and others of similar taste and discernment. Peer review is instantaneous, forthright, and sometimes raucous. It prevents the uncomfortable position of being up one’s self. It is also cheaper.

  16. James says

    “academia today resembles “a priesthood or a guild” or even a “cult“,…”

    I quit a PhD program because I saw that truth, and didn’t want to be what we grad students jokingly called a “Priest of Knowledge”. That was 30 years ago.

  17. Aaron Goldenberg says

    As an economist, I can say that articles in Research.One on economics are total nonsense.

Comments are closed.