• 38 Posts
  • 1.19K Comments
Joined 2 years ago
cake
Cake day: June 27th, 2023

help-circle





  • At least this example grew out of actual humans being suspicious.

    Dozens of academics have raised concerns on social media about manuscripts and peer reviews submitted to the organizers of next year’s International Conference on Learning Representations (ICLR), an annual gathering of specialists in machine learning. Among other things, they flagged hallucinated citations and suspiciously long and vague feedback on their work.

    Graham Neubig, an AI researcher at Carnegie Mellon University in Pittsburgh, Pennsylvania, was one of those who received peer reviews that seemed to have been produced using large language models (LLMs). The reports, he says, were “very verbose with lots of bullet points” and requested analyses that were not “the standard statistical analyses that reviewers ask for in typical AI or machine-learning papers.”

    We seem to be in a situation where everybody knows that the review process has broken down, but the “studies” that show it are criti-hype.

    Welcome to the abyss. It sucks here (academic edition).







  • Scientific Reports did not have what one would call a sterling reputation prior to this. Mathematical physicist John Baez wrote,

    If you’re a physics crackpot who wants to publish in a prestigious-sounding journal, I recommend Nature Scientific Reports! You have a good chance of getting your paper in!

    Try making it look like “Mass–Energy Equivalence Extension onto a Superfluid Quantum Vacuum”. […] This paper looks like a lot of the emails I get. It would never be published in a serious physics journal:

    https://www.nature.com/articles/s41598-019-48018-2

    And it’s not the only crackpot physics paper that’s been published by Nature Scientific Reports!

    Here’s a much crazier paper in Nature Scientific Reports:

    https://nature.com/articles/s41598-019-46765-w

    It’s called Maximum Entropy (Most Likely) Double Helical and Double Logarithmic Spiral Trajectories in Space-Time. You have to read it!

    My guess is that Nature Scientific Reports doesn’t have mechanisms built in to enforce the oppressive hidebound orthodoxy that dominates the other physics journals. So if you have a revolutionary new theory, submit your paper here!!!

    Flavio Nogueira in the comments:

    I have been in a meeting with the editors of SRs and its editor in chief years ago during an APS March Meeting. I can tell you that some editors were truly pissed off, as papers rejected after peer reviewing ended up being published anyway…