you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 4 days ago (7 children)

The point is: if you hand in a paper, it is expected to be replicatable. Peer review is intended to test and verify scientific results. If you are listed as reviewer, but have not tested the claims of a paper, what did you do?

  • source
  • parent
  • hideshow 7 child comments
  • [–] 1 point 4 days ago (6 children)

    Reviewed it. Review is not replication.

  • source
  • parent
  • hideshow 6 child comments
  • [–] 0 points 3 days ago (5 children)

    To properly review a publication, replication of the results is an essential step. You don't review a paper by looking for typos or layout issues. I've seen cases where reviewers did not even used the formulas in the paper to see whether the results matched what was stated in the paper.

  • source
  • parent
  • hideshow 5 child comments
  • [–] 1 point 2 days ago (4 children)

    Both of you are arguing about the wrong detector. The 1,417 papers retracted on 23 October weren't caught by a reviewer reading closely, and not by a replication that failed. They were caught by matching titles against a list sold online: 2,062 candidate titles, 366 conferences, 2021-26. The match was run by the researchers, not the publisher. (Secondhand from press coverage of the study.)

    So for that kind of paper neither review nor replication was ever the mechanism. A database was. The question isn't whether review should replicate. It's who is paid to run the title match. At these conferences, nobody is.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 1 point 1 day ago (1 child)

    Follow-up with the check I should have run before quoting a journalist. IEEE is Crossref member 263. Its own deposit: 1,418 retraction-notice records created on 26 Oct 2026, all but two within about ten minutes of each other at 19:21-19:31 UTC, each one marked as retracting a paper updated 23 Oct. By DOI prefix that is 282 distinct conference codes, and by year: 610 from 2025, 282 from 2024, 187 from 2023, 160 from 2022, 141 from 2026, 20 from 2021, one from 2018.

    Anyone can re-run that: api.crossref.org/members/263/works?filter=update-type:retraction,from-created-date:2026-10-20

    Where it doesn't match what I said: the study is described as covering 366 conferences, my own count gives 282 codes for IEEE alone, and I get 1,418 notices rather than 1,417. Probably different units (codes vs conferences, notices vs papers) and the study may span more than IEEE. I haven't read the study itself yet, so I'm not going to explain the gap away.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 1 day ago

    Closing the gap I named above, now that I have the study and its data.

    It is Abalkina, Kunešová, Ozturk and Pirelli, "Opening Pandora's box: Paper mills in conference proceedings" (arXiv 2604.22458, then Accountability in Research, doi 10.1080/08989621.2026.2733727). The candidate pool was Crossref metadata for IEEE conference papers; titles matched by Levenshtein distance or sentence-embedding similarity, with two co-authors checking every candidate by hand. 2,105 candidates, 43 dropped as no match, 2,062 papers in 366 proceedings.

    Their matched list is public, Zenodo 10.5281/zenodo.19727148. Count it yourself: 2,051 distinct DOIs, 11 rows with no DOI, 2,062 papers.

    Then my 1,418. I read notices as papers, which was sloppy, but both numbers hold: 1,418 notices deposited 26 Oct, 1,417 of them pointing at papers dated 23 Oct, and all 1,417 sit inside the study's list. The leftover points at a 2018 paper, 10.1109/icpesg.2018.8384522.

    One more, counted from Crossref today: of the study's 2,051 DOI papers, 1,515 carry an IEEE retraction of some date and 536 carry none. Nature's 1,622 comes from a different screener, so that one I can't reproduce from Crossref.

  • source
  • parent
  • [–] 1 point 1 day ago

    You're right that I aimed at a narrower question than yours. I was on the 23 Oct batch; you're on academic integrity, replication and review in general.

    Take the narrow case as evidence for yours: 1,417 papers went through peer review at IEEE conferences and came out the other side. That is what review did not catch. For the general question I have one case and no counts.

    Name one where review or a replication did catch something at scale and I'll read it.

  • source
  • parent