The method
How review works
Manuscripts submitted to the journal are examined by independent artificial-intelligence models under a fixed rule of decision. Where two reviewers disagree a third arbitrates, and the reports are published in full beside every article accepted.
The method in brief
Submission
The author sends the manuscript to the editorial office. There is no charge, and no limit of length.
Preliminary check
The manuscript is checked for completeness and for conformity with the journal’s requirements before it goes to review.
Two independent reviews
Two models, working separately, each read the whole manuscript and return a reasoned report with a recommendation: accept, minor revisions, or reject.
The rule of decision
If both recommend acceptance or minor revisions, the article is published. If both recommend rejection, it is declined. If they disagree, the matter goes to arbitration.
Arbitration
A third model reviews the manuscript afresh, and the majority of the three decides.
Publication with the reviews
An accepted article is set in the house layout and published in a quarterly issue, with every review it received.
The rule of decision
The outcome follows from the two primary recommendations by a rule fixed in advance, so that no manuscript is treated differently from another.
| First reviewer | Second reviewer | Outcome |
|---|---|---|
| Accept | Accept | Published |
| Accept | Minor revisions | Published |
| Minor revisions | Accept | Published |
| Minor revisions | Minor revisions | Published |
| Reject | Reject | Declined |
| Accept | Reject | Third review; the majority decides |
| Reject | Accept | Third review; the majority decides |
| Minor revisions | Reject | Third review; the majority decides |
| Reject | Minor revisions | Third review; the majority decides |
The Society calls this arrangement the Tri-Model Consensus. The primary reviews have ordinarily been given by Claude and ChatGPT, with Gemini as arbiter; the particular models have varied as the instruments themselves have advanced, and each published review names its reviewer.
What the reviewers examine
Every report addresses the same five matters: the rigour of the method and the validity of the statistics; the consistency of the argument; the accuracy and completeness of the citations; the clarity of the presentation; and the originality of the work and its contribution to its field.
Principles
The whole manuscript is read. Each reviewer reads the submission in its entirety.
The reviewer has no interest in the outcome. A model has no colleagues to favour, no rivals to obstruct and no institution to defer to.
One standard for all. The same criteria are applied to every submission, whatever its origin.
Promptness. Reviews are ordinarily complete within forty-eight hours, and the author receives the decision and the reviews within three working days.
Why the method was adopted
Conventional peer review is slow, and it is not always blind to matters that ought not to count. Studies of the practice have reported that the judgement of a manuscript may be swayed by the standing of its author or of the author’s institution. In one experiment, articles by authors from eminent departments, already published, were submitted again under invented names and institutions, and most were rejected by the very journals that had printed them.1 In another, reviewers who could see the authors’ names were more favourable to papers from famous authors and leading institutions than reviewers who could not.2 In a third, the same manuscript drew markedly fewer recommendations to reject when it bore the name of a Nobel laureate than when it bore that of a junior colleague.3
The founder’s own reasons, which arose from clinical work during the pandemic, are given in his statement.
Why the reviews are published
Most journals keep their reviews from the reader. The Society publishes them in full, so that anyone may examine how an article was judged, see the grounds on which it was accepted, find the limitations the reviewers identified, and form an independent opinion of the fairness and thoroughness of the process. A review that can be read can also be corrected.
The limits of the method
No method of review is infallible, and this one has limits of its own. Artificial reviewers may share blind spots; agreement between models is not proof of truth; and a model may be mistaken with great fluency. The Society does not ask that its method be taken on trust. It publishes the reviews so that error may be found by any reader, and it studies the reliability of the method as one of its own programmes of research: see Machine peer review.
Responsibility for the conduct of review rests with the Editor, to whom any published decision may be referred at richard@scottishsciencesociety.org.
Notes
- D. P. Peters and S. J. Ceci, ‘Peer-review practices of psychological journals: the fate of published articles, submitted again’, Behavioral and Brain Sciences, 5 (1982), 187–195. ↩
- A. Tomkins, M. Zhang and W. D. Heavlin, ‘Reviewer bias in single- versus double-blind peer review’, Proceedings of the National Academy of Sciences, 114 (2017), 12708–12713. ↩
- J. Huber, S. Inoua, R. Kerschbamer, C. König-Kersting, S. Palan and V. L. Smith, ‘Nobel and novice: author prominence affects peer review’, Proceedings of the National Academy of Sciences, 119 (2022), e2205779119. ↩