The Journey From Lab to Journal: the Peer Review Process

By Dr. Liji Thomas, MDReviewed by Danielle Ellis, B.Sc.

Any scientist with something worthwhile to share with the scientific community wants to publish. However, any published article in a creditable journal has to run the gauntlet of the peer-review process.

Introduction

Peer review refers to the process of evaluating the quality of a submitted manuscript prior to publication by other scientists in the same field. It is said to have been mentioned first by Syrian physician Ishaq bin Ali al-Rahwi, in his work titled Ethics of the Physician. He referred to the need to make notes of the condition of the patient at each visit and to submit such notes to review by a local medical body to decide if the care given was appropriate by the standards for physicians at the time. This work was from a man living between 854-931 CE.

Once the printing press came into being, publications proliferated with amazing rapidity. This made the regulation of published matters a necessity, and it was done mostly by peers. The English journal Philosophical Transactions of the Royal Society was the first to detail the peer review process back in 1665 formally. At this time, the main aim was to select papers for publication.

Soon afterward, the aim switched to ensuring that the study was accurate and worthy of publication. The Royal Society of Edinburgh described their peer review process in 1731, “Memoirs sent by correspondence are distributed according to the subject matter to those members who are most versed in these matters.” And the Royal Society of London set up their Committee on Papers to examine manuscripts submitted for their Philosophical Transactions.

In today’s world, peer review has become a systematic and standardized aspect of journal publication. The review is carried out by scientists who are doing research in the same or related areas but working independently of the authors of the index study.

Presubmission

Peer review follows an initial assessment of whether the submitted manuscript fulfills the criteria for the journal involved. The subject dealt with by the manuscript must fit the aim of the journal and the platform of the editors. Many manuscripts are rejected at this stage.

Once the editor determines that the journal criteria have been met, expert peer reviewers are chosen depending on their ability to do a fair, effective, and timely review. The editor must rule out conflicts of interest.

The Process

The peer reviewers, or referees, are asked to scrutinize the manuscript and decide if it meets the high standards of their field of research. The aspects that come under assessment include the originality of the research, the soundness of the experimental approach, and the ethicality and appropriateness of the techniques used.

In addition, the validity and significance of the paper are evaluated – will the paper move the field forward in some way? Other aspects noted by the reviewers may include missing or inaccurate references.

Title

The process begins with the title. The title must be clear, concisely describing the concept, the variable altered, and the systems used in the study. Most readers skim titles when looking for papers relevant to their needs, indicating their importance.

Abstract

The abstract is equally relevant for the peer reviewer, as it states the purpose, methods, the most important results, and conclusions of the study. The introduction, describing the paper’s scope and purpose in detail, with brief notes on the methods used, is scanned for its informativeness. The hypothesis and predictions are determined to be clear or otherwise.

Methods

The methods section of the study must be vetted for adequate detail as to what experiment was performed and when along with the equipment used. These should be appropriate to the study. The peer reviewer will otherwise suggest details that need to be added.

Results

The results section describes the experimental outcomes and data trends fairly and accurately. The reviewers will check the raw data to make sure the conclusions match the data, scan that all supporting tables and figures are relevant and necessary, and are captioned properly.

Discussion

The final discussion section provides the interpretation and links it to past studies. The reviewers look at whether the meaning of the results is properly deciphered in terms of the original research question, whether the discussion is focused, and whether the study limitations and strengths are clearly mentioned. This should also include the future applications and theoretical importance of the experimental work.

If a manuscript offers data that can be used for research outside the original area, it is still very useful even though the starting hypothesis may not have been answered specifically. The novelty of the idea, the validity of the conclusions, and the accuracy of the data may be more important in these cases.

The references list all sources of information used by the authors in any part of their work. The reviewer ensures that all citations are correct, appropriate, and properly formatted and suggests missing or relevant citations.

Keeping all this in mind, the paper is judged to be worthy of publication or not. If the reviewers think that in terms of content, writing, and logical appeal, the manuscript is among the top quarter of papers in the same field, they will recommend its publication.

Guidelines for Reviews

There are guidelines for peer reviewers to write their reviews, such as those published by the American Physiological Society. These are intended to make sure that they are easily read and understood.

The review structure typically starts with an overview of the suggestions provided by the reviewer.

It may then progress through the structure of the paper, the data sources, the methods of investigation, the flow of the reasoning, and the validity of the conclusions. Finally, the review may address problems of style and language.

Reviewers are expected to do this task from the standpoint of both the editor and the authors to ensure a timely, professional, constructive, and reasoned review. Good reviews can help students and other researchers to learn how to peer review themselves. Manuscripts must be kept strictly confidential, even if the reviewer thinks a colleague could add value to the review.

However, permission may be obtained for such contributions, and the colleague must be credited as required. Confidentiality is key for all who participate in the process throughout and extends to destroying the document after the review is complete, without saving any document electronically.  

Overall, the reviewers comment on whether the paper should be accepted, rejected, or revised before publication. The editor regulates the process, making sure that the reviewers make reasonable requests for revision, deciding on the relative importance of various review comments, and setting aside those requirements that are not necessary within the area of the study.

Journals typically have a large number of reviewers on their panel so that reviewers are not overloaded and can return the paper in a timely fashion and allow editors to pick the most suitable reviewer for a given manuscript. Typically, reviewers take about six hours to review one paper, and most reviewers say they are happy to review papers occasionally.

Motivation of Peer Reviewers

Many peer reviewers review papers in order to keep themselves abreast of current work and in recognition of the fact that others review their work as well. In addition, reviewers are often the friends of the editor who requests their review. Keeping track of new experiments can spark new ideas for the reviewers themselves.

Some eminent reviewers may later become part of the journal for which they review, or may use their reviewing as part of their resume or to accelerate their careers. Peer reviewing may demonstrate a deep familiarity with the subject and a dedication to advancing in the field, which is often useful when trying for promotions.

Types of Peer Review

There are four types of peer review: single-blind, double-blind, open peer, and transparent peer review. These differ in the degree to which the author and reviewers reveal their identity.

Single-Blind and Double-Blind

In single-blind peer review, the authors do not know who reviewed their manuscript unless the reviewers sign their reports, but the reviewers do know whose paper they are assessing. In the double-blind type, neither authors nor reviewers know each other’s identities. This can be useful when the paper is from an unknown author or another country, for instance, since it avoids possible bias.

However, distinctive variations in the style, subject matter, and citations often make it easy to tell the country and even the researcher in many cases. The value of concealing author identity is thus controversial.

Open and Transparent  

With open peer review, both authors and reviewers are known, and the reviewers’ reports are published along with the accepted manuscript, as are the authors’ responses. With transparent review, the authors do not know who reviewed their manuscript unless the reviewers sign their report. Still, as above, both the reports and the authors’ responses are published with the manuscript if accepted.

Open and transparent peer review processes prevent reviewers from inserting harmful or careless comments and encourage timely reviews. In addition, plagiarism is discouraged. An open review process can also discourage honesty, however.

Transparent review is most advantageous in that it allows reviewers to offer frank criticism, especially when the author is someone already established in the field, without fear of it impacting their relationship with the author.

A major drawback with concealing the identity of the reviewer is that the manuscript may be delayed deliberately to allow reviewers to publish their work first if it competes with the work under review.

Common Errors to be Spotted

Some mistakes that often occur in scientific papers include illogical or contradictory statements, hasty conclusions, falsely attributing causality, overlooking confounding factors, leaving out key experimental details, and using inadequately defined terms.

What Happens Next?

Once the manuscript is assessed, the journal editor evaluates the comments made by the reviewers and suggests changes be made in accordance with them. Following these revisions made by the authors, the manuscript may be accepted for publication. The editorial team will then carry out formatting and typographical editing before it is published in the journal.

Alternatively, the reviewers’ reports may lead to the manuscript being rejected or transferred to another journal.

The Need for Peer Review

Peer review is considered to validate the manuscript since the reviewers are experts in the same field. Gaps in explanation or experimental work, clumsy writing or a lack of clarity, and the general relevance of the paper, may all be pointed out by the reviewers in an attempt to make it more useful to others.

When the authors know their work will need to be vetted by others skilled in the same field, the manuscript will be of a higher standard. Moreover, there is a kind of censorship applied at this point to make sure that the ideas expressed in the manuscript are sufficiently backed by science before being released to the world at large.

That is, the reviewers examine the relevance of the research question, as well as the quality and accuracy of the experimental data and the conclusions drawn from it. Thirdly, they determine that the authors’ conclusions are not subject to personal bias or interpretations.

Finally, reviewers can help authors improve their work by avoiding ruthless attacks and pointing out correctable deficiencies instead. The guidance should not be primarily about the writing style or typos but instead about the quality and accuracy of the research or the validity of the conclusions.

The reviewed manuscript can become sounder, easier on the reader, and more complete and relevant for the reader.

Since editors will usually not insist on making changes that would necessitate a complete redoing of the whole experiment, even if suggested by the researchers, the authors should use this guidance to improve their work via revision before resubmitting their work to the same or another journal.

The Drawbacks of Peer Review

Even though most journals have accepted peer review as a necessity, this process has faced extensive criticism on a number of grounds. For one, it is extremely slow, with the manuscript being in the hands of the chosen peer reviewers for weeks or months, during which the findings may lose their relevance.

Secondly, the process leaves every new manuscript extremely vulnerable to the personal opinions and biases of the journal editors as well as reviewers, especially since scientists are often as or more resistant to challenges to their strongly held convictions as other people are. For example, conservative scientists may suppress new ways of experimental verification.

Peer reviewers often fail to detect plagiarism.

Even more serious is the verified fact that peer review is rarely thorough enough to rule out the publication of errors. Several researchers have proved this to be the case, with over a hundred nonsense papers churned out by a single computer program accepted by prestigious journals and institutes.

The number of qualified reviewers is simply far short of that required to review the million or more papers that are published each year. This means that many papers have not been reviewed adequately, and scientists are being exposed to bad science as a result.

Peer review is a flawed process, full of easily identified defects with little evidence that it works. Nevertheless, it is likely to remain central to science and journals because there is no obvious alternative, and scientists and editors have a continuing belief in peer review. How odd that science should be rooted in belief.”

Many researchers in this area have suggested or designed potentially better systems for peer review, which could lead to shorter turnarounds but improved results, with less time being invested in repeated rounds of peer review and more into actual scientific research. At present, post-publication peer review is being tried out, as well as dynamic peer review. This allows a broader base of discussion and evaluation, avoids plagiarism, speeds up publishing time, and allows the public to see the paper and the comments.

Other newer approaches include the transferable peer review, with journals dealing with the same branch transferring reviewed manuscripts between themselves, thus avoiding further review and the need to reformat the paper, and collaborative reviews, with different people using their varying specialized skills to review the same work. Many scientists also support results-masked peer review to avoid publication bias, leaving the results, impact, and relevance of the paper to be evaluated after publication.

References

• Kelly, J. et al. (2014). Peer Review in Scientific Publications: Benefits, Critiques, & A Survival Guide. EJIFCC. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4975196/.
• Smith, R. (2006). Peer Review: A Flawed Process at The Heart of Science and Journals. Journal of the Royal Society of Medicine. https://dx.doi.org/10.1258%2Fjrsm.99.4.178. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1420798/
• Kumar, M. (2009). “A Review of the Review Process: Manuscript Peer-review in Biomedical Research.” Biology and Medicine. A Review of the Review Process: Manuscript Peer-review in Biomedical Research
• An Overview into Peer Review Process. (2019). Retrieved from: https://www.enago.com/academy/an-overview-into-peer-review/. Accessed on March 1, 2022.
• Peer Review Process. (2022). Retrieved from: https://www.biomedcentral.com/getpublished/peer-review-process#:~:text=Peer%20review%20is%20the%20system,be%20published%20in%20their%20journal. Accessed on March 1, 2022.
• Understanding Peer Review: A Guide for Authors. (2022). Retrieved from: https://authorservices.taylorandfrancis.com/publishing-your-research/peer-review/. Accessed on March 1, 2022.
• Hames, I. Peer Review in A Rapidly Evolving Publishing Landscape. Eds. Campbell, R. et al. Academic and Professional Publishing, Chandos Publishing, 2012, pp. 15-52. https://doi.org/10.1016/B978-1-84334-669-2.50002-0. https://www.sciencedirect.com/science/article/pii/B9781843346692500020.
• What is Peer Review? (2022). Retrieved from: https://www.elsevier.com/reviewers/what-is-peer-review. Accessed on March 1, 2022.