Read this article where we sought to better know what the top peer reviewers around the world look for in their own reviews.
By: Jim Fischer, Marketing Manager, Wolters Kluwer Health
Rosalyn S. Yalow, winner of the 1977 Nobel Prize in Physiology or Medicine and the second woman to win the prestigious award, will forever be remembered as a co-developer of the radioimmunoassay technique—a discovery which revolutionized the field of medicine1. However, did you know the original research was rejected? Reviewers from The Journal of Clinical Investigation in 1955 found that the “data you present are indeed suggestive but any more positive claims seem unjustifiable at present.” 2 Dr. Yalow is not the only Nobel Prize winner this has happened to. As a matter of fact, it has happened seven other times in the sciences.
The topic of peer review is debatable and outlying examples such as this can justifiably force researchers to question if the process merits a necessary place in scientific publication. The system has its flaws, but remains a necessity. It is the method by which grants are allocated, papers published, academics promoted, and Nobel Prizes won3. Most important is the rudimentary role a strong review can have on for a researcher. A well-executed review serves a dual-purpose gate keeping role: first, it informs an editor’s decision about the disposition of a manuscript and second, it provides a crucial series of comments and suggestions that the author can take into account if he or she decides to revise the manuscript and resubmit for publication4-5. In the case of Dr. Yalow, she persevered. With further research and a discreet change to the paper's title, the report appeared in 1956. In later years, Yalow delighted in illustrating her lectures with a slide of the rejection letter6.
The work of peer reviewers can remain somewhat mysterious, particularly for the less experienced author. With this mind, we sought to better know what the top peer reviewers around the world look for in their own reviews. To accomplish this, we contacted our colleagues at Publons. A huge thank you are in order to Tom Culley, Head of Marketing, and Jo Wilkinson, Communications Manager, who were able to connect with two of their 2017 Publons Peer Review Award winners, Dr. John Cooper of Columbia University and Professor Maya Zumstein-Shaha of Bern University of Applied Sciences for this interview.
About the reviewers
Dr. John Cooper is a board-certified, fellowship-trained orthopaedic surgeon specializing in adult reconstructive orthopaedic surgery of the hip and knee. He is actively involved in academics, teaching orthopedic residents and fellows. Dr. Cooper has research interests in primary and revision hip and knee replacement surgery, focused on patient outcomes, anterior approach hip replacement surgery, and complications such as periprosthetic joint infection, periprosthetic fracture, and various failure modes of prosthetic devices.
Profressor Maya Zumstein-Shaha is a professor in nursing science at the Bern University of Applied Sciences, Department of Health. She is currently the deputy leader of the Master of Science in Nursing program at this institution. Her areas of expertise include new roles and service provider models in nursing such as advanced nursing practice, interprofessional collaboration, ethical issues, theory development and oncology nursing.
The questions
What does the Editor-in-Chief encourage peer reviewers to look for when reviewing a manuscript?
Dr. Cooper: “The editors typically ask us to briefly summarize the take-home points of the study and to give our opinion on whether or not the study is evaluating an important research question that is within the scope of the intended journal. Regarding methodology, we commonly assess whether we are comfortable evaluating the statistical methodology used in the paper, and whether there are any methodological issues with the study design. Further, we are often asked to evaluate how clearly the results have been presented and whether there are any limitations not already mentioned by the authors. Regarding the discussion, we look for any opportunities to improve the discussion to present the results within the context of what is already known. Finally, we try to evaluate whether the conclusions are fair or overreaching?”
Profressor Zumstein-Shaha: “I simply have been performing article reviews for the past 20 years. Hence, I can only state that the invitations seem to be based on the respective profile that I have with each journal. Sometimes, I receive invitations to review based on the requests of the authors themselves.”
Has increased scientific output from international authors impacted your approach to peer review?
Dr. Cooper: “Reviews are typically blinded so I usually don't know the country of origin of an individual paper. However, the volume of requests can be tremendous at times, and it would be impossible to accept all requests. When I receive a request for review, I think about the content and what value my input might add. It's also important for me to be honest with myself about how much time I have to devote to a specific review. If I don't believe I have sufficient time to do a thorough review and provide appropriate feedback, I'd rather quickly decline so the editors can find another suitable reviewer without delaying the review process.”
Profressor Zumstein-Shaha: “Not at all. Generally, my review invitations are blinded.”
What are some of the common reasons you’ve rejected a paper?
Dr. Cooper: “The four most common reasons are (1) concerns about the methodology used and the effect those methodological questions have on potential results, (2) incorrect or overreaching conclusions drawn from the results presented, (3) a poorly-edited manuscript with an abundance of typographical and grammatical errors throughout, and (4) a well-done study, but one that doesn't fit with the purpose or scope of the particular journal.”
Profressor Zumstein-Shaha: “Rejecting a manuscript outright is difficult and needs substantiation. When I do have to consider rejection, I try to be very concrete in my review. Often, flawed methodological approaches are conducive to rejection. In such cases, I try to point out the flaws and propose strategies for improvement.”
How do you try to assist authors in your feedback?
Dr. Cooper: “Whether or not the submission is a good fit for the journal, it's important to offer constructive feedback for the authors. I try to assist the authors in making sure they are presenting their results clearly, and in a manner that answers the research questions they have posed. Even in a paper that is rejected, there is often really good data which can have a clinical impact, but it may just not be presented well by the authors. Suggestions on reorganization, structure, and how to rephrase questions can help improve these tremendously. Additionally, a reviewer's fresh perspective to a paper can often help authors think about new ways to approach the data.”
Profressor Zumstein-Shaha: “I always start with a positive point about the manuscript. Subsequently, I try to point out as clearly as possible a potential area for improvement. I often propose strategies for obtaining improvements. Sometimes, I provide additional references that may be helpful.”
Is there any advice you can offer for newer authors who may be looking to submit a manuscript for peer review?
Dr. Cooper: “I would advise all authors to make sure their study fits within the stated scope of a particular journal before submitting a paper for review. Additionally, it's important to follow the detailed instructions for submission published by the journal, and to thoroughly check for spelling, grammatical, and content errors before submitting. Finally, don't take a negative review personally. The vast majority of reviewers, with rare exception, approach the peer review process with the intention to help improve the project with constructive criticism so that the study can achieve its maximum impact.”
Profressor Zumstein-Shaha: “Writing a manuscript can happen very speedily or may involve several rounds of re-writing. It has been my experience that re-writing is very helpful. As an author, one gets clearer about the message in the article and it is more likely that duplications are identified. I have also found it helpful to involve at least one good friend with a sound professional background in nursing (my area) as a consultant. This friend should look through the manuscript and point out passages that are potentially difficult to understand. In some cases, it has proven helpful to submit a manuscript to an existing journal writing group. Thus, several readers can provide feedback. Thus, it is more likely that a manuscript is coherent. Finally, it is essential to consider the author guidelines of the journal the manuscript is intended for.”
References
- Kahn, C. R., & Roth, J. (2012). Rosalyn Sussman Yalow (1921–2011). Proceedings of the National Academy of Sciences, 109(3), 669-670. doi:10.1073/pnas.1120470109
- MacDonald, F. (2016, August 19). Eight Scientific Papers That Were Rejected Before Going on to Win a Nobel Prize. Retrieved August 28, 2018, from https://www.sciencealert.com/these-8-papers-were-rejected-before-going-on-to-win-the-nobel-prize
- Smith, R. (2006). Peer review: A flawed process at the heart of science and journals. Journal of the Royal Society of Medicine, 99, 178-182.
- Milardo, R. M. (2015). Crafting scholarship in the behavioral and social sciences: Writing, reviewing, and editing. New York, NY: Routledge.
- Miller, S. A. (2014). Writing in Psychology. New York: Routledge.
- Roth, J. (2011). A tribute to Rosalyn S. Yalow. The Journal of Clinical Investigation. 121(8), 2949-2951. doi:10.1172/JCI59319.