Help

Help! I’ve Been Asked to be a Peer Reviewer

Guidelines on how to critique a paper, manuscript, or book

By Daryl Boucher, MS, RN, CCEMT-P, Allied Health Coordinator, Northern Maine Community College

Would you trust the medical advice given in the Globe? What differentiates Ladies Home Journal or People from JEMS, Nursing 2006, or JAMA?

The latter (JEMS, Nursing, JAMA) are peer reviewed journals. The articles and content in these journals have been reviewed by experts in the field to assure that the work to be published is accurate, reliable, and appropriately written. Peer review lies at the core of medical, nursing and healthcare education. Editors’ decisions about whether or not to publish a work are largely based on the judgments of the reviewers; authors and editors rely heavily on field experts to provide feedback on the quality of the work. According to Snell & Spencer (2005), a strong peer review system has come to be seen as a marker of a quality work.

Oftentimes, instructors or EMS leaders are asked by an editor to review an article. Typically, editors choose reviewers because of expertise in a given area. They look for reviewers who are able to balance positive and critical comments, and provide advice before the work goes to print. Being invited to be a reviewer is an honor, but unfortunately, too few professionals participate in the process, oftentimes because they are unaware of how it works. The purpose of this article is to provide a brief overview of the peer review process, and to promote peer review as an excellent method of professional development.

The Review Process

Editors recognize that a quality peer review will contribute to a better quality work. The more feedback they receive before a text goes to print, the better the text will be, and the more copies they will sell. Reviewers can be identified in several ways. First, those who have authored a book, presented at a national conference, or have been identified as content experts may be known to the editor. (I have met many editors and publishers at conferences, and have provided my contact information to them. Subsequently, I received calls or e-mails asking if I would be interested in reviewing certain articles or even writing a chapter or article). Publishers often provide their editorial departments with instructor lists. So, if you request a desk copy of a book because you are teaching that particular course, you may be asked to review a subsequent edition of that text. Most peer reviewed journals have a mechanism to identify potential authors or reviewers, or they actually have an appointed review board that is involved in critiquing all of the articles. Typically, this small print information can be found on the first few pages of the journal, or on their web sites. Finally, most of the review work I have received has come from the many online listserves I belong to. A publisher or editor such as Brady or F.A. Davis will put out a general request looking for reviewers in certain content areas. Once you review for a publisher, you become part of their reviewer database, and you are contacted at regular intervals to see if you might be interested in reviewing another work. Sometimes after several reviews, you may even be asked to author a work!

Why review? The Perks

There are several reasons to become a peer reviewer. First, reviewers get to see and contribute to research articles before they go to print. This assures that they are always aware of new developments in the world of EMS and healthcare. The research shows that most reviewers do it to “stay up to date” and for the educational experience (Snell & Spencer, 2005). Others have identified that they viewed peer reviewing as an academic responsibility, or that they found it to be a useful process as they were preparing to publish a work. For me, reviewing is simply enjoyable. It allows me to look at articles or books before they are commercialized. I also enjoy getting my name published in the front of the textbook as a reviewer, and it adds credibility to my role as an educator. I should also note that most high quality publishers provide honorarium pay for reviewing submissions. Though I have done many “free” reviews, I generally get paid anywhere from $50.00 to $500.00 per review, depending on the size and complexity of the work.

What and how do I review?

It is much easier to provide critique on a topic area that you know very well. However, sometimes, authors are looking for feedback on a topic where you are not a content expert. This is especially true with textbook reviews, where it is expected that the target audience will be students who are unfamiliar with the content. All publishers with whom I have worked have provided a standard questionnaire that the reviewer must complete, along with a reviewer profile. (See table 1 for a sample reviewer’s questionnaire). The purpose of the reviewer profile is to assist with marketing, and to assure the reviewer meets the needed qualifications.

Typically, the review is double blinded. You will not know who wrote the article, and the author will not know who reviewed it. This assures that the feedback provided is unbiased, fair and honest.

The editors usually are looking for the reviewer to provide professional opinions about both the presentation of the article, as well as content, accuracy, and relevance. Because authors have to submit proposals to publishers before actually submitting the work, sometimes the review involves simply evaluating a product to see if it is worth pursuing. Works that are nearing completion need to be reviewed for format or presentation. Editors have several expectations of reviewers. Reviewers must balance positive and negative comments. Comments provided must be tactful. It is unlikely that a reviewer would be used again if all of the feedback he provided was negative, or if he only made general, positive comments.

Reviews must be done quickly, in order to meet submission deadlines. Reviewers must provide useful comments that aid in improving the work. I have found that reviewing a published work is a lot like providing feedback to EMS students. Rather than just telling students their technique was poor while observing a skill, I tell them how to improve it. Editors and authors appreciate the feedback.

Reviewer qualifications

There is no specific training to become a reviewer; however, in order to be effective, you must be able to communicate in writing effectively, succinctly, and clearly. Because most reviews are now submitted electronically, computer literacy is important. The ability to do literature searches is also important, as sometimes verification of content is required. For example, I recently reviewed a work where I questioned the author’s interpretation of specific research findings. I had to find the original article that the author cited to assure that his comments were accurate.

The skill that most editors are looking for the most, however, is that the reviewer be knowledgeable about the content. In medical journals and textbooks, it is critically important for the content to be correct, and editors with little or no medical knowledge rely on the reviewer to provide content expertise.

Finally, it is important for reviewers to choose their review work carefully. If one is asked to review a research article, but has little experience or knowledge about the research process or statistics, it will be difficult to provide a useful critique. In these situations, it is important to be assertive in the review process. Occasionally, I have agreed to review a work, and discovered, upon receiving the article, that I simply am not knowledgeable enough to provide useful feedback. In these cases I have immediately notified the publisher to alert them so that they can reassign the article to a more qualified reviewer.

Typically, the publishers will use two to three reviewers per paper, selected from a pool of several hundred reviewers. Because of this small number, it is important for each reviewer to provide effective feedback. I have found that once you establish credibility with one publisher, they will continue to utilizes your services.

Challenges

Manuscript reviewing is incredibly rewarding, especially when you see your name in print, knowing that you have contributed a small part to the expansion of knowledge in EMS. Conversely, there are times when it can be challenging. On occasion, I have encountered a paper or submission that was poorly written. It was difficult to provide succinct feedback in a non-hurtful way. It can also be stressful reviewing works when you don’t know the author. Imagine my surprise recently when I reviewed a submission, provided multiple negative comments, and then saw the article printed in a professional journal several weeks later – only to learn that it had been written by a close friend and colleague. The double blind approach prevents bias, which is a good thing. Had I known my friend authored it though, I might have been less critical in my comments. (I haven’t yet admitted to him that I provided feedback to his work, but it was cool to see that he had integrated some of my comments into his product!)

Table 1: Sample Questionnaire for Review of a Textbook Chapter

REVIEWER’S NAME:

DATE:

Courses/Subject Matter You Teach:

Area(s) of expertise:

# of Years You Have Taught:

School/Level of Program:

Do you currently work in a healthcare/nursing setting?

After reading the proposal/chapter/text, please comment on the following.

1. Is the authors’ approach appropriate for the intended audience? If not, why?

2. What are the typical problems that your students encounter in mastering this material? How have you successfully presented this material in the past?

3. Please comment on the organization of the table of contents. Do you have any suggestions for a different organization/emphasis?

4. What is the strongest feature of this proposal/product? What are the weakest features?

5. How could this product be improved?

6. Has any essential item been left out of the proposal’s content? Is there any material that should be eliminated?

7. In your opinion, what percentage of your students would buy this text?

8. Please provide your opinion on the charts and tables of the text. Is the reference list complete and comprehensive?

9. Describe the readability, language, and overall presentation.

10. Please provide any other thoughts you might have on this proposal.

Please comment on:

Coverage of the topic:

Accuracy of information:

Overall quality:

YOUR RECOMMENDATION:

Accept this proposal in its present form

Accept this proposal with minor revisions as indicated

Accept this proposal only with substantial revisions as indicated

Do not accept this proposal

References

Snell, L. & Spencer, J. (2005). Reviewers’ perceptions of the peer review process for a medical education journal. Medical Education 39, 90-97.