Dear Members,
I would be grateful for your comments and perspective. A member of my Editorial Board wrote to urge me to stop using blinding for reviews of papers (we use double-blind assessment). He wrote "Virtually all journals send out reviews unblinded."

Is this statement correct? I know it is true for journals in some scientific disciplines (e.g., geology), but is it true for the majority of medical journals? I strongly believe in the blind review process, even if referees can sometimes deduce who the authors are. I am unwilling to change to unblinded reviews, but I need to know if my position is "out-of-step" with respect to the real world of medical journals.

Thank you for your thoughts.

Diony Young
Editor, Birth: Issues in Perinatal Care

___________________________

In my experience with basic science journals in the biomedical field, most journals use single-blind assessment (the reviewers know who the author is, but the author can only guess who the reviewers are). This is how the NIH and NSF conduct external peer-review of grant proposals (although the NIH does provide a list of all the reviewers in the panel). Of course, I've seen journals with double-blind assessment as well as with total-disclosure assessment (more rarely).

Roberto Refinetti
Editor-in-Chief, Journal of Circadian Rhythms (www.jcircadianrhythms.com)
Managing Editor, Sexuality & Culture (www.csulb.edu/~asc/journal.html)

___________________________

It is true that the majority of journals do not blind the reviewers to the identity of authors. In the past 15 years a number of studies have looked at this practice (including several RCTs) and could not find any difference between the reviews produced by reviewers who were blinded, or not. However, often overlooked is that the only thing assessed in these studies was the quality of the review (that is, its technical quality), and no assessment was made of its fairness.

The major concern in reviewers knowing the authors (or their institutions) is not that they'll do a review that gets a lower quality score, but that they will be biased. There is no study that has tried to measure the "fairness" of blinded vs. unblinded reviewing. Reviewers (NIH or otherwise) would say they are not biased by this knowledge, but as we know, the person who might be biased is not the best judge of whether bias is present.

Our journal has blinded reviewers to authors for over 2 decades. Even if a randomized controlled trial showed that this had no impact on actual fairness, we would continue to do it, because it contributes to the author's perception of fairness. Like conflict of interest, perceived fairness is almost as important as fairness itself.

I encourage you to continue your practice.

Michael Callaham

___________________________

We use single-blind review, ie, authors' names known to reviewers. Even if double-blind review is used, it is often possible to 'guess' who the authors are. Especially if it is a continuation of their studies and they will use a sentence in the text such as "we have previously shown that...".

Rob Siebers
Editor, New Zealand Journal of Medical Laboratory Science

___________________________

Well said! Thank you. I have been trying to decide whether to change from blinding both reviewers and authors to blinding only the reviewers from the authors. The reason I have not done so is that our field (transplantation) is so small that I have great concerns about the ability to remain unbiased. Your comment about perception is very important to authors.

Thanks.

Linda Ohler
Editor, Progress in Transplantation

___________________________

I thought I would share my earlier note to Diony with all members, since it complements comments from others:

"I agree with Michael Callaham's recommendation for Birth to continue with its policy to mask manuscripts from reviewers. Not just because of perceptions, but because at present, it is the best way to ensure the fair review of most manuscripts. Even when reviewers identify the institution or the investigator group within an institution responsible for the conduct of a study, rarely do they guess the identity of the first author. (Of course, we must recognize that in some departments the head assigns authorship, sometimes long before a manuscript is submitted for publication, so the rightful author may not always be listed as the first.)

My experience with reviewers over more than a decade repeatedly confirmed that masked reviews can protect a manuscript from being evaluated in a biased manner. We did not conduct a study to evaluate fairness of double-blinded and single-blinded reviews; the evidence is anecdotal. Nevertheless, from the feedback we received from many reviewers after a manuscript's review process was completed, I believe that, in most instances, double-blind reviews improve the fairness of reviews."

Cristina I. Cann
Former Associate Editor, Epidemiology

___________________________

There seems to be an assumption in all of this discussion that anonymity is a virtue. Those arguing that reviewers should not know the identity of authors and vice versa seem to be assuming that this is a good thing. The talk has been about it being somehow "fairer".

Let me put the opposite view. We all understand that anonymous notes or e-mails are usually written from "coward's castle." They allow people to make comments and hide behind the cloak of anonymity. Most newspapers refuse to publish anonymous letters unless there are obvious concerns for legal or safety issues of authors. Judges do not hand down anonymous legal judgements. At university, students know who will mark their papers (although at my university students are forbidden to put their names on their assignments because of the argument that a piece of work should be judged on its own merits, rtaher than through potential biases about good and bad students).

The reason for lack of anonymity is that it is assumed that people should have the courage of their convictions and that anonymity makes it easier for gratuitous and poor quality criticism to be provided. Blinded reviews allow reviewers to hide behind a multitude of sins, not the least being that they sometimes reveal how sloppy or slapdash their own understanding of the subject is. Reviewers would seem to me less likely to sign poor quality reviews than those which would identity them, although there is not research support for this—see below. I found that result counter-intuitive, and would like to see further work on it. Requiring reviewers to sign their reviews may well lift their standard.

My sense is that the twin Achilles heels of peer reviewing are poor quality reviews, and the difficulty of obtaining reviewers. This lethal combination often results in authors being forwarded very superficial reviews, and the quality of revisions being less than it might otherwise be.

I always sign my reviews, have probably lost a couple of people as friends as a result, but firmly believe that the principle involved is important. Richard Smith, the former BMJ editor, wrote an editorial on this http://bmj.bmjjournals.com/cgi/content/full/318/7175/4 and I think it is still BMJ's practice to unblind all reviews.

There is an RCT on the topic at http://bmj.bmjjournals.com/cgi/content/full/318/7175/23, which concludes "Asking reviewers to consent to being identified to the author had no important effect on the quality of the review, the recommendation regarding publication, or the time taken to review, but it significantly increased the likelihood of reviewers declining to review. "

The outstanding issue that remains for me is the widespread perception that some reviewers fear providing criticism of people senior to them because they fear some sort of subtle retribution via things like promotion committees or subsequent "tit for tat" in reviewing. This is such a depressing consideration that is currently being sanctified & reinforced by anonymous reviewing.

Simon Chapman
Tobacco Control

___________________________

Is it possible that we can say there's a case for open reviewing and for double-blind reviewing but not for single-blind reviews?

M.E. Kerans
Barcelona, Spain

___________________________

Peer review has been studied in a couple of randomized, controlled trials and found to not effect the quality of reviews. See:

Justice AC, Cho MK, Winker MA, Berlin JA, Rennie D, and the PEER Investigators. Does masking author identity improve peer review quality? a randomized controlled trial. JAMA. 1998;280:240-242. Erratum in: JAMA 1998;280:968.

Godlee F, Gale CR, Martyn CN. Effect on the quality of peer review of blinding reviewers and asking them to sign their reports: a randomized controlled trial. JAMA. 1998;280:237-240.

Having said that, we still remove the authors from a manuscript before sending it to a reviewer, knowing that this likely has no effect on the quality of the review, to send the message that we care about providing fair reviews of manuscripts submitted to our journal.

I think it is important that the reviewer know who the authors are. Good reviewers will know if a manuscript is a rehash of prior published work (or will look it up on PubMed or some other database). There is always a chance that there will be a conflict of interest that the reviewer will not divulge, but let's be honest-it is impossible to truly blind an article to the reviewer anyway. The possibility that a review may be biased is the reason why we get more than one (we get 3) and the editor also reads the manuscript.

Bill Tierney

___________________________

I became concerned about reviewer bias, so I stripped names of authors and institutions from the version of the manuscript I sent to reviewers. Then I became concerned that the Associate Editors would be subject to the same bias, so removed the names of authors, their institutions, reviewers, and their institutions from the versions that went to the Associate Editors for adjucation. Then the Associate Editors became concerned about my bias, so we decided to ask authors to submit manuscripts anonymously, using an e-mail alias (hotmail.com was pleased). We then decided to publish worthy articles anonymously as well (Medline was not pleased) in order to prevent reader bias in assessing the results. To preserve total anonymity we removed our names from the masthead and decided to remove the names of the Editorial Board members as well (they didn't notice)...

You get the idea. This is not as fantastical as it sounds (cf. the path many IRBs have taken). If the purpose of the exercise is to eliminate bias, I would suggest that we are holding back the sea with a pitchfork. In its simplest nonstatistical meaning , bias refers to a systematic preference for something. We are all guilty, and if we weren't, it would be very hard to deal with the world. We bring a set of preferences, a template for examination, to the party, and I would suggest that that is as it should be. The real problem is when people fail to recognize bias in themselves (as Dr. Callaham has pointed out). I think all of us become good at spotting the irrational, pigheaded, unremediable commitment to an idea--we use our own biases to filter those of others. I say: don't pursue of futile exercise of eliminating bias, but embrace it. We ask reviewers to tell us what they think, and if thinking that a particular author or group doesn't do a very good job, that's part of the bargain (as per Dr Tierney's comments).

As an aside, I might mention that I think double blinding became popular many years ago not in an effort to reduce bias, but to reduce character assassination. Many reviewers felt they had the licence to release whatever venom they wanted to (and we still see some of that), and double blinding had a salutory effect. But that was 30 years ago, and this is now.

Rich Rothenberg

___________________________

In 1990 we published a paper describing an RCT of blinding at the Journal of General Internal Medicine (McNutt et al. JAMA. 1990;263:1371-1376). In addition to scientific quality of the review, both the editors and the submitting authors were asked to evaluate the review for constructiveness, fairness and courteousness. Neither editors nor authors found differences between blinded and unblinded reviews (although, in contrast to the collaborative larger and more recent studies, blinded editors graded blinded reviews as scientifically better than unblinded reviews).

Suzanne Fletcher
___________________________