September 17, 2005 Chicago, Illinois, USA
This case discussion was held as part of the WAME meeting that took place during the Fifth International Congress on Peer Review and Biomedical Publication. The WAME meeting began with Case Discussions of ethical issues, chaired by Michael Callaham, Chair of the WAME Ethics Committee and Vice President of WAME, and discussed by Fiona Godlee, Editor of BMJ, Chair of Council on Publication Ethics, UK (COPE), and Past President of WAME; Mary Scheetz, Director, Office of Research Integrity (ORI), US Dept.of Health and Human Services; and Lorraine Ferris, Judicial and Policy advisor, University of Toronto and member of the WAME Ethics Committee. More than 150 individuals attended the Ethics Cases discussion and 3 cases were discussed with extensive audience participation.
Case 1: We rejected a manuscript after peer review. Although we had received 2 reports that were quite positive, asking for minor corrections before publication, the third report was very negative, arguing that the authors hadn't provided enough documentation of scientific results to demonstrate their point. None of the 3 reviewers declared any competing interests.
A few days after the author was informed of our decision to reject the paper, the corresponding author wrote back to us that the referee who had written the negative report had in fact failed to mention an important conflict of interest. The rejected paper presented a new method to pre-select the sex of offspring, and the referee was the holder of a patent on the method that's commonly used. Of course, one can see the economic impact the publication of this paper might have had on the referee's royalties, etc. We didn't contact the referee, but told the author that we would be willing to consider a resubmission of the manuscript.
BMJ was the journal involved in this case, and Fiona Godlee discussed that the bias in this case was apparent only because of BMJ's open peer review process, in which authors know the identity of the peer reviewers. The BMJ also provides clear instructions to reviewers regarding divulging conflicts of interest (referred to as competing interests by BMJ). The reviewer had indicated that (s)he had no conflicts of interest.
Panelists pointed out that WAME and other organizations recommend very clear and explicit statements be required of reviewers on each review regarding their possible conflict of interest. No investigator believes they have a conflict of interest (or that it has influenced their behavior), so they need a sample list of possible conflict of interests so they can state whether any exist in their case. A simple statement of "do you have a conflict of interest" is often not enough, and allows evasion. Also, it is known that this sort of bias, like many others, is often not a conscious decision, and thus that much harder for reviewers to detect by themselves.
Lorraine Ferris stated that editors should contact both the author and the reviewer in a neutral, nonconfrontational way to gather facts before reaching a conclusion as to whether the reviewer had a conflict. If the editor determined that a conflict did exist, the journal could further investigate or, more likely, the journal would refer the case to the reviewer's institution. WAME's publication ethics policy recommends a series of graded steps in response to allegations of misconduct, starting with a brief investigation to determine that there is some possible merit to the charges. The severity of the response, and to whom it is directed, can be based on the nature of the misconduct and the initial responses of the alleged violator.
Mary Scheetz said that the case would not be handled by ORI, since handling allegations of bias would open the floodgates for ORI cases. ORI also does not address authorship disputes. She did state that it is very important for journals to have clear instructions to reviewers as to disclosing conflicts of interest. Policies of both WAME and COPE state that journals should make explicit to reviewers (as well as authors) their policies about conflicts of interest. Members of the audience pointed out how difficult it is to collect conflict of interest information from reviewers, and that large sums (such as legal fees testifying as experts) can be involved and never disclosed. The panelists pointed out that there is no possibility of eliminating conflict of interest in human affairs, and that is not the goal. The goal is to produce transparency so that readers have as much information as possible to make their own assessments.
Extensive discussion brought out the following points. If 2 reviews were positive and 1 was negative, why was the manuscript rejected? Fiona indicated that the 2 positive reviews were short and not especially helpful, whereas the negative review was extensive. Some commented that an additional review should have been obtained. Others suggested that even if the reviewer has a conflict, the reviewer may be correct in the assessment of the study. Still others pointed out that the final decision is the editor's, regardless of what the reviewers say and recommend.
Some editors from smaller countries and smaller fields pointed out that their communities are smaller, and they would handle this situation via personal communication with the author and/or reviewer. (This normally would be the recommended initial step in larger countries as well). Smaller countries generally do not have organizations to turn to to aid in investigations. In some smaller fields (the example given was asthma research), it is difficult to find reviewers without conflicts, but those in the field generally know what each others' conflicts are.
There was also some discussion about what should be done if the paper were later accepted and published; should readers be told of the conflict of interest and behavior of the reviewer? Some discussants recommended publishing an announcement or editorial describing the events. Audience members described the difficulty and reluctance of many universities to investigate such matters or take action when they are informed of potential ethical lapses or misconduct. Lorraine Ferris said that even if the editor does not see direct evidence of action on the part of the institution, it can be very helpful for an institution that is compiling information about a given researcher to learn of potential scientific misconduct or ethical violations, as this information may lead to the investigator receiving additional supervision, review, or sanctions, which the editor may never become aware of.
An alternate perspective on the issue was this: what if the reviewer had a bias (in favor of the article due to self-interest) and had recommended acceptance? Would the bias have come to light? Participants agreed it would have been unlikely to do so, and Fiona Godlee pointed out again that this was a strong argument for open peer review.
Case 2: An editor in the audience presented the following case: This journal posts articles online before print; the print journal comes out several weeks after the articles are posted online. Several days after an article was posted online, the editor received an e-mail from the fourth author stating that informed consent had not been obtained for the study. At that point the editor was faced with the decision as to whether to go ahead and publish the study in print or to delay publication until an investigation could be completed.
Fortunately the journal had contacts in the country of origin of this research, and thus was able to contact the first author (and later the fourth author), who produced proof that the study had included informed consent. The e-mail address of the "whistleblower" actually was not that of the fourth author, but apparently was a spoofed address from a common e-mail service provider (eg, aol.com). The authors thought that the e-mail had come from a competing researcher, but the identity of the individual could not be determined. The editor asked whether the journal should have gone ahead and published the article, particularly if the investigation had not been so quickly informative.
In the ensuing discussion, Fiona Godlee and several members of the audience stated that posting the article online is publication, and the journal proceeded exactly as they should have up to that point, conducting an investigation to determine the seriousness of the allegations. Had the investigation taken much longer and the editors been unsure of the seriousness of the charges, they should have left the online publication up (since it had already been published) but could delay the article in print until they were more sure of the facts. Others suggested that if the investigation continued well after online publication and the editor had significant concerns, the journal could publish an "expression of concern" about the article. Other journals, eg, New England Journal of Medicine, have done this; when the investigation was completed and if no wrongdoing was discovered, they removed the expression of the concern and contacted PubMed who also removed the PubMed listing of the expression of concern. Some expressed concerns about the legal liability of the journal in making such a statement, but neither Lorraine Ferris nor any who volunteered their opinions knew of journals being sued over revealing ongoing investigations. All agreed that there were potential legal ramifications of making public allegations about ethical issues related to specific articles and authors. Lorraine Ferris also emphasized that although legal ramifications are important, these are ethical issues as well and we should fulfil our ethical obligations, in which case we will probably fare well in the court of public opinion.
Case 3: An audience member asked what the process would be if an allegation of scientific misconduct were made years after the paper was published. If the editor found enough evidence to pursue the allegation, in the United States the case should be turned over to the ORI for investigation. One editor said that his journal attempted an investigation several years after publication, and found that the "trail had gone cold": the author in question was a graduate student at the time, moved out of the country, and could not be found to pursue the allegation. In this case, the journal could consider publishing an expression of concern while the investigation was ongoing. Richard Horton proposed in the Lancet the concept of "withdrawal of aegis," when rather than retracting a paper the journal states that it no longer stands behind the results of a paper. This would be another option, depending on the strength of the evidence. In Canada, cases may be handled by the , but only if government funding is involved.
Mary Scheetz explained how the ORI handles cases referred to them. If the ORI finds no evidence of scientific misconduct, the editor will not hear from the ORI; the author's institution may or may not contact the editor. If the ORI finds evidence of scientific misconduct, the investigation will include investigating all related articles. The ORI will contact the editors of journals that have published all related articles with a recommendation for how the journal should proceed, eg, withdrawing the paper. A recent case was referred to the Department of Justice when the author sued the institution for alleging misconduct; this suit escalated the investigation and ultimately the author was found guilty of scientific misconduct.
Finally, the audience and panelists discussed whether journals actually retract papers after the ORI has found evidence of scientific misconduct. According to Mary Scheetz, the settlement the ORI reaches with authors now is likely to include the actual language of the retraction and explanation, since in the past some authors have used language that intentionally obscures whether any misconduct was involved. To the knowledge of those on the panel, no journals have been sued over retraction of papers. Some believed that those in smaller countries would be unlikely to pursue allegations of scientific misconduct and retract papers because the medical community is so small and editors may not want to make enemies. The withdrawal of aegis was again discussed, and Lorraine said that the journal has an ethical obligation to make the facts and/or concerns known.