As the editorial team we requested the raw data (database in SPSS or ...) from an author to check their analyses. The author refuses, and we have to present evidence that this is the right of the editors to request the data. I have seen such a policy somewhere in another journal, but now I don't know the source. In WAME, I did not find any sentence supporting our request. Could anybody help me find the evidence? Besides, is it really necessary to persuade the author?

---

Although I agree with the points made (ie, it is the *right* of the journal to request data and the *right* of the author to refuse), I cannot forsee any instance when I would request raw data to analyze. Kim keeps me busy enough reviewing data, let alone analyzing data. *I* think  this is a waste of your board's resources. Either the authors respond to criticisms or, if you doubt the veracity of the data, simply reject the submission. This also sets a bad precedent, when do you request data, when don't you analyze data? Journals are not in the data analysis business.

Gregory E. Gilbert

Editorial Board, Journal of the National Medical Association

---

I could not possibly disagree more. Allowing oneself the option to investigate further (by requesting raw data) is not tantamount to obligating oneself to do so. If authors know that they may be compelled (at least as a condition of publication) to provide raw data, then they may be more diligent than they currently need to be in ensuring that the summary data tabulations are supported by the raw data. Note that this is true whether or not the journal actually checks. The implicit threat of an audit may suffice.

Vance

---

On this topic of releasing raw data, WAME readers may be interested in this commentary by Andrew Vickers in Trials, "Whose data set is it anyway? Sharing raw data from randomized trials" http://www.trialsjournal.com/content/7/1/15

He argues that "Journals and funding bodies should insist that trialists make raw data available, for example, by publishing data on the Web".

Matt Hodgkinson-Barrett 
Senior Editor, BMC-series Journals

---

As seen below, the ethical guidelines of the American Psychological Association states that data must be shared for verification after publication and implies (but does not state) that it may be asked for in the review process (http://www.apa.org/ethics/code2002.html#8_14). 

“8.14 Sharing Research Data for Verification
(a) After research results are published, psychologists do not withhold the data on which their conclusions are based from other competent professionals who seek to verify the substantive claims through reanalysis and who intend to use such data only for that purpose, provided that the confidentiality of the participants can be protected and unless legal rights concerning proprietary data preclude their release. This does not preclude psychologists from requiring that such individuals or groups be responsible for costs associated with the provision of such information.

(b) Psychologists who request data from other psychologists to verify the substantive claims through reanalysis may use shared data only for the declared purpose. Requesting psychologists obtain prior written agreement for all other uses of the data.

8.15 Reviewers
Psychologists who review material submitted for presentation, publication, grant, or research proposal review respect the confidentiality of and the proprietary rights in such information of those who submitted it.”

----------------

Out of curiousity, what prompted the decision to request data? I can only imagine doing so if the results were so counter-intuitive or against the grain of the literature that I thought the author was in error.

Nancy Darling

---

Bill makes some excellent points (as usual!) -- and of course editors can only ASK for data, they are not like courts which can subpoena material. If the author refuses to supply the data, this must leave lingering doubts in the editor's mind, and I wonder how this fits with COPE's guidelines for editors which state:

'If editors suspect misconduct by authors…then they have a duty to take action. This duty extends to both published and unpublished papers.'

and goes on to state:

'If the editors are not satisfied with the response, they should ask the employers of the authors...or some appropriate body...to investigate.'

It seems to me that refusal to supply data in cases of suspected misconduct constitutes a definitely unsatisfactory response. By simply rejecting the submission, the editor has not fulfilled his/her responsibilities to investigate (which COPE notes, is 'an onerous but important duty') (and presumably the author will simply submit to another journal). However, I also appreciate that editors may be reluctant to raise concerns with employers or authorities on the basis of suspicion rather than evidence (which is presumably why they need to see the data).

I'd be interested to hear what other WAME members think. If you have suggestions about other aspects of COPE's guidelines, we would also like to hear them (perhaps best to send them off-line so we don't clog up the WAME listserve) as the COPE Council is currently reviewing the guidelines and plans to revise them.

Liz Wager
(Member of COPE Council)

---

I'm not sure if Farhat Farrokhi's initial question was about asking for raw data to investigate possible fraud or misconduct, or simply to understand the analyses in the paper concerned. If the journal just wanted to understand more fully the analyses, it would have been routine and non-contentious to ask for more detail by posing some very specific statistical questions. I don't see why the authors would have refused.

Asking to see raw data—particularly a whole dataset—is a much bigger request. It tends to imply that the editors mistrust the authors' work in some way. If that's what we're discussing here, WAME members may find this editorial useful: 

Jane Smith and Fiona Godlee

Investigating allegations of scientific misconduct  BMJ, Jul 2005;331:245-246;doi:10.1136/bmj.331.7511.245

http://www.bmj.com/cgi/content/full/331/7511/245

It said:

"In practice there's a limit to what journals can do—because they have  neither the resources nor the authority to conduct investigations to  resolve suspicions about data. Yet they are, as Smith points out, in the  position of "privileged whistleblowers." Privileged because it is often their expert peer reviewers who first raise the suspicions about odd looking data in a research study; because they can ask authors for raw data and ask them to explain discrepancies (which may remove or strengthen the existing doubts); and because they can then ask a legitimate authority (such as an employer, university, or funding body) to investigate. The problems arise when there is no authority or the authority doesn't see it as its task to investigate."

And the BMJ's policy goes a bit further than the general guidelines from COPE by referring anonymised cases to the journals' own ethics committee.

These links will tell you more:

http://bmj.bmjjournals.com/advice/bmj_ethics.shtml

http://bmj.bmjjournals.com/advice/editorial_policies.shtml#miscon

http://bmj.bmjjournals.com/advice/transparency_policy.shtml

Lastly, here's a detailed case study:

 
EDUCATION AND DEBATE:

Caroline White

Suspected research fraud: difficulties of getting at the truth BMJ, Jul 2005;331:281-288;doi:10.1136/bmj.331.7511.281 http://www.bmj.com/cgi/content/full/331/7511/281

Trish Groves
Deputy editor, BMJ 

---

Leading scientific culture denoting trust, dignity, and prestige among its members to spy work and surveillance activities undervalues knowledge and those who have strived for it throughout their lives. Let's remain trustful of one another and be confident that time reveals the truth in case of fraud or any cheating act.

Dr. Manoochehri

---

Trish makes good points, and the BMJ has been a real leader in this area. It seems to me that if the editor, based on reviewer input, has questions about the integrity of the data, he or she has only 2 recourses: reject the manuscript or allow the authors a chance to convince the editor that the data are correct. Assuming that an author would not want his or her manuscript rejected, then what are the options to asking for the data (if the editor, the journal's biostatistical reviewer, or the external peer reviewer is willing to evaluate  them)?

Should the editor ask the authors to convince him or her that the data are correct? The authors have presumably already made their best stab at it in the manuscript. It was suggested that sending the analyses might help. Yet they can edit any output from any statistical package to say what they want it to say.

One commenter on this exchange suggested that we trust the authors to be honest, and that eventually, dishonest authors and bad data are uncovered. First, we have no idea how much bad data and dishonest authors are published, only what has been uncovered. Second, in almost all cases, we DO trust the authors and don't ask for data. The results make sense to the reviewers and tell a compelling and consistent story.

However, sometimes, as in the case that started this all off, there are questions about the integrity of the data and the honesty of the authors. The easy way out is to simply reject such a manuscript (knowing that the authors will likely submit it to another journal). The journal editor willing to check the data is actually providing a service to the author. The author can refuse to comply, of course, which would result in rejection because the question of data integrity cannot be answered.

Bill Tierney