Proposal of a Centralized Manuscript Assessment System for Peer Review

Please concentrate only on the central concept of the idea and not on the specific examples and exact numbers (such as number of reviewers, holding period of articles etc), which can always be modified after consensus.

I am open to any criticism and suggestions to perfect the system.

V Raveenthiran
Associate Editor, Indian Journal of Surgery

Proposal of Centralized Manuscript Assessment System (CMAS)

“Conventional wisdom says that all manuscripts will get published one day in some journal provided the author keeps trying” - Peter Morgan, Former Editor of CMAJ.

The present system of peer-review

In the existing system of peer review, authors submit their manuscript to a chosen journal (by online or by airmail). After in-house scrutiny for suitability of the paper and for formatting of the manuscript, editor sends the paper for one or more (usually maximum 4) reviewers. Based on the reviewers’ comments the paper is either accepted, rejected or sent again to author for modifications. The modified manuscript is again sent to reviewers or decided by editor and a final decision is reached.

When the manuscript is rejected, the author submits the paper to another journal and it undergoes all the previously stated steps. This process continues until either the paper is published in some journal (no matter what the impact factor is) or the authors are tired of revising the paper and abandon it.

Disadvantages of the present system

1. By repeated peer review cycles, precious time of reviewers and economic resources are wasted.
2. Many times, after tedious peer review process, journals reject manuscripts simply because they have too many good papers or to offer variety to their readers. Ensuing time delay is a loss for authors and wasting of resources is a loss for the journal
3. Review comments are used as tutorials by authors to “patch-up” their blunders during subsequent submission. In considerable instances they do not gather additional data to rectify the mistakes but they simply modify the text and submit it elsewhere. Modifying a study conclusion by additional data collected after completion of the study is also equally wrong.
4. Who reviews the manuscript is debatable. Some journals use expert reviews but many journals simply use peer reviews. Peer reviewer need not be an research-expert in a given topic. For example, there will be a vast difference in the quality of review when a paper on fetal surgery is assessed by Michael Harrison (world authority on this) and by an (non-specific) academic pediatric surgeon. Experts tend to be too stringent while ordinary reviewers tend to be more lenient. Limited reviewer pool of many journals does not permit a combination of expert and ordinary peer review of each paper to have balanced opinion.
5. Difficulty of editors (especially small journals) to find a suitable reviewer for a given manuscript especially if it is an unusual topic.
6. Interdisciplinary research pose dilemma to authors as to the choice of journal. For example, a paper on “Breast mammography for Cancer detection” can be submitted to surgery, radiology, cancer and preventive medicine journals. Often the author chooses to publish in a journal of his specialty.. If the author is a surgeon he/she would prefer surgery journal instead of radiology journal in the above case. Thus dissemination of knowledge to interrelated discipline is restricted or at least delayed.

The proposed new scheme (CMAS)

There will be a common online manuscript submission system (CMAS) to which the authors will submit their manuscript irrespective of the specialty. This web site will only be accessible to editors of member journals and accredited reviewers of the system. Entry is restricted by password. However, any author can submit her/his manuscript and track its progress without having access to the rest of database.

Let us take an example of the following paper:

Comparison of Mammography versus MRI in early detection of ductal carcinoma in women

The CMAS will classify the manuscript under different sub groupings like gynecology, radiology, surgery, medicine etc. In our example, the paper will be classified under multiple headings namely surgery, radiology, cancer, prevention etc. A full time managing editor of CMAS will send all the papers for reviewing by assigning the job to various appropriate reviewers. At least one of the reviewers will be an expert reviewer of the given topic and the other will be an end-user of the article (ordinary reader or an expert in the given broad field; but he/she need not be a research-expert in the given topic). (For example all urologists need not be research-experts in urethral injury). In case of interdisciplinary research, the number of reviewers will be multiplied by the involved number of disciplines. In our example, the paper will be sent to two research-experts - one in radiology and one in oncology (and a Statistician if needed). It will also be sent to an onco-surgeon (clinician) and preventive medicine expert as they are the end-users of the conclusions.  Thus, it will be reviewed by at least 4 persons. (At this conceptual level let us not concentrate at the exact number of reviewers. At the stage of implementation, WAME can draw a consensus on this).

CMAS will have a pool of trained reviewers. All accredited reviewers should have undergone some tutorial course of WAME. Only such certified experts will be enrolled as reviewers in CMAS. This will ensure uniformity of review qualities. The CAMS reviewers will assess the papers assigned to them. The reviewers shall have a password to enter the site. A reviewer will be allowed to access only the papers assigned to him / her. The review process of CMAS will be slightly different from conventional review. In CMAS, experts will scrutinize papers for scientific blunders and interpretation errors and they should not concentrate on language, style, presentation format etc.

Once the review process is complete, detailed reviewer comments will be made available at CMAS online site. Editors of various journals will periodically log on to the site and browse the manuscripts and review comments. Sub grouping of manuscripts will enable easy browsing of relevant articles of a given specialty. The editors will be given a password for security reason. All papers with completed review process along with reviewer comments will be made open to all editors of member journals. The editors will choose those articles, which they deem suitable for their readership. More than one editor may choose a given article. In our example the paper may be chosen by editors of Atlantic Journal of surgery, Peruvian Journal of Cancer Prevention and Greenland Journal of Radiology. Two journals of the same specialty may also compete for a single good article.

In the next step, all the editors interested in a paper will contact the concerned author offering to publish the work in their respective journals. Now the onus is on the author to choose the journal where he/she wants the work to be published. Once the author decides to publish it in a journal X, an agreement is entered between the journal and the author about copyright etc. At this stage the manuscript will be removed from CMAS database and transferred to the concerned journal office. The editor of the journal X can make alterations in language, style, formatting according to his/her journal’s preference. Figures may be added or deleted, length may be cut short and title may be modified etc with authors’ concurrence. And finally the manuscript is published in journal X.

“If your manuscript is rejected by half-a-dozen journals then you should consider burning it in cross roads and start working on a new research project” - Peter Morgan, Former editor of CMAJ in “An insider’s guide for authors and editors”

If no one claims a paper submitted in this system, it must have been a poor-science article that is worthy of burning at cross roads. All such unclaimed articles can be deleted from the CMAS data pool after a holding period of 5 years.

Advantages of the CMAS

1. No time or resource is wasted in repeated peer reviewing of the same rejected article by various journals.
2. Authors need not bother too much about where to submit the work and about formatting according to the journal’s requirement.
3. Authors have the unique privilege of selecting the journal to publish their work. This is in contrast to conventional system where editors choose.
4. Small journal editors can get rid of their nightmare of getting adequate number of suitable articles for their next issue. A vast pool is open to them in CMAS.
5. Scientific blunder can never get published, as the authors loose their chance of manipulating the text during a subsequent submission.
6. It is easy to detect fraud, plagiarism, duplicate publication and such misconduct. Once detected it is easy to keep away such misbehaving authors from polluting science.
7. Peer review of member journals will have comparable quality of peer revieweing. In traditional system various journals differ in their peer review quality. (please do not confuse this with quality of papers)
8. For other advantages see discussion on criticisms.

Some possible criticisms and my explanations

I anticipate the following criticisms of the new proposal and my possible answers are here.

Criticism 1

Who will fund the project and maintain the Web site?

Even now individual journals/publishers maintain or hire various Web sites for manuscript submissions. The cost of maintaining a combined Web site will be definitely cheaper than the cost of maintaining individual Web sites. Each member journal, according to their subscription status, can contribute money to the maintenance of the Web site. One or two full-time paid managing editors is sufficient to maintain the site.

Criticism 2

Who will be the member journals?

It is ideal that all biomedical journals be the member of such system. But reality may be different. At least WAME journals can be members initially. Success of the system will attract others.

Criticism 3

Will it not affect the interest of a journal in terms of sales, impact factor etc?

Editors of small journals will have this apprehension. If “New American Journal of Medicine” and “Unknown Journal of Medical Care” simultaneously offer publication of an article with best review comments, authors will prefer the former which is well known and has a high impact factor. By corollary, New American Journal will always attract and grab good papers while the Unknown journal will have to publish only leftover articles. This apprehension need not be true. Other than impact factor, there are many ways of attracting a good paper even to a small journal. Offering publication in the next issue (so called fast-track publications, but sans payment), free reprints, free subscription to the journal for next one year, waiving of colour charges, allowing the author to retain copyright, etc are some of the techniques to lure authors of good papers. Affordable journals may even offer financial reward to authors. Will not such awards be beneficial to authors?

Criticism 4

Will it not be equal to publishing the paper online before the actual publication in print?

As access to the site and manuscripts is restricted to editors of various journals, it cannot be considered as online publication. Even the accredited reviewers will not have access to the entire database; they can access only those papers assigned to them. Similarly authors can only track the progress of their articles without having access to the rest of the Web site content.
________________________
A similar system already exists as Index Copernicus Publishers Panel

Mark R. Graczynski
CEO, Index Copernicus International, Inc