Currently, indexing systems like PubMed, EMBASE/Excerpta Medica and ISI, through a selecting system, select some journals and index their articles. The selection procedure certainly aims at choosing the best journals. These indexing systems for limited resources can certainly not index all the publications. However, imagine that they have unlimited resources and can index all the published materials. Should they index all the journals or they should still use some selection criteria and index only some of them?

Farrokh Habibzadeh
Archives of Iranian Medicine
______________________________
The use of indexing systems may ensure quality, but it discriminates against smaller journals and new journals trying to get a foothold. Since many countries use the number of articles published in indexed journals as criteria for professor status, often physicians will preferentially send their articles to indexed journals instead of newer journals that are trying to establish themselves. As such, they never get a chance to improve—other than begging for the good will of established authors to send an article that for whatever reason was not accepted elsewhere.

I am in favor of opening the indexes up to all journals.

Yosef Leibman
Founding Editor, IJEM (not indexed, but tried)
______________________________
Some academic centers use the number of published articles of their professors for their career promotion since it is well-known that current indexing systems screen journals and select high quality ones. My question, really, is—what is the main purpose of indexing? Is it to provide information to its end-users on high quality published articles or  should it provide all pieces of data published (regardless of their quality), hence, the name "indexing system"? I, for one, believe that the latter condition is important for many researchers, such as those who want to do a meta-analysis/systematic review. They want to access all the available data; they first select their articles and then, in an appraisal procedure, select those articles that meet their criteria (quality, etc).

I think the main goal of indexing systems is to provide enough information on "what" article was published on "which" subject and to help us retrieve that article easily. Certainly, it is great that a center (maybe a section of the indexing organization, itself) evaluates the quality of the indexed articles and reports it.

Farrokh Habibzadeh
Archives of Iranian Medicine
______________________________
I believe that in the not-too-distant future, indexing will become obsolete as Google-like searches of the scholarly literature become more available and sophisticated. It will be interesting to see what effect this has on ISI's impact factor and standards for tenure and promotion.

Diana J. Mason
Editor-in-Chief, American Journal of Nursing
______________________________
What is the main purpose of indexing literature? A very good question...but as I understand from the tone of discussion, this question should be split in two:

[1] What is the purpose of indexing literature [Medline-like]?

[2] What is the purpose of ranking literature [ISI-like]?

Then we may ask yet another question:

[3] What should be the selection criteria for indexing/ranking of literature?

The answer to the first question is obvious: it is to create "a one-stop" source of information for further research. It is literally impossible to search individual journals, for there are too many of them on the market.

Scientists may [and often are] not aware of all journals within their subject of interest. Therefore an indexing site like Medline is a great help to stay updated.

Why do we need ranking of literature, like ISI does with their "Impact Factor"? The answer here is a little more complicated. ISI, IF-like journals ranking has been set up for librarians to help them select most suitable journals for university libraries. Because, at the time ISI journals master list was born, there was no technology available for indexing all journals, the evaluation procedures has been set up to select a group of "best" journals. This is now another subject if these selection criteria are fair to all.

In years, this impact factor-based system has been adopted by universities for judging scientific quality of individual researchers’ careers. There has been a lot of criticism on using impact factor or citations to judge scientific careers for a number of reasons, like manipulation with so called "self citations" or "negative citations", to name only two.

Because "citation-based" scientists’ evaluation is evaluation of one parameter only, it cannot truly and objectively weight scientists’ career or research potential/experience. To do this we need a multi-criteria scientific output evaluation system, part of which should be evaluation of the literature.

So what should the system for indexing/ranking of the literature be like?

First of all it should be OPEN. Open means, that ALL [scholarly and peer-reviewed] journals should have a chance for evaluation and ranking.

There is no more place for obscure selection policies. The journals that are to be indexed [ie, "which journal "is good enough" to EXIST] should not be "the committee’s" decision. This IS THE READER’S decision.

Initial journal evaluation should be based on TRANSPARENT and UNEQUIVOCAL multi-parameter methodology.

Continuous, annual evaluation should be based on DOWNLOADS rather than citations, because citations do not guarantee actual reading of the article.

This is often "a copy-paste" mechanism. Downloads reflect actual reading more closely, as this is mostly done AFTER reading the online abstract.

Let me propose a philosophy of OPEN INDEXING based on the following rules:

[1] Inclusive:

All existing literature indexing sites—including Thomson Scientific—are exclusive in nature. They use review committees and procedures to select journals for coverage. Nowadays, technology, especially computer technology, allows vast data collection and analysis, so all eligible journals should have an opportunity to be evaluated.

[2] Transparency:

Science should be transparent by the definition. So the evaluation of science has to be transparent as well. There should be no more unclear rules and policies, undisclosed selection committees and methodologies. All players of the "scientific market", such as scientists [who are authors and readers], editors, and publishers deserve—no, have the right—to know the rules that govern evaluation of their work, as this directly affects their career development or journal development. Simply, it directly affects their future.

[3] Decentralization:

With a transparent selection policy and evaluation methodology, there is no more need to keep one "central committee". Instead, we propose to establish local [like country] indexing committees, which will take responsibilities for local journals. With transparent procedures there is a very low risk for local bias.

[4] Allows journals’ self-administration:

Transparency enables it. Journals should be given a dedicated access to their accounts at the indexing site for self-management of journal information, such as contact addresses and phone numbers, list of editors, information on policies, and instructions for authors. Journal staff should also be allowed to upload abstracts and or PDF files, and even set up pricing policies. Such an approach eliminates mountains of problems with timely updates to information.

[5] Fair access to information:

Open Indexing idea goes hand-in-hand with the Open Access publishing model in which it is assumed that publicly funded research should be freely available to the public without restriction at the time of publication since tax-payers have already paid for it. However, there still is the question of “who is a tax-payer?”.

Should research paid by tax-payers in one country be freely available to readers from another country who paid their taxes elsewhere? This problem can also be solved in the Open Indexing model; however, there is no time now to describe it.

[6] Part of multi-parametrical career evaluation:

We are all aware, that use of Impact Factor in evaluation of a scientist’s career is a simple misuse. However, we use it because there has been no other system. Some attempts to create a better indicator include H-factor, Y-factor, or Prestige Factor. The common weak point of all these initiatives was that they all have manipulated only with citation counts and/or impact factor calculation.

[7] Networking scientists.

Looking forward to your comments,

Mark R GraczynskiEditor/Publisher, Medical Science Monitor
______________________________
I wonder whether indexing will ever become obsolete, Diana. It's like saying indexing in a library will become obsolete as well. For what is indexing but categorising in a data base maintained in a library, real or virtual?

As long as matter gets written, some form of categorising/indexing will be needed. And as long as quality is a criterion, discrimination between those that pass the test and those that don't will remain a criterion for indexing in a quality data base.

Having said that, I agree with Yosef that smaller journals do suffer because they may not be indexed. They have no option but to struggle up the hard way, and make their mark establishing their own USP. All late entrants everywhere have to face this.

It's a harsh reality, but no amount of papering over it will smooth the path.

However, new entrants have an advantage. They need not repeat the mistakes of older journals. They may have 'greater fire in their belly'. They can be less fossilized in their approach.

If they learn quickly, and remain persistent enough, indexing in a quality database may not remain that distant a dream.

Ajai Singh
______________________________
I still think those who access the indexes should decide—occasionally poor journals do publish very good articles, and sometimes very good journals produce very poor articles.

A system where indexes are opened, but the journals list impact factors and other pertinent information (such as whether it was industry sponsored, etc) would greatly help researchers and readers alike. Furthermore, those who want to weed out low impact journals could insert this limit in their search.

IJEM for example is not indexed, but the debate between NEXIS and the Canadian C SPine rules appeared in our journal first, before being revised and appearing in the New England Journal. Those looking for all the references on the subject would have missed this first article.

Meta-analyses are always plagued by articles that do not fit their criteria and, as such, they have fewer numbers and less impressive results. Getting all the studies would greatly change this.

Yosef Leibman
Founding Editor, IJEM
______________________________
It seems that there is a need for a wider coverage of a so called "Main or Major Index body" within the vicinity of medical indexing, where this will not eliminate the need for those highly critical index systems available which aimed at high-quality article production and grounds for competition. This has no conflict with what newly established journals may intend to achieve. If such main indexing systems could be performed and established under the direct supervision of WAME, then other well known indexes would remain and even become more critical for including criteria while the chance of indexing for the purpose (stated by some) of availability of materials would automatically be met. So I encourage the formation and registration of a World Wide Index for Medical Journals (WWIMJ)—including every published material in available languages—which needs the support of all currently recognized index systems and WAME members. Please comment and advise.

Ghassem Ansari
Former Editor in chief, Beheshti University Dental Journal
______________________________
The issue of indexing is complex and interesting. The need for indexing systems arose in the print era, when basic catalog information wasn't complete enough for people to find information about specific articles in journals or chapters in books. Much has changed since the machine readable and electronic era. There is a whole academic field/specialty of study that deals with this issue—library and information science.

There is also the business side where millions are generated from indexing products.

In the medical field several indices exist to tackle the issue of indexing thousands of journals (MEDLINE, CINAHL, Embase, PsyINFO, etc.). It is interesting to learn how today's MEDLINE/PubMed database grew from the previous print product called Index Medicus and why different journals are indexed with different levels of indexing.

There are quite a number of fact sheets that explain indexing in MEDLINE.  http://www.nlm.nih.gov/pubs/factsheets/factsheets.html - for all fact sheets http://www.nlm.nih.gov/pubs/factsheets/jsel.html - journal selection

http://www.nlm.nih.gov/tsd/serials/lji.html - list of journals indexed

The following link has information on the indexing initiative at NLM - http://ii.nlm.nih.gov/.

The following link is a good overview of Indexing. The value of indexing: http://www.factiva.com/infopro/indexingwhitepaper.pdf

Jessie McGowan
Associate Editor, Journal of Clinical Epidemiology Institute of Population Health
______________________________
Considering that WAME is a virtual association with limited funds, I hardly believe that it can directly supervise such a huge indexing system, which surely needs large amounts of money, many staff, complicated hardware, and vast physical space, among other things, to run appropriately.

Farrokh Habibzadeh
Archives of Iranian Medicine
______________________________
We are ready to provide all of this. We have enough infrastructure and staff to provide quality service. As you know IC has been running well for a couple of years, and now even links articles with the full profiles of their authors. In our system scientists/authors can network themselves creating "virtual research groups" for further research.

Please contact me if you want to continue...

Mark R Graczynski, Head, Index Copernicus Project