Citation metrics for appraising scientists: misuse, gaming and proper use

To the editor: Ioannidis and Boyack discussed misuse of and gaming mechanisms for citation metrics in the publication records of scientists.1 Studies have suggested some other limitations regarding citation‐based bibliometric indices in the evaluation of articles2 or journals.3 A recent study showed the confounding effect of highly cited items on impact factor calculation.3 Previously, we measured a considerable effect for a series of highly cited articles affiliated with the Institute for Health Metrics and Evaluations (IHME; University of Washington, Seattle) on the impact factor of The Lancet.4 Here, we evaluate the impact of these articles on the researchers’ quantitative bibliometric indices.

Using the Scopus (Elsevier) database, we identified the 20 authors with the greatest involvement in IHME‐affiliated articles. We then extracted and calculated the scientometric indices (H‐index, total number of published items, and citations) of these authors, with and without their IHME‐affiliated articles.

The mean (standard deviation) percentage of IHME‐affiliated papers for the authors was 47.5% (20.6%) of their total publications, leading to 81.6% (13.9%) of their total citations. Further, 49.6% (18.3%) of all authors’ H‐indices were related to IHME‐affiliated papers (Box). Accordingly, collaboration in HME‐affiliated articles can easily provide a large number of citations for authors and increase their H‐index.

We believe that scientometric indices based on the citation count may require correction to avoid the confounding effect of such highly cited items. Further investigations considering all IHME collaborators are needed for a better understanding of the IHME effect on scientometric indices. We recognise that IHME‐affiliated papers are but a sample; the effect of other similar series of highly cited articles on bibliometric indices requires further evaluation.

Box – Effect of articles affiliated with the Institute for Health Metrics and Evaluations (IHME) on scientometric indices (total published items, citation count and H‐index), by the 20 most published authors

---