eMJA     The Medical Journal of Australia

Home | Issues | eMJA shop | Classifieds | Contact | More... | Topics | Search | Login | Buy full access   

Conference report

Blood, sweat and tears at the International Congress on Peer Review in Biomedical Publication

Mabel Chew and Ruth M Armstrong

MJA 2001; 175: 573-574

What happens at such a congress? Editors of biomedical journals, researchers, writers and others interested in biomedical publication meet to present and discuss research on aspects of the editorial process, including peer review. Importantly for journal editors like ourselves, the congress provides a forum for us to "practise what we preach" when it comes to publishing with an evidence base. This was the fourth such congress, organised by JAMA and the BMJ Publishing Group, and held in Barcelona on 14-16 September 2001.¹

The first plenary session asked the provocative question, "What exactly makes an author?". Guidelines from the International Committee of Medical Journal Editors (ICMJE) state that authorship credit should be based only on:

1. Substantial contributions to study conception and design, or acquisition of data, or analysis and interpretation of data;

2. Drafting the article or revising it critically for important intellectual content; and

3. Final approval of the version to be published.

Four presentations highlighted the incongruity between this definition and actual practice (Frank Davidoff, Annals of Internal Medicine; John Overbeke, Nederlands Tijdschrift voor Geneeskunde; Susan van Rooyen, BMJ; and Graham Mowatt, University of Aberdeen). Should we be "fleshing out ghost authors and flushing out honorary ones?" asked a member of the audience (George Lundberg, Medscape). Some editors discussed their shift towards the more flexible concept of "contributorship", rather than the ICMJE-defined "authorship", which is regarded by many as unrealistic and restrictive. It was also agreed that authors should be made more aware that the credit of authorship carries with it accountability for the article. This may be particularly problematic when a published article does not reflect all co-authors' views on interpretation of the study findings or design weaknesses. Richard Horton (Lancet) encountered this "censorship" of dissenting authors' views by their co-authors when he attempted to approach all authors of 10 research papers published in his journal. Contributors frequently disagreed about key findings, weaknesses, implications and directions for future research. Those least involved in writing the article were most likely to criticise the study's design, but were unlikely to have their criticisms recorded in the published article.

As might be expected, the congress included plenty on peer review. That this form of review has many weaknesses is not a new notion, and was further supported by results of several studies. Presenters concluded that there is variable agreement between reviewers and between reviewers and editors (Ana Marusic, Croatian Medical Journal), that it is difficult to improve the standard of reviews (Michael Callaham, University of California), and that it is questionable whether the process improves the quality of published research (Tom Jefferson, UK Cochrane Centre). Even the positive results of a study of the effect of statistical review on manuscript quality failed to reach statistical significance (Erik Cobo, Medicina Clinica). So, are we editors pawing at the ground and snorting about nothing? Participants discussed the fact that current peer-review practices are empirical, and that we need to test different models and find appropriate instruments to measure the value (or otherwise) of this sacred cow.

Giving the term "bull" a more colloquial twist, even the most timid editor likes to believe she would jump in the ring and slay studies with poor methodologies or misleading interpretations of results. Several presentations examined the quality of published studies. Commenting on a review of screening mammography that has generated much controversy, Peter GØtzsche (Nordic Cochrane Centre) described crucial flaws in major published trials of this technique. He claimed that misclassification bias and selective reporting of trial results may have led the benefits of mammography to be overestimated, with several countries subsequently adopting national mammographic screening programs. As an aside, we wonder whether anyone who proposed abolishing screening mammography in light of these findings would suffer the same fate as the average bull. At question too was the failure of many authors to interpret their results in the context of the rest of the medical literature (Mike Clarke, UK Cochrane Centre).

A journal's correspondence column can be a useful tool for highlighting study weaknesses after publication. However, Richard Horton (Lancet) presented evidence that letters highlighting flaws and limitations in published studies may be ignored in subsequent practice guidelines. Incidentally, half the criticisms in the letters he studied went unanswered in authors' published replies, leaving the audience wondering whether editors have become apathetic or merely inattentive.

All this was like waving a red cape for Douglas Altman (ICRF/NHS Centre for Statistics in Medicine), who charged to the microphone to point out that reviews of randomised controlled trials in peer-reviewed journals have long shown that many are published with significant flaws. He called for improved standards of peer review and full and improved reporting, including publication of research protocols at the time trials are established rather than when the results are reported. He also urged editors to facilitate post-publication correspondence, to be tougher on quality and to speak out against the link between research publications and career advancement.

Devotees of blood sports will be no strangers to debates on ethical issues. During the congress, several participants locked horns on the issue of blinding and masking in peer review, and real-life editorial ethical dilemmas were discussed in an open meeting of the World Association of Medical Editors. Research presented showed that major journals have improved their reporting of informed consent and ethics committee approval (Veronica Yank, University of California), as well as funding sources and potential conflicts of interest (Anu Gupta, Yale University). However, both studies also showed that journals could do better. Delegates agreed that transparency is the key to preserving public confidence in the reporting of medical research.

As with all successful public spectacles, much of the action at the congress took place between scheduled events. Delegates shared insults and anecdotes, and then a three-minute silence to consider the dead, dying and grieving after the September 11 attack in the United States. The bullfight had a subdued and diminished crowd (135 of the 410 registrants were unable to attend, and four of 41 main presentations were cancelled, because of travel disruptions in the wake of September 11). An observer might wonder how we could muster the enthusiasm to wrangle over the finer points of editing and publishing while the forces of real violence were unleashed in an arena just across the Atlantic. However, as we stepped from the ring and dusted ourselves off, we were also reminded that, just as war traverses all aspects of life, so too do health issues. The public deserve the highest possible standards for reporting of research affecting their health.

Acknowledgements

We thank our fellow Australian bullfighters Rhana Pike (NHMRC Clinical Trials Centre, Sydney, NSW) and John Dowden (Australian Prescriber, Canberra, ACT) for their comments and for championing a questionable metaphor.

References

1. Fourth International Congress on Peer Review in Biomedical Publication. Program and abstracts. <http://www.ama-assn.org/public/peer/prc_program2001.htm> Accessed Nov 2001.
2. International Committee of Medical Journal Editors. Uniform requirements for manuscripts submitted to biomedical journals. <www.icmje.org> Updated October 2001.

Authors' details

©MJA 2001
Make a comment