eMJA Internet Peer Review Studies

eMJA Internet Peer Review Study I

From March 1996 to June 1997, the eMJA, in cooperation with the University of Sydney Library, conducted a trial of open peer review using the Internet. In fact, the website of The Medical Journal of Australia was first developed as part of that experiment.

In the study, articles that had been accepted for publication after conventional peer review were published on the web before they were edited for print. The reviews were published with the article, and readers were invited to comment on the article and reviews before the text was finalised. Results of the first study suggested that open review on the Internet has some benefits.

The results of the first eMJA Internet peer review study were presented at The International Congress on Biomedical Peer Review and Global Communications, 17-21 September 1997, Prague, Czech Republic.

A full report was published in The Lancet:

Bingham CM, Higgins G, Coleman R, Van Der Weyden MB. The Medical Journal of Australia internet peer-review study. Lancet 1998; 352: 441-445.

  Click here for a short description of the study.

  Click here for the original study proposal (May 1995).

The last six articles to enter the first internet peer review study can still be viewed as they were presented then:

• A high incidence of melanoma found in patients with multiple dysplastic naevi using photographic surveillance John W Kelly, Josephine M Yeatman, Cheryl Regalia, Grahame Mason and Amanda P Henham
  Abstract - Article

• Management of childhood gastroenteritis in the community. Perceptions of general practitioners Jennifer E Porteous, Richard L Henry, Edward V O'Loughlin, Malcolm Ireland, J Lynn Francis and Robyn G Hankin
  Listed: Thursday 12 June 1997. Abstract - Article

• Hospitalisation for congestive heart failure: burden and outcomes Fiona M Blyth, Ross Lazarus, David Ross, Michael Price, Gary Cheuk and Stephen R Leeder
  Listed: Monday 2 June 1997. Abstract - Article
  Now edited and published in print, 21 July - MJA 1997; 167: 67-70.

• A model for the management of self poisoning Ian M Whyte, Andrew H Dawson, Nicholas A Buckley, Gregory L Carter and Catherine M Levey
  Listed: Friday 9 May 1997. Abstract - Article
  Readers' comments with reply from author added, Thursday 22 May.
  Further readers' comments with reply from author and extra text for article added, Thursday 29 May.
  Now edited and published in print, 4 August - MJA 1997; 167: 142-146.

• Prevalence of abdominal aortic aneurysms in diabetic men Eugen Mattes, Timothy M E Davis, Danian Yang, Dorothy Ridley, Helen Lund and Paul E Norman
  Listed: Monday 28 April 1997. Abstract - Article
  Now edited and published in print, 16 June - MJA 1997; 166: 630-633.

• Providing costing information to general practitioners -- will this intervention change behaviour and create cost savings? A systematic review Justin Beilby and Chris Silagy
  Listed: Monday 28 April 1997. Abstract - Article
  Now edited and published in print, 21 July - MJA 1997; 167: 89-92.

eMJA Internet Peer Review Study II

In September 1998 we launched a second, more ambitious, study. In this study, articles submitted for publication were circulated to reviewers via the world wide web (using password-protected access) and the review process was conducted as an online discussion between the journal editors, reviewers, authors and a small panel of consultants who represented a wider range of expertise and journal readership.

When an article was accepted for publication, both the article and the record of its review process were rapidly published on the world wide web for open review by the Journal's readers. After four weeks of open review, the article was finalised and published in print.

  Click here for the editorial that launched the second study.

  Click here for the protocol of the second study

However, Internet Peer Review Study II never advanced beyond the pilot stage. After piloting our open-interactive peer review system with ten articles, the study collapsed. This was not because authors or reviewers objected. Nine out of the ten (self-selected) authors said that they preferred the new system to the old, and so did 23 of the 26 reviewers. But there were two problems within the journal itself. One was technical: we lacked efficient tools for managing the electronic peer review process, in the context of a journal that was still paper-based, and this meant that the study was, for us, premature. The other was the reluctance of the editors of the journal, who objected to the new process for various reasons (among them uncertainty about how to moderate the review debate, a feeling that the process entailed extra work, and unwillingness to work with electronic rather than paper documents).

Both the technical and the human limitations on internet-readiness have largely disappeared from the MJA in the last couple of years, and we may venture again into new peer review methods. The increasing popularity of email and the continuing rise in computer skills within the medical profession mean that electronic manuscript submission and peer review are now favoured alternatives to paper-based systems.

Seven of the ten articles that entered IPRS II were published by the MJA and they are presented here as they appeared in the study:

• Sallie Newell and Rob W Sanson-Fisher. Australian oncologists' self-reported knowledge and attitudes about non-traditional therapies used by cancer patients Electronically published 9 November 1999, then edited and published in print on Monday 7 February 2000.
  Click here to see the final edited version.
  Click here to see the unedited version and peer review comments.

• Ian Mackie. Patterns of drowning in Australia. Electronically published 29 October 1999, then edited and published in print on Monday 6 December 1999.
  Click here to see the final edited version.
  Click here to see the unedited version and peer review comments.

• Alan Cass, Adrian Gillin, John Horvath. End stage renal disease in Aborigines in New South Wales: a very different picture to the Top End. Electronically published 21 July 1999, then edited and published in print on Monday 18 October 1999.
  Click here to see the final edited version.
  Click here to see the unedited version and peer review comments.

• Richard E Ruffin, David Wilson, Anne Marie Southcott, et al. A population survey of the reported ownership of asthma action plans. Where to from here?
  Electronically published 6 September 1999, then edited and published in print on Monday 4 October 1999.
  Click here to see the final edited version.
  Click here to see the unedited version and peer review comments.

• Girgis S, Ward JE, Thomson CJH: General practitioners' perceptions of medicolegal risk. Using case scenarios to assess the potential impact of prostate cancer screening guidelines
  Electronically published, then edited and published in print on Monday 4 October 1999.
  Click here to see the final edited version
  Click here to see the unedited version and the peer review comments.

• Cantor CH, Neulinger K, De Leo D: Australian suicide trends 1964-1997 - youth and beyond?
  Electronically published on Tuesday 8 June 1999, then edited and published in print on Monday 2 August 1999.
  Click here to see the final edited version
  Click here to see the unedited version and the peer review comments.

• March et al.: How best to fix a broken hip
  Electronically published on Monday 22 March 1999, then edited and published in print on 17 May 1999.
  Click here to see the final edited version
  Click here to see the unedited version and the peer review comments.

Theory of open peer review

The eMJA studies in internet peer review arose from a theoretical position that peer review processes would benefit from greater openness and procedural transparency. The rationale is explained more fully in the following publications:

• Bingham CM. Peer review and the ethics of Internet publishing. In: Hudson Jones A, McLellan F, editors. Ethics in biomedical publication. Baltimore: Johns Hopkins University Press, 2000.

• Bingham CM. Peer review on the Internet. Are there faster, fairer, more effective methods of peer review? In: Jefferson T, Godlee F, editors. Peer review in health sciences. 2nd edition. London: BMJ Publications, 2003. Pages 277-296.

• Bingham CM, Van Der Weyden M. Peer review on the internet: launching eMJA peer review study 2 [editorial]. Medical Journal of Australia 1998; 169: 240-241. <eMJA full text>

• Bingham CM, Higgins G, Coleman R, Van Der Weyden M. The Medical Journal of Australia internet peer review study. The Lancet 1998; 358: 441-445.

• Bingham CM. Peer review on the internet: a better class of conversation. The Lancet 1998; 351 (suppl I): 10-14.

• Bingham C, Coleman R. Enter the web: an experiment in electronic research peer review [editorial]. Medical Journal of Australia 1996; 164: 8-9.<eMJA full text>

Page created: 1996. Last revision: 21 December 2005.