Studies were eligible if they were comparative studies evaluating the quality of RCT reporting; one of the comparator groups included RCTs reported in CONSORT-adopting journals; and outcomes reported included any of the 22 items on the CONSORT checklist, summary scores based on the CONSORT checklist, or any measures of overall trial quality.11 Study quality was not an exclusion criterion, and studies were not excluded based on publication status or language of publication.

Journal adoption of CONSORT was defined as a statement in the “instructions to authors” to follow CONSORT in preparing manuscripts, or a requirement for authors to submit a completed CONSORT checklist with their manuscript.

Eligible studies fell into three groups: those that (a) compared CONSORT adopters and non-adopters after the publication of CONSORT, (b) compared CONSORT-adopting journals before and after publication of CONSORT, or (c) a combination of (a) and (b). We did not specify a primary outcome for our review, but included as outcomes: (1) reporting of any of the 22 items on the CONSORT checklist, (2) summary scores based on the CONSORT checklist, or (3) any measures of overall trial quality.

The main search strategy was developed in MEDLINE (1996 to 2005 week 28) and customised for EMBASE (1996 to 2005 week 32); both were executed through the Ovid interface. In addition, the Cochrane Methodology Register (up to 2nd Quarter 2005) and the Cochrane Database of Methodology Reviews (up to 2nd Quarter 2005) were initially searched via Update Software in 2004 and then via Wiley InterScience for the 2005 update. The Science Citation Index, Social Sciences Citation Index and Arts & Humanities Citation Index (June 2005) were searched through the ISI Web of Knowledge interface.

We searched for grey literature, checked the bibliographic references of relevant articles, and consulted with CONSORT experts.

The search was designed to retrieve citations published since 1996, when CONSORT was first published. Keywords included randomised controlled trial, publication bias, CONSORT, and epidemiological methods. The full search strategy is available from the authors.

Two reviewers (A C P and A M) independently reviewed titles and abstracts using broad screening criteria (comparative study, and outcomes included some measure of trial reporting quality). Articles of potential relevance were further assessed (by A C P and D M) for eligibility. Disagreements were resolved by consensus. The reviewers were not blinded to authors or journals.

A single non-blinded reviewer (A C P) assessed reporting quality using a five-item checklist based on principles of internal and external validity.12 Quality assessment included the use of a control group, RCT cohort ascertainment, how the authors reached agreement on whether a checklist item was included, and whether the reviewers were blinded when assessing outcomes.

Data were extracted by one reviewer (A C P) using a standardised data form, and then checked for accuracy by a second reviewer (D M). Only “consensus” data were used in the review. All data were entered in Review Manager 4.2 (Nordic Cochrane Centre, Copenhagen, Denmark). Dichotomous data were expressed as risk ratios (RR) with 95% confidence intervals. Continuous data were expressed as standardised mean differences with 95% confidence intervals. Consistency across the studies was examined using the I-squared test. Data were pooled using fixed effect models. A priori subgroup analyses included a comparison of RCTs published in journals that specifically require submission of a completed CONSORT checklist with journals that are not CONSORT adopters.

Authors of several possibly relevant studies were contacted for clarification of eligibility and invited to contribute unpublished data. We did not contact the authors of all included studies for additional unpublished data.

Our search strategy identified 1128 studies; 248 were potentially relevant and eight of them7,8,13-18 met our inclusion criteria (Box 1). One study was included after the author provided unpublished data.17 Six studies that examined changes in the quality of trial reporting over time or compared quality of trial reporting between journals were excluded, either because they did not include a comparator group or were not published in CONSORT-adopting journals,9,19-21 or because insufficient detail was available to determine eligibility.10,22

The number of RCTs in individual studies ranged from 13 to 240, and the number of journals in individual studies ranged from 1 to 68 (Box 2). All studies were published in English. One study used a different definition of CONSORT adoption (journals in which the editor applied the CONSORT checklist were considered CONSORT adopters),7 but 88% of RCTs in the study met our definition of CONSORT adoption. For all other included studies, the definition of CONSORT adopters matched our definition.

Overall, 37 different outcomes were reported (range, 1–16 per study) (Box 3). This variability resulted in most outcomes being reported in only one study. Outcomes used in more than one study were reporting of allocation concealment (five studies), sequence generation (three studies), blinding (four studies), participant flow (two studies) and overall number of CONSORT checklist items (two studies, one using a total of 30 items, the other 40 items). Blinding as an outcome was assessed as reported or not in one study,17 and blinding of participants and data analysts in three studies.7,17,18

Box 2 summarises the methodological quality of the studies. All studies were quasi-experimental, with only one study having an a priori defined control group.8

CONSORT-adopting journals had better reporting of the method of sequence generation and allocation concealment (Box 4A). For the two studies that reported total number of CONSORT items, the standardised mean difference was 0.83 (95% CI, 0.46–1.19). CONSORT adoption appeared to have less effect on reporting of participant flow and reporting of blinding of participants or data analysts.

Although we planned a priori to determine the influence of journals requiring submission of a completed CONSORT checklist when manuscripts are submitted, this comparison was impossible as very few journals actually require checklist submission. The paucity of data also precluded meaningful examination of publication bias. Tests for heterogeneity were all non-significant.

The descriptions of the method of sequence generation, participant flow and total CONSORT items (standardised mean difference, 3.67 items; 95% CI, 2.09–5.25) were better after adoption of CONSORT (Box 4B). CONSORT adoption appeared to have less effect on allocation concealment. The paucity of data precluded meaningful examination of publication bias. There was no evidence of heterogeneity.