Study selection: Randomised comparisons of fully automated web-based interventions specifically targeting adolescents and young adults (ie, typically school or tertiary students, ≤ 25 years old) versus other interventions.
Data synthesis: 16 relevant studies were identified, and data were extracted from 13 of the 14 reporting on alcohol use by young adults. The alcohol interventions had a small effect overall (d = − 0.22) and for specific outcomes (level of alcohol consumption, d = − 0.12; binge or heavy drinking frequency, d = − 0.35; alcohol-related social problems, d = − 0.57). The interventions were not effective (d = − 0.001) in preventing subsequent development of alcohol-related problems among people who were non-drinkers at baseline. Due to methodological differences, data from the two studies reporting on tobacco interventions among adolescents were not combined.
Conclusions: Based on findings largely from tertiary students, web interventions targeting alcohol-related problems have an effect about equivalent to brief in-person interventions, but with the advantage that they can be delivered to a far larger proportion of the target population. Web-based interventions to prevent the development of alcohol-related problems in those who do not currently drink appear to have minimal impact. There are currently insufficient data to assess the effectiveness of web-based interventions for tobacco use by adolescents.
The prevalence of mental health disorders is greatest in those aged under 25 years, with 26% having a disorder, including 13% with a substance use disorder.1 In addition, a substantial proportion of young adults use substances in a risky manner without reaching the formal diagnostic criteria for a disorder — more than 40% of 18–19-year-olds use alcohol in a risky or high-risk fashion every month.2 Despite the prevalence of these problems, the use of treatment services is limited, with only 35% of those with a diagnosis in the previous 12 months having accessed mental health services for that disorder.1
Between 1998 and 2006–07, the proportion of Australians with access to the internet at home rose from 16% to 64%,2 and a further 25% reported having used the internet at other locations.2 The 15–24-years age group has the highest proportion of internet users. Across all ages, most users (58%–80%) report that they search the web for health information.3,4 Many adolescents think that the internet is a useful source of information on topics that are hard to discuss,5 and online health information is regarded as trustworthy and relevant by both sexes and across socioeconomic groups.6 Therefore, the internet provides an alternative vehicle for delivering health interventions. It may also provide a means of delivering services to those who are unable or unwilling to access conventional health services.
There is extensive evidence that brief interventions are an effective means of treating some substance use problems, especially for those who do not reach the criteria for dependence.7-10 Techniques from brief interventions, traditionally delivered face-to-face, can also be delivered electronically.11 However, the quality of electronic interventions remains in doubt. An assessment of 294 health behaviour change websites found that only 8.1% fulfilled the basic “5 A’s” (advise, assess, assist, anticipatory guidance, arrange follow-up) thought to be required to initiate change in behaviour.12,13 None of the alcohol-related websites met all five guidelines.13 In addition, many of the early publications on web-based interventions were descriptive or reported on feasibility or on studies that were methodologically weak.14,15 A recent review by Bewick and colleagues on web-based interventions for alcohol consumption16 found that only one intervention had used a randomised design that included a control of the highest standard according to their criteria.17
Further studies have been published since May 2006 (the cut-off date for Bewick et al’s review16), and there have been developments in web delivery across a range of common mental health problems. In addition, interventions and content designed for adults may not be appropriate or equally effective in younger groups. Given the importance of this age group in the initiation of substance use, their high level of use of internet resources, and willingness to access online health information, we aimed to review the current literature on interventions designed to target adolescents and young adults.
We searched MEDLINE, PsycINFO and Current Contents in February 2009 using the search strategy: “Substance abuse or alcohol or drug or tobacco and ([internet or web] and [intervention or RCT])”. The search was limited to English-language results and yielded 391 reports. Titles and abstracts were checked, and potentially eligible papers accessed for final assessment by one of us (R J T). References in eligible articles and reviews, as well as key journal sites (eg, Journal of Medical Internet Research), were used to identify further potentially relevant reports.
The inclusion criteria for the review were that studies had to use a randomised design to compare a web-based intervention with at least a no-treatment control. The eligible age groups were adolescents (typically targeted via interventions delivered through schools) or young adults (ie, specifically targeting tertiary students or other people aged 25 years or less). Outcomes had to include a measure of consumption of the target substance, not just change in attitude.
From a population perspective, for prevention programs to be useful in reducing health problems, they must not only have demonstrated efficacy and effectiveness but also the potential to be scalable, as well as having available resources to allow the program to be widely replicated while maintaining treatment fidelity.18 We therefore restricted our focus to include only interventions that used fully automated treatment programs and excluded those that required additional elements, such as in-person motivational interviews, due to the difficulty of delivering these at a population level. Interventions delivered via a stand-alone computer or CD-ROM were similarly excluded.
To avoid overestimating the magnitude of effects in studies with repeated measures, effect sizes (d) were calculated as between-group differences at follow-up divided by the pooled standard deviation from the baseline data.19 Where baseline data were unavailable, pooled standard deviations from the outcome results were used. In studies reporting medians and ranges, we estimated means and standard deviations using the method of Hozo and colleagues.20
Data were combined using Meta-Analysis software, version 5.3 (Ralf Schwarzer, Berlin, Germany), employing a random effects model. Where outcomes were significantly heterogeneous, the cluster analysis software incorporated in the same program was used to identify potential outliers.
The search strategy yielded 13 studies involving tertiary students and one with young employed adults (Box 1). All 14 studies targeted alcohol consumption, with one study also addressing other types of health behaviour (physical inactivity, low fruit and vegetable intake17). One study provided outcome data as change scores subdivided by sex;32 we did not include these in the meta-analysis phase.
The overall effect size for the outcomes (summarised in Box 2) was d = − 0.22 (SE, 0.06; 95% CI, − 0.34 to − 0.10), but with significant heterogeneity (Q = 249.03, df = 55; P < 0.00001). In light of the range of different types of outcome measures used and the different levels of baseline alcohol exposure, potential sources of heterogeneity were sought by separately analysing the results for three key outcome measures (average quantity of alcohol, frequency of heavy or binge-drinking events, and alcohol-related social problems) and for those studies that separately reported outcomes for people who were non-drinkers at baseline.
Ten studies reported on the quantity of alcohol consumed (Box 2; variables 8, 12, 18, 23, 28, 36, 42, 48, 50, 53). Overall, those who received the interventions had a lower level of alcohol consumption at follow-up than those in the control groups, with a mean difference of d = − 0.12 (SE, 0.05; 95% CI, − 0.22 to − 0.02), with the effect being homogeneous (Q = 7.36, df = 9; P = 0.600).
Seven studies reported on the frequency of heavy or binge drinking (Box 2; variables 1, 10, 20, 24, 29, 37, 44; variable 43 was an early outcome in the same study as 44 and was not included in our analysis). Young adults receiving the interventions had a lower frequency of heavy or binge drinking than controls (d = − 0.35; SE, 0.15; 95% CI, − 0.64 to − 0.06), although this measure still showed significant heterogeneity (Q = 29.74, df = 6; P = 0.00004). An inspection of a cluster plot showed the effects for the studies by Bersamin et al21 (d = − 0.99) and Kypri et al27 (d = − 0.80) to be outliers.
Six studies reported on alcohol-related social consequences, as assessed by measures such as the Rutgers Alcohol Problem Index36 or Alcohol Problems Scale37 (Box 2; variables 3, 21, 30, 38, 52, 55), with the overall effect size being d = − 0.57 (SE, 0.21; 95% CI, − 0.98 to − 0.15). These data showed significant heterogeneity (Q = 24.20, df = 5; P = 0.0002). A cluster analysis identified three clusters, with the smallest (variable 21) and largest (variable 38) effect sizes separate from the remaining variables.
Two studies reported outcomes separately for those who were non-drinkers at baseline.21,24 The overall effect for this subpopulation was d = − 0.001 (SE, 0.06; 95% CI, − 0.12 to 0.12), an effect not significantly different from zero (Z = − 0.016; P = 0.499). Our analysis did not include 751 people in Croom et al’s study who were non-drinkers at baseline and follow-up, as disaggregated data were not available for these people. The overall effect size for all participants in the study, regardless of baseline alcohol status, was d =− 0.02.24
Our systematic search yielded two eligible studies targeting adolescents (Box 3). Both studies targeted adolescent smoking: one randomly assigned schools to intervention or control groups,38 and the second used individual randomisation.39 Due to these methodological differences and differences in the target populations (all students v only smokers), these data were not combined for meta-analysis.
Our search strategy revealed two categories of web-based intervention for problematic substance use in young people. Web-based interventions targeting alcohol use by young adults, predominantly tertiary students, appear to be effective for alcohol problems in current drinkers, but there is insufficient evidence to support their use in preventing the development of alcohol-related problems among those who do not drink alcohol. The second category was web-based interventions addressing smoking cessation in adolescents and school children; however, there are currently insufficient data to assess the utility of such interventions in this group.
Brief in-person interventions for non-treatment-seeking individuals are an effective means of reducing alcohol consumption, with effect sizes in the range of 0.14 to 0.67 (with positive values showing better outcomes).8 Brief interventions have also been found to be effective among adolescents, with an overall effect size of 0.275.40 Thus, the effects reported here for web-based interventions are consistent with the magnitude of effects obtained from in-person interventions. There is preliminary evidence that computer-based interventions are cost-effective (eg, compared with cognitive behaviour therapy for depression).41 There is also evidence to support the scalability of web-based interventions,42 a potential benefit compared with in-person interventions. Nevertheless, it should be noted that some of the interventions reviewed here21,24 are more time-consuming than typical brief in-person interventions, which are designed to be delivered in less than four sessions.8 Cost–benefit analyses comparing web-based with in-person interventions are required.
An important caveat must be noted. In general, the web-based interventions in this review reported short-term outcomes, usually 3 months or less, and these may not represent a meaningful change in behaviour. Only four studies reported outcomes for 4 months or longer.27,29,33,34 The outcomes for these studies (Box 2) show effect sizes ranging from 0.12 to − 2.73, suggesting that persistent change in behaviour is possible with web-based interventions. Nevertheless, confirmation of long-term impacts is needed.
Considerable heterogeneity was noted in many of the measures and across the studies. Inspection of the interventions and measures did not reveal any clear explanation for this. For example, the measure of heavy drinking in Kypri et al’s 2008 study27 that was identified as an outlier in our sub-analysis was the same measure used in their 2004 study,29 with both studies using similar interventions and similar target populations (tertiary students recruited in a health care setting). Additionally, although the target populations across all but one of the alcohol interventions26 were tertiary students, the study samples included subgroups of this population ranging from non-drinkers at baseline21 to students who had been referred for counselling after breaches of university alcohol or drug policies.25 Furthermore, both Bersamin et al21 and Croom et al24 noted that the web interventions were ineffective with young people who were non-drinkers at baseline — findings confirmed by the combined data reported here. Although all the studies used randomised designs, there was considerable diversity in the intensity of the interventions delivered, ranging from an online course24 to a 15-minute assessment and feedback session.25 Finally, not all studies provided a control group with an intervention of similar intensity as the experimental intervention. Therefore, it is unsurprising that there were overall differences in the effectiveness of interventions or the effect sizes.
A previous review of brief interventions to reduce smoking in adolescents noted an effect size of 0.037.40 This included interventions delivered at the population level43 and for existing smokers.44 Our review also identified one population-level approach and one targeting existing smokers. The population-level intervention recruited participants in both Australia and the United States. The Australian arm of the study reported a lower prevalence of smoking in the intervention group compared with the control group and a lower level of initiation of smoking by non-smokers.38 However, these findings were not replicated in the US sample, and the authors concluded that web-based interventions are likely to have little practical impact on the level of smoking by adolescents.38
Another review45 estimated that the overall quit rate for stand-alone computer-based smoking-cessation interventions for adolescents was 13%.46-48 Patten et al39 reported that none of the adolescents in their study attained complete abstinence, but the 30-day abstinence level at 36 weeks was 13% for brief office-based intervention, compared with 6% for the web-based intervention. A potential reason for the poor performance of the web-based approach is that participants accessed the website from home (on a mean of 6.8 days over 24 weeks), with 86% visiting the site at least once, but less than a third visiting the site weekly after the third week. In contrast, delivery in a school setting may encourage greater compliance (eg, 77% completing all sessions,46 90% completing two of three sessions47). Therefore, motivation of adolescents and compliance may be key factors in the effectiveness of this form of intervention.
Our review has some potential limitations. The techniques and process of meta-analysis have received detailed and at times trenchant critiques,49,50 with key concerns being the quality of the studies included (“garbage in, garbage out”), combining different measures or interventions (“apples and oranges”), including multiple measures from studies (“inflated Ns”) and publication bias (“file drawer problem”).51,52 Nevertheless, the systematic assembly of data fulfilling clear criteria has come to the forefront in summarising scientific evidence.
Further, we did not identify any studies on the effectiveness of web-based interventions with adolescent drinkers. Given that 70% of 17-year-old students report having used alcohol in the past month and more than 40% report using it in a risky fashion,53 this would appear to be an important target for future research, even though it is difficult to obtain the necessary ethics approval to conduct research in this age group, especially when investigating interventions to convert behaviour.
Adolescence and young adulthood are the key period for initiation of substance use and the development of substance use disorders. Although the lifetime prevalence of licit drug use has remained stable, the age of initiation has fallen in more recent birth cohorts,54 with a concomitant increase in the risk of developing disorders in later life.55 Thus, there is an imperative to design and deliver interventions that address substance use by adolescents and young adults. Web-based interventions have the potential to provide interventions at a population level, with initial findings supporting their effectiveness in reducing problematic alcohol use in tertiary students and young adults.
1 Interventions for problematic substance use in young adults
2 Outcome measures and effect sizes of interventions for problematic substance use in young adults
3 Interventions for problematic substance use in adolescents