The health care and societal costs of depression in Australia are substantial.1 Most people with this high prevalence disorder seek care in primary care settings, from general practitioners. For example, 78% of people with affective disorders in Australia seek care either solely from a GP or from a GP in addition to other health care providers.2 The pattern is similar for anxiety disorders. Primary mental health care in Australia has therefore been targeted for improvement through research, innovation and service-funding streams. For example, specific incentive payments for GPs to deliver Focused Psychological Strategies are available to those GPs who have taken part in approved training.3
Economic evaluation of health interventions can help inform health policy.4,5 Innovations in primary mental health care should undergo proper economic evaluation to determine which approaches offer value for money. This project is an exploratory economic analysis of two mental health interventions in primary care — the Panic Online study and the Primary Care Evidence Based Psychological Interventions (PEP) study.
The Panic Online study is a randomised trial of an Internet-based cognitive behavioural therapy (CBT), with the support of either a psychologist or GP, for the treatment of panic disorder within a primary care setting. The PEP study is a cluster randomised trial in which GPs are allocated to training in CBT strategies using the SPHERE CBT training package, or to a waiting list control. This training program is accredited as a Level 2 Mental Health Skills Training course by the General Practice Mental Health Standards Collaboration.6
Both studies are funded by the beyondblue Victorian Centre of Excellence in Depression and Related Disorders, a collaborative initiative of beyondblue: the national depression initiative and the Victorian Department of Human Services. As both studies are in progress, patient level data are not available to inform the economic analyses. However, insights into the potential cost-effectiveness of each intervention (considering both costs and benefits) are possible through the use of economic modelling techniques.
Our aim was to provide insights into the economic credentials of both interventions, as well as to comment on their sustainability, both financial and in terms of the broader issues of workforce requirements and consumer acceptance.
As the two projects had different information available, they were evaluated with different methodology. The analyses were perfomed using the year 2004 as the reference year, from the perspective of the health sector (divided into costs to government and to individuals).
To facilitate comparison with existing studies, we evaluated Panic Online with the economic modelling methodology developed for the Australian ACE-MH project (Assessing Cost-Effectiveness in Mental Health).5 We used specifically the ACE-MH modelling methods for panic disorder (which included face-to-face CBT), which allows cost per disability-adjusted life-year (DALY) to be calculated.7
We sought to determine the cost-effectiveness of the intervention, assisted by either a psychologist or a GP, for people with panic disorder in Australia receiving care in the general practice setting, compared with current practice (defined as people receiving either non-evidence-based medicine in general practice or evidence-based GP care specified to comprise primarily CBT, which was assumed to be face-to-face).
To cost Panic Online, we specified key activities as follows:
Panic Online assisted by a psychologist is a 12-week intervention comprising an average of 12 sessions of 45 minutes with a public sector psychologist responding to the participant’s emails, one consultation with a GP (required to introduce eligible people with panic disorder to the intervention), and computer and software to undertake the online intervention.
Panic Online assisted by a GP is also a 12-week intervention, comprising an average of six consultations with a GP (under the Focused Psychological Strategies), and computer and software to undertake the online intervention.
We determined the likely effectiveness of Internet-based psychological therapy from a review of the existing literature, concluding (similarly to other reviews8) that computerised CBT (with delivery methods that included palmtop computer and stand-alone computers) has equal effectiveness to traditional face-to-face CBT. We estimated DALYs averted using the assumptions of the ACE-MH study of panic disorder.7
We obtained the 12-month prevalence of panic disorder (with or without agoraphobia) from the National Survey of Mental Health and Wellbeing,9 and applied it to the Australian population in 2004.10 We then estimated the number of people with panic disorder who consulted a GP, differentiated according to whether or not they received evidence-based medicine (EBM), based on published estimates.11 As EBM includes medication and CBT, we determined the proportion receiving CBT (in the EBM category) from the estimates used in the ACE-MH study of treatment of panic disorder.6 Lastly, we used estimates from the Australian Bureau of Statistics for the proportion of the population who have access to a computer and the Internet.12
We determined the likely costs of the intervention from the Panic Online study itself, and extrapolated these to the population of Australia. We included the professional costs associated with delivering the intervention, but excluded computer acquisition costs (as computers were assumed to be already owned), the cost of the Panic Online software (currently available free of charge), start-up and development costs of the software (largely a sunk cost), and time costs. We also excluded the potential effects of the intervention on other health services (eg, emergency department visits, hospitalisations, alternative therapies), as we had no evidence on which to base any cost offsets or inclusions.
The cost offsets included were the costs of non-EBM received in general practice (defined as an average of 2.8 consultations with a GP, based on the estimate from the ACE-MH study)6 and the cost savings associated with shifting people from EBM CBT (defined as traditional face-to-face CBT comprising 12 sessions with a psychologist) to the Panic Online interventions. Published unit costs were preferentially used (eg, Medicare Benefits Schedule [MBS] costs for GP contacts) (further information is available from the authors).
The robustness of the results was tested by probabilistic sensitivity analysis.13 In this analysis, multiple parameters of the model are varied simultaneously, allowing 90% uncertainty intervals to be determined around all point estimates (further information is available from the authors).
We assessed PEP using a threshold analysis. This analysis can assist decisions about resource allocation as it identifies critical values of parameters that influence a decision to invest. For example, a decision-maker may specify an acceptable level of investment or cost-effectiveness ratio. The analyst then determines from available information which combinations of parameter estimates cause the threshold to be exceeded or achieved.13 We chose this type of analysis as there is insufficient published evidence of the benefit of PEP on which to base an economic evaluation. Our review of existing studies of GPs delivering CBT found that they either showed little or no benefit over usual care, or that the intervention itself was delivered by other providers.14,15
For the threshold analysis, we determined the likely costs of implementing a PEP intervention on a national basis and the effect size required to achieve an acceptable cost-effectiveness ratio. We defined this as $50 000 per DALY averted, based on the ratio considered acceptable in the ACE-MH project for the treatment of depression by CBT.16 Cost-effectiveness ratios below this figure were considered to provide good value for money.
We obtained the 12-month prevalence of depression from the National Survey of Mental Health and Wellbeing9 and applied it to the Australian population in 2004.10 We then used the estimate of the proportion of people with depression who seek care from a GP and do not receive EBM previously derived from the National Survey3 (as PEP is likely to be of benefit to people currently not receiving appropriate care).
We determined the costs of expanding a PEP-type intervention nationally, comprising:
Training costs estimated directly from the PEP study, including costs of printed materials, trainers, venue hire, and catering, but not GP recruitment. We assumed 5% of GPs would undergo this training nationally.
Costs of GPs delivering the intervention under the Focused Psychological Strategies program.3 We assumed GPs would have six consultations with patients with depression. All GP unit costs were sourced from the MBS and differentiated according to costs to governments and costs to patients.
To determine the net costs of the intervention, we calculated the costs of non-EBM using the same assumptions as Panic Online, and subtracted these from the total costs of the intervention.
We then determined the number of DALYs that had to be averted to achieve a similar cost-effectiveness ratio to that achieved in the ACE-MH study of CBT in depression, and further determined the effect size required for PEP to show acceptable cost-effectiveness ($50 000 per DALY averted).
The mean cost of the Panic Online intervention was found to be $3.8 million (90% uncertainty interval [UI], $2.3–$5.3 million) when assisted by a psychologist, and $2.8 million (90% UI, $1.7–$3.9 million) when assisted by a GP. When the psychologist was in the public sector, there was a total saving to patients of about $15 000, because of the reduction in non-EBM that carried greater patient out-of-pocket costs than the intervention. When Panic Online was assisted by a GP, 17% of costs were patient out-of-pocket costs.
The average health benefit associated with Panic Online was about 870 DALYs (90% UI, 540–1200 DALYs). The incremental cost-effectiveness ratio when assisted by a psychologist was $4300 per DALY averted (90% UI, $3500–$5400), and when assisted by a GP was $3200 per DALY averted (90% UI, $2700–$3900). The uncertainty analysis showed that 100% of all iterations fell below $10 000 per DALY for both variants of the intervention.
The total costs of providing CBT training to 5% of Australian GPs was found to be about $1 million. The total cost of the intervention (including therapy with the GP) was $44 million ($6 million are out-of-pocket costs to patients, and the rest are costs to government). The net cost (total costs minus cost offsets associated with non-EBM service provision) was $35 million ($5 million are out-of-pocket costs to patients). Threshold analysis shows that quite a modest effect size of benefit (in the vicinity of 0.1) would be sufficient to incur an acceptable cost-effectiveness ratio.
Our study shows that Panic Online has the potential to be a very cost-effective intervention, while the training of GPs to provide CBT can also be quite cost-effective with moderate effect sizes of benefit. However, clear evidence of benefit is required for both interventions, particularly PEP-type interventions. The PEP findings pose a particular research challenge, as randomised trials powerful enough to reliably detect the modest effect size compatible with cost-effectiveness would need to be very large.
Our methods had limitations, including key assumptions made in the modelling. For example, we assumed that Panic Online is available to all eligible people and is perfectly adhered to, and that the number of GPs trained in PEP can meet the needs of the eligible population. Although many of these assumptions require verification, the modelling nevertheless provides an initial indication of the likely cost-effectiveness.
However, even if both interventions demonstrate definitive value for money, their sustainability must be addressed before widespread implementation. An available and trained workforce is needed to support and undertake both interventions. GPs may require training to gain competence in using Internet-based mental health treatments.17 In addition, the interventions must be acceptable to eligible consumers. Acceptability may vary according to patients’ cultural backgrounds and socioeconomic status (eg, CBT may be more acceptable to people of higher socioeconomic status). Not all consumers have confidence in the ability of GPs to deliver effective mental health treatment.18 Importantly, Panic Online may be able to fill an important gap in service delivery to people living in rural or remote areas, as the professional support associated with the intervention can be provided from a distance. Similarly for PEP, rural and remote populations are likely to have better access to GPs than to trained mental health professionals.
Finally, financial sustainability is particularly important. Our modelling shows that PEP and Panic Online incur greater costs than current practice, raising the issue of financing. Most of the costs accrue to governments, with the fee-for-service Medicare system financing GP services, but not training. The costs of public sector psychologists supporting Panic Online are also not currently financed (most public sector psychologists are employed in community mental health centres, which are specialist treatment facilities largely servicing people with psychotic problems or severe mental illnesses). While private psychologists might provide the support, this reduces the cost-effectiveness of the interventions, as amply demonstrated in the ACE-MH studies,7,16 and potentially affects equity because of the increase in patient out-of-pocket costs. Equity is currently the biggest problem in the universal provision of CBT for depression and anxiety in Australia. Patient out-of-pocket costs are probably a key reason why few people with depression or anxiety currently receive CBT, despite considerable evidence for its cost-effectiveness.
When GPs deliver these interventions, our modelling assumes that appropriate delivery of the service requires extra consultations. Whether this is the case, or whether GPs will replace services they currently provide with the newer services, remains to be seen. If substitution occurs, then the costs of both interventions are likely to be less, although, commensurately, any benefits of the dropped services need to be factored into the cost-effectiveness analyses. There is currently no evidence on which to assess these possibilities. The facilitators and inhibitors to the sustainability of Panic Online and PEP are summarised in the Box.
In conclusion, economic analyses such as those reported here, or even the ACE-MH series, go only a small way towards ensuring that cost-effective services are provided, as issues of sustainability must also be considered. However, these analyses are an important information source for policy makers and health care providers to help determine resource allocation and provide strategies to encourage service delivery.
Sustainability of Panic Online and Primary Care Evidence Based Psychological Interventions (PEP)
Facilitators
Most people with depression or anxiety seek help in the general practice setting.
Out-of-pocket costs associated with both Panic Online and PEP are relatively small when the interventions are delivered by general practitioners or public sector psychologists.
Remuneration of GPs trained in focused psychological strategies is currently financed under Medicare.
Inhibitors
Effectiveness of both interventions remains to be determined.
The workforce is probably inadequate (in terms of numbers) to meet demand (public sector psychologists and GPs willing to undergo training).
Acceptability to various key stakeholders, including consumers, policy makers and health professionals, may vary.
- Cathrine Mihalopoulos
- Sophy Ting-Fang Shih
- Litza Kiropoulos
- Grant Blashki
- Graham Meadows
- Jane Gunn
- 1 Program Evaluation Unit, Melbourne University Health Economics Group, Melbourne, VIC.
- 2 Department of General Practice, Monash University, Melbourne, VIC.
- 3 Department of Psychological Medicine, Monash University, Melbourne, VIC.
- 4 Department of General Practice, University of Melbourne,VIC.
We acknowledge the contribution to this work of our esteemed colleague and friend, the late Professor Jeffrey Richards, the advice of Associate Professor Rob Carter (University of Melbourne) in early discussions of this project, and the assistance of Associate Professor Theo Vos (University of Queensland) and Dr Michelle Haby (Department of Human Services) in making available the ACE-MH data referred to in this study. The study was funded by a research grant from the Australian Primary Health Care Research Institute of the Australian National University. The Panic Online program was initially funded with the assistance of an Australian Rotary Research Health grant.
Grant Blashki has received payment from Educational Health Solutions to train general practitioners using the SPHERE cognitive behavioural therapy training program.
- 1. Meadows G, Liaw T, Burgess P, et al. Australian general practice and the meeting of needs for mental health care. Soc Psychiatry Psychiatr Epidemiol 2001; 36: 595-603.
- 2. Blashki G, Hickie IB, Davenport TA. Providing psychological treatments in general practice: how will it work? Med J Aust 2003; 179: 23-25. <MJA full text>
- 3. Carter R, Stone C, Vos T, et al. Trial of program budgeting and marginal analysis (PBMA) to assist cancer control planning in Australia. Melbourne: Program Evaluation Unit, University of Melbourne (formerly Centre for Health Program Evaluation), 2000. Research Report 19.
- 4. Haby MM, Carter R, Mihalopoulos C, et al. Assessing Cost-Effectiveness — Mental Health: introduction to the study and methods. Aust N Z J Psychiatry 2004; 38: 569-578.
- 5. Royal Australian College of General Practitioners. General Practice Mental Health Standards Collaboration [website]. Available at: http://www.racgp.org.au/folder.asp?id=650 (accessed Sep 2005).
- 6. Heuzenroeder L, Donnelly M, Haby MM, et al. Cost-effectiveness of psychological and pharmacological interventions for generalized anxiety disorder and panic disorder. Aust N Z J Psychiatry 2004; 38: 602-612.
- 7. Kaltenthaler E, Shackley P, Stevens K, et al. A systematic review and economic evaluation of computerised cognitive behaviour therapy for depression and anxiety. Health Technol Assess 2002; 6(22): 1-89.
- 8. Australian Bureau of Statistics. Mental health and well being: profile of adults, Australia, 1997. Canberra: ABS, 1998. (Cat. No. 4326.0.)
- 9. Australian Bureau of Statistics. Population by age and sex, Australian states and territories. Canberra: ABS, 2004. (Cat. No. 3201.0.)
- 10. Andrews G, Carter GL. What people say about their general practitioners’ treatment of anxiety and depression [published erratum appears in Med J Aust 2001; 175: 560]. Med J Aust 2001; 175 Suppl: S48-S51.
- 11. Australian Bureau of Statistics. Household use of information technology. Canberra: ABS, 2004. (Cat. No. 8146.0.)
- 12. Drummond MF, O’Brien B, Stoddart GL, Torrance GW. Methods for the economic evaluation of health care programmes. 2nd ed. Oxford: Oxford Medical Publications, 1997.
- 13. King M, Davidson O, Taylor F, et al. Effectiveness of teaching general practitioners skills in brief cognitive behaviour therapy to treat patients with depression: randomised controlled trial. BMJ 2002; 324: 947-50.
- 14. Ward E, King M, Lloyd M, et al. Randomised controlled trial of non-directive counselling, cognitive-behaviour therapy, and usual general practitioner care for patients with depression. I: Clinical effectiveness. BMJ 2000; 321: 1383-1388.
- 15. Vos T, Corry J, Haby MM, et al. Cost-effectiveness of cognitive-behavioural therapy and drug interventions for major depression. Aust N Z J Psychiatry 2005; 39: 683-692.
- 16. Austin DW, Pier C, Mitchell J, et al. How comfortable are GPs with the use of electronic mental health resources? Presentation to the General Practice and Primary Health Care Research Conference; 2005, Adelaide.
- 17. Wirthlin Worldwide Australasia. National mental health benchmark. Sydney: Royal Australian College of General Practitioners, 2001.
Abstract
We evaluated an Internet-based psychological intervention supported by either general practitioners or psychologists (Panic Online), and a Primary-care Evidence-based Psychological-interventions (PEP) strategy which involves training GPs to deliver specific psychological interventions.
Economic modelling suggests that Panic Online is cost-effective when supported by either GPs or psychologists.
Threshold analysis of the psychological training of GPs suggests that a modest effect size for clinical benefit would be sufficient to provide an acceptable cost-effectiveness ratio.
The sustainability of these approaches depends on a range of factors, including funding, workforce availability, and acceptability to consumers and health care providers.