MJA
MJA InSight
MJA Careers
Research

Over 150 potentially low-value health care practices: an Australian study

Adam G Elshaug, Amber M Watt, Linda Mundy and Cameron D Willis
Med J Aust 2012; 197 (10): 556-560.
doi:
10.5694/mja12.11083
Abstract

Objective: To develop and apply a novel method for scanning a range of sources to identify existing health care services (excluding pharmaceuticals) that have questionable benefit, and produce a list of services that warrant further investigation.

Design and setting: A multiplatform approach to identifying services listed on the Australian Medicare Benefits Schedule (MBS; fee-for-service) that comprised: (i) a broad search of peer-reviewed literature on the PubMed search platform; (ii) a targeted analysis of databases such as the Cochrane Library and National Institute for Health and Clinical Excellence (NICE) “do not do” recommendations; and (iii) opportunistic sampling, drawing on our previous and ongoing work in this area, and including nominations from clinical and non-clinical stakeholder groups.

Main outcome measures: Non-pharmaceutical, MBS-listed health care services that were flagged as potentially unsafe, ineffective or otherwise inappropriately applied.

Results: A total of 5209 articles were screened for eligibility, resulting in 156 potentially ineffective and/or unsafe services being identified for consideration. The list includes examples where practice optimisation (ie, assessing relative value of a service against comparators) might be required.

Conclusion: The list of health care services produced provides a launchpad for expert clinical detailing. Exploring the dimensions of how, and under what circumstances, the appropriateness of certain services has fallen into question, will allow prioritisation within health technology reassessment initiatives.

Internationally, there is a groundswell of activity seeking to identify and reduce the use of health care interventions that deliver marginal benefit, be it through overuse, misuse or waste. England’s National Institute for Health and Clinical Excellence (NICE) began this work in 2005,1 and most recently, the Choosing Wisely campaign led by physician groups in the United States is attracting worldwide attention.2 Other countries, and individual jurisdictions within countries, are also considering the best approaches to reducing the use of low-value health care practices. One problem has been fairness and transparency in identifying and prioritising suboptimal health care practices for consideration. Here, we report on Australian activities; in particular, on a collaborative project aiming to identify existing health care interventions that might warrant analysis from a health technology reassessment and practice optimisation perspective.

Australia’s Medicare Benefits Schedule (MBS) — a cornerstone of the Australian universal health care system — lists the rebates that are payable to patients for private medical services provided on a fee-for-service basis, and describes these services. In 2012, the MBS contains almost 6000 items (not including pharmaceuticals); only around 3% of these (accounting for about 1% of total MBS expenditure) have been formally assessed against contemporary evidence of safety, effectiveness and cost-effectiveness.3

In the 2009–10 Budget, the Australian Government announced funding over 2 years for a range of projects to develop and implement a new evidence-based MBS Quality Framework — subsequently named the Comprehensive Management Framework for the MBS (CMF)3 — for managing the MBS into the future. The CMF set out to establish new listing, fee-setting and review mechanisms to ensure that prospective and already listed items: (i) meet agreed standards for effectiveness and safety; (ii) are likely to lead to improved health outcomes for patients; and (iii) represent value for money. The CMF is consistent with international efforts to maximise health outcomes and efficiency. CMF reform sought to improve transparency and provide a stronger evidence base for services listed on the MBS. Box 1 lists the key elements and principles of the framework.

Before the initial Quality Framework was introduced on 1 January 2010, there was no formal process for evaluating existing MBS items that had not been assessed by the Medical Services Advisory Committee (MSAC). Without formalised reviews or a built-in method to update MBS items as clinical practice evolves, items on the MBS have become outdated. Thus, patients may receive treatments that have not been proven to be clinically effective, and financial incentives within the MBS may not always be aligned with best clinical practice.

A universal challenge in this area is to establish a systematic and transparent strategy to identify potential “low-value” clinical services for review.4-7 Traditional literature search strategies for “unsafe or ineffective care” offer limited utility in isolation.4 In this report, we describe one CMF project that used a range of information sources to identify items for review through an expanded “environmental scanning” approach. The 2-year CMF timeline dictated an expedited process. This work was developed and undertaken over 8 months in the financial year 2010–11.

Methods

A multiplatform approach for searching for and identifying potential medical services for review was developed. This comprised the following three key elements.

Peer-reviewed literature search: a detailed search strategy was applied to the PubMed search platform (Box 2).

Targeted database search: these were conducted of the Cochrane Library, National Institute for Health and Clinical Excellence (NICE) “do not do” recommendations,8 BlueCross BlueShield Association Technology Evaluation Center assessments9 and the Canadian Agency for Drugs and Technologies in Health (CADTH) health technology assessments.10

Opportunistic sampling: drawing on our experience (from a previous and ongoing program of work in this area) and links with clinical and non-clinical stakeholder groups, both within Australia and internationally, from whom nominations (with evidence) for candidate services were collected.

Each of these three elements contributed to the final sample that was screened for potential candidate services for reassessment.

Peer-reviewed literature search

We used a series of keyword and medical subject heading (MeSH) strings (Box 2) across the bibliographic databases to identify potential candidate services for prioritisation. Exclusion criteria were applied to screens of titles, abstracts and full texts of retrieved articles (Box 3), with further limits and filters applied as shown in Box 4. Subsets of results from Filters 2A (Level I evidence11), 2B (Level II evidence11) and 2C (remaining literature search) were selected based on their date of publication, with the most recently published studies (2000–2010) forming the subsets (Box 4). Additionally, we undertook relative oversampling from Filter 2A in consultation with representatives from the Department of Health and Ageing, based on the assumption that the higher level of evidence represented in the results would provide greater yield for the final list of services.

Targeted database search

All reports from the Cochrane Library and BlueCross BlueShield Association Technology Evaluation Center assessments were considered, after standard filters (humans, English language, not pharmaceuticals) were applied. All available reports from the NICE “do not do” recommendations and CADTH health technology assessments were considered for inclusion on the master list. These databases offer targeted and specific findings. NICE, for example, teamed with the Cochrane Collaboration to focus their search within Cochrane Reviews and guidelines.1 This complemented our broader method, but when mapped against existing MBS items, numerous services were filtered out as not relevant to the Australian funding context.

Opportunistic sampling

All reports identified by opportunistic sampling were included on the master list before inclusion and exclusion criteria were applied.

Inclusion and exclusion criteria

All reports retrieved from the targeted database searches and opportunistic sampling were placed on a master list, alongside results from the peer-reviewed literature search.

After the exclusion criteria (Box 3) were applied to titles, the abstract or executive summary of each included study was obtained and screened. Studies that reported the value of a medical service as inferior or similar to placebo were included, while studies that reported no difference between a service and an active comparator were excluded (because identifying the inferior service from such studies would likely require additional clinical expertise). Articles were screened by the authors of this report, with disagreements resolved through open discussion.

Medical services identified through opportunistic sampling (where evidence supported inclusion) were afforded prioritised inclusion, given they were nominated by clinical and other stakeholders and evidence existed in support. Services described in articles or reports that met the inclusion criteria were mapped to MBS items, with any services not covered by the MBS excluded from further analysis. Pharmaceuticals do not fall under the purview of the MBS and were excluded.

Eligible services were then tracked across search methods to triangulate medical service identification. This enabled us to identify services that appeared across the multiple elements of the search strategy. Triangulation may have value in prioritising further work, along with other criteria that we developed previously.4 This entire process was completed over 8 months by a two-member full-time-equivalent workforce.

Results

A total of 5209 articles were screened for eligibility, resulting in 156 potentially ineffective or unsafe services being flagged for consideration (Appendix). The list includes examples where practice optimisation (ie, comparing the relative value of one treatment option against others) might be required. The Appendix details all the services we identified, including any citations that drew attention to their status as potential candidates, and an extract from the article highlighting key issues relevant to the service. Box 5 lists the 13 services identified by more than one search method; three services were identified by all three methods. While this serves to highlight the crossover points of the search strategies we used, there are other factors related to the candidate services that may influence their relative priority in any assessment process (eg, predominant safety concerns, strong evidence, high volume, cost-effective alternative, etc).4

Discussion

In this project, we sought to develop and implement a systematic, evidence-based and transparent process for identifying potentially low-value services in health care. We present this list of candidate services for analysis and debate within and between clinical, research, patient and policy stakeholder communities. Services were identified through a novel search strategy and, although created for and mapped against Australia’s MBS, they offer insights for any health care system considering a health technology reassessment agenda. The specificity of services is open for critique, and we expect that context-specific clinical detailing will exclude some services from consideration and/or refine the questions that have been raised within the literature about their uses.

The process we describe in this report has a number of limitations, primarily related to the short time frame imposed on it. Sampling from the broad literature searches based on date of publication is likely to identify technologies or services for which recent evidence may suggest a level of ineffectiveness, and therefore risks missing those whose safety, effectiveness or efficacy has not recently come into question. In addition, time and resource constraints also limited the number of articles retrieved through each filter that could be reasonably evaluated. Thus, only a fraction of potentially relevant articles were included. However, combining these searches with broad reviews of key assessment agencies (CADTH, NICE, etc), as well as obtaining expert clinical input, helps to moderate this potential bias and captures a breadth of medical services that are of key interest across clinical settings and stakeholder groups. Importantly, our process was not intended to be exhaustive or to act as a tool for prioritisation; rather, it aimed to provide a transparent, evidence-based approach to identifying potentially ineffective services. Further testing and refinement of search terms, inclusion and exclusion criteria and database sources is likely to yield important insights into how this process may be improved and tailored to suit specific needs.

Our analysis has highlighted some of the tensions that exist between the paradigm of health technology assessment and the nature of guided service reimbursement, including fee-for-service. Health technology assessment and other clinical assessments of health services are, by nature, geared towards examining services and technologies in very specific populations and for very specific indications. This can be at odds with the broader nature of schedule or service item descriptors. Our work has confirmed that services that are ineffective and/or unsafe across the entire patient population to which they are applied are probably quite rare. Most often, a service shows differential effectiveness profiles, dependent on the characteristics of the population in whom it is applied. Research must indicate the populations most likely to benefit from or be harmed by services, thus allowing the development of effective policies for refining the indications for coverage and minimising use outside these indications. How this is achieved in various systems will differ: fee-for-service systems might require tighter clinical item and patient descriptors and fee refinements, whereas program budget, bundled or capitated systems can introduce incentives for optimal use of services that offer the best patient outcomes.

For groups pursuing a health technology reassessment agenda, the next steps in the process requires further prioritisation of candidate services to a shortlist of those that may go on to formal review. Numerous methods have been proposed for this, each being somewhat context-specific.4-7 The assessment type that offers the greatest efficiency needs to be decided on. For example, initial rapid reviews as opposed to full health technology assessments may offer an efficient means of generating value of information to enhance the prioritisation process.

We also acknowledge that there are challenges in reducing or removing candidate services that are confirmed as having low value. Existing technologies or practices have complexities that do not beset those that are new or emerging, mostly because of their established status in medicine and society. These challenges have been discussed elsewhere.12-19

Limited resources mean that nations cannot escape having to make difficult health care choices. Identifying and reducing the use of low-value care is becoming a priority for an increasing number of jurisdictions. Each recognises that cost savings or cost-neutral changes can be made within existing health budgets by reducing the use of existing services that offer little or no benefit relative to the cost of their public subsidy. This would allow funding to be reallocated to more beneficial or cost-effective services, thus maximising health gain. We share this project as a step towards fulfilling that objective.

1 Key elements and principles of the Comprehensive Management Framework for the Medicare Benefits Schedule (MBS)

Elements

  • Introducing a time-limited listing for new MBS items that do not undergo an assessment through the Medical Services Advisory Committee

  • Requiring an evaluation process for all time-limited items at the end of the time-limited period and before items can be approved for long-term MBS listing, as well as evaluation of amendments made to MBS items

  • Strengthening arrangements for appropriately setting fees for new MBS services

  • Establishing systematic MBS monitoring and review processes to inform appropriate amendment or removal of existing MBS items

Principles

  • Processes will focus on using evidence to support best outcomes for patients

  • Processes will be timely, transparent and offer opportunity for stakeholder participation

  • Conflicts of interest will be addressed and actively managed

  • Continuous improvement techniques will be applied, and feedback mechanisms will be embedded in processes to foster a quality-improvement culture

Principles to guide MBS reviews

  • Reviews have a primary focus on improving health outcomes and the financial sustainability of the MBS, by considering potential:

    • patient safety risk

    • limited health benefit

    • inappropriate use (underuse or overuse) and/or

    • intentional misuse of MBS services

  • Reviews are evidence-based, fit-for-purpose and consider all relevant data sources

  • Reviews are conducted in consultation with key stakeholders including, but not limited to, the medical profession and consumers

  • Review topics are made public, with identified opportunities for public submissions and outcomes of reviews are published

  • Reviews are independent of government financing decisions and may result in recommendations representing costs or savings to the MBS, as appropriate, based on the evidence

  • Secondary investment strategies to facilitate evidence-based changes in clinical practice are considered

  • Review activity represents efficient use of government resources

Source: Medical Benefits Reviews Task Group. Development of a quality framework for the Medicare Benefits Schedule discussion paper.3

2 Search terms

String

Terms


String 1: safety

(unsaf*) OR (danger*) OR (adverse event) OR (poor outcome) OR (low quality) OR (poor quality) OR (harm*) OR (contraindicat*)

OR

String 2: effectiveness

(ineffect*) OR (supersede*) OR (irrelevant*) OR (outdated) OR (new evidence) OR (overuse*) OR (unproven) OR (inappropriat*) OR (equivoc*) OR (uncertain*) OR (obsolete) OR (inferiority) OR (superiority)

OR

String 3: policy solutions

(disinvest*) OR (coverage with evidence development) OR (CED) OR (access with evidence development) OR (AED) OR (access with evidence generation) OR (reallocat*) OR (resource release) OR (reinvest*)

NOT

String 4: pharma exclusion

(drug therapy [mh]) OR (drug industry [mh]) OR (pharmaceutical services [mh]) OR (pharmaceutical preparations [mh]) OR (pharmacogenetics [mh]) OR (pharmacoepidemiology [mh]) OR (technology, pharmaceutical [mh])


* = truncation character. AED = access with evidence development. CED = coverage with evidence development. [mh] = medical subject heading.

3 Exclusion criteria applied in screening articles

No.

Description


1

No technology/procedure/intervention identified in article (eg, study restricted to epidemiological data)

2

Pharmaceutical technology or codependent technology (ie, intervention dependent on pharmaceutical)*

3

Non-clinically defined intervention (eg, public health interventions)

4

Studies presenting favourable data with no comparator

5

Studies without clinically meaningful outcome measures (eg, quality-adjusted life-years, costs)

6

Studies that do not report data relating to safety, effectiveness, inferiority or superiority of intervention

7

Studies that report no difference between intervention and active comparative technology

8

Studies considering technologies excluded from this project’s remit, as defined by the Australian Government Department of Health and Ageing, given they were already undergoing (or slated to undergo) review. Specifically, those relating to: pulmonary artery catheterisation; colonoscopy; obesity surgery; ophthalmology

9

Technologies/procedures that cannot be mapped to existing Medicare Benefits Schedule item numbers

10

No abstract or summary statement to evaluate


* Not within the purview of this project. Excluded because identifying the inferior service from such studies would likely require additional clinical expertise beyond the scope of this project.

4 Search process

CADTH = Canadian Agency for Drugs and Technologies in Health. HTA = health technology assessment. NICE = National Institute for Health and Clinical Excellence. RCT = randomised controlled trial. TEC = Technology Evaluation Center. * Standard filters: humans, English, not pharmaceuticals. Final contribution to list (Appendix) after filtering and mapping evidence for relevance and applicability to existing Medicare Benefits Schedule items; the final list consists of health care services identified by more than one strategy (Box 5).

5 Services identified by more than one search method

No.

Broad service description


1

Testing of patients for factor V Leiden gene mutation

2

Arthroscopic surgery for knee osteoarthritis*

3

Testing for C-reactive protein*

4

Use of chest x-ray for acute coronary syndrome, preoperatively, or in diagnosing respiratory infections

5

Chlamydia screening

6

Exercise electrocardiogram (ECG) for angina

7

Imaging in cases of low back pain*

8

Liver function tests

9

Blood, urine or plasma testing in end-stage renal disease

10

Radical prostatectomy

11

Radiotherapy for patients with metastatic spinal cord disease

12

Routine dilatation and curettage

13

Surgery for obstructive sleep apnoea


* Denotes services identified by all three search elements. C-reactive protein tests for community-acquired pneumonia from two sources, for urinary tract infections in children in a third. Refer to Appendix for evidence and context (eg, specified indications) for each item.

Received 
10 Jul 2012
accepted 
21 Sep 2012
Adam G Elshaug, MPH, PhD, NHMRC Sidney Sax Public Health Fellow,1,2 and Inaugural Visiting Fellow3
Amber M Watt, MPH, Senior Research Officer2
Linda Mundy, MPH, Senior Research Officer, National Horizon Scanning Unit, Adelaide Health Technology Assessment2
Cameron D Willis, PhD, NHMRC Sidney Sax Public Health Fellow2,4
1 Department of Health Care Policy, Harvard Medical School, Boston, Mass, USA.
2 School of Population Health, University of Adelaide, Adelaide, SA.
3 The Commonwealth Fund, New York, NY, USA.
4 Centre for Clinical Epidemiology and Evaluation, Vancouver Coastal Health Research Institute, University of British Columbia, Vancouver, BC, Canada.
Acknowledgements: 

Funding for this project was provided by the Australian Government Department of Health and Ageing. The findings and views expressed in this article are those of the authors and do not necessarily represent those of the Commonwealth Fund, including its directors, officers or staff, or those of the Australian Government Department of Health and Ageing. Adam Elshaug and Cameron Willis hold NHMRC Fellowships (627061 and 1013165, respectively). We are grateful to Amy Lambart and Kelly Cameron of the Department of Health and Ageing for their thoughtful guidance and insight throughout the design and subsequent implementation of this project.

Competing Interests: 

No relevant disclosures.

Reference Text: 
Garner S, Littlejohns P. Disinvestment from low value clinical interventions: NICEly done? BMJ 2011; 343: d4519. doi: 10.1136/bmj.d4519.
Reference Order: 
1
Reference Text: 
Cassel CK, Guest JA. Choosing wisely: helping physicians and patients make smart decisions about their care. JAMA 2012; 307: 1801-1802.
Reference Order: 
2
PubMed ID: 
22492759
Reference Text: 
Medical Benefits Reviews Task Group. Development of a quality framework for the Medicare Benefits Schedule. Discussion paper. Canberra: Australian Government Department of Health and Ageing, 2010. http://www.health. gov.au/internet/main/publishing.nsf/Content/C38EFE94C3035988CA257713001DA46C/$File/Development%20of%20a%20Quality%20Framework%20for%20the%20MBS%20-%20Discussion%20Paper.pdf (accessed Sep 2012).
Reference Order: 
3
Reference Text: 
Elshaug AG, Moss JR, Littlejohns P, et al. Identifying existing health care services that do not provide value for money. Med J Aust 2009; 190: 269-273.
Reference Order: 
4
Reference Text: 
Ruano Raviña A, Velasco González M, Varela Lema L, et al. Identification, prioritisation and assessment of obsolete health technologies. A methodological guideline. Santiago de Compostela: Galician Health Technology Assessment Agency, 2009.
Reference Order: 
5
Reference Text: 
Ibargoyen-Roteta N, Gutiérrez-Ibarluzea I, Asua J. Guiding the process of health technology disinvestment. Health Policy 2010; 98: 218-226.
Reference Order: 
6
PubMed ID: 
20638745
Reference Text: 
Nuti S, Vainieri M, Bonini A. Disinvestment for re-allocation: a process to identify priorities in healthcare. Health Policy 2010; 95: 137-143.
Reference Order: 
7
PubMed ID: 
20015568
Reference Text: 
National Institute for Health and Clinical Excellence. NICE “do not do” recommendations. London: NICE, 2011. http://www.nice.org.uk/usingguidance/donotdorecommendations/index.jsp (accessed Sep 2012).
Reference Order: 
8
Reference Text: 
Blue Cross Blue Shield Association. Technology evaluation center assessments. http://www.bcbs.com/blueresources/tec/tec-assessments.html (accessed Sep 2012).
Reference Order: 
9
Reference Text: 
Canadian Agency for Drugs and Technologies in Health. Health technology assessment. Ottawa: CADTH, 2012. http://cadth.ca/en/products/health-technology-assessment (accessed Sep 2012).
Reference Order: 
10
Reference Text: 
National Health and Medical Research Council. A guide to the development, implementation and evaluation of clinical practice guidelines. Appendix B. Canberra: NHMRC, AusInfo, 1999. http://www.nhmrc.gov.au/_files_nhmrc/publications/attachments/cp30.pdf (accessed Oct 2012).
Reference Order: 
11
Reference Text: 
Elshaug AG, Hiller JE, Tunis SR, Moss JR. Challenges in Australian policy processes for disinvestment from existing, ineffective health care practices. Aust New Zealand Health Policy 2007; 4: 23.
Reference Order: 
12
PubMed ID: 
17973993
Reference Text: 
Cotter D. The National Center For Health Care Technology: lessons learned. Health Affairs Blog 2009; 22 Jan. http://healthaffairs.org/blog/2009/01/22/the-national-center-for-health-care-technology-lessons-learned/ (accessed Sep 2012).
Reference Order: 
13
Reference Text: 
Sheingold S, Sheingold BH. Medical technology and the US healthcare system: is this the road to Abilene? World Med Health Policy 2010; 2: Article 5.
Reference Order: 
14
Reference Text: 
Wirtz V, Cribb A, Barber N. Reimbursement decisions in health policy – extending our understanding of the elements of decision-making. Health Policy 2005; 73: 330-338.
Reference Order: 
15
PubMed ID: 
16039351
Reference Text: 
Donaldson C, Bate A, Mitton C, et al. Rational disinvestment. QJM 2010; 103: 801-807.
Reference Order: 
16
PubMed ID: 
20530539
Reference Text: 
Hodgetts K, Elshaug AG, Hiller JE. What counts and how to count it: physicians’ constructions of evidence in a disinvestment context. Soc Sci Med 2012; Aug 27 [Epub ahead of print].
Reference Order: 
17
Reference Text: 
Watt AM, Willis CD, Hodgetts K, et al. Engaging clinicians in evidence-based disinvestment: role and perceptions of evidence. Int J Technol Assess Health Care 2012; 28: 211-219.
Reference Order: 
18
PubMed ID: 
22980696
Reference Text: 
Henshall C, Schuller T, Mardhani-Bayne L. Using health technology assessment to support optimal use of technologies in current practice: the challenge of “disinvestment”. Int J Technol Assess Health Care 2012; 28: 203-210.
Reference Order: 
19
PubMed ID: 
22792877

Comments

This study and the recent CareTrack study by Runciman et al show the need for better access to evidence at the point of care, and more importantly the ongoing generation of new evidence based on actual clinical practice.  Whilst the term "big data" is bandied around in the technology world, its real benefit in healthcare is in providing near real time clinical decision support. There is an opportunity for some real leadership in this arena in Australia. A combination of the PCEHR,  the NBN,  and a desire for a reformed healthcare system should be able to deliver that. Unfortunately the last part of the equation seems to be missing. 

Dr George Margelis