Connect
MJA
MJA

Quality and consistency of clinical practice guidelines for diagnosis and management of osteoarthritis of the hip and knee: a descriptive overview of published guidelines

Marie L Misso, Veronica J Pitt, Kay M Jones, Hayley N Barnes, Leon Piterman and Sally E Green
Med J Aust 2008; 189 (7): 394-399. || doi: 10.5694/j.1326-5377.2008.tb02086.x
Published online: 6 October 2008

Abstract

Objective: To present a descriptive overview of the quality and recommendations of clinical practice guidelines (CPGs) on diagnosis or management of osteoarthritis (OA) of the hip and/or knee.

Data sources: CPGs were identified from several research databases (MEDLINE, EMBASE and The Cochrane Library) and guideline-specific databases from 1966 to August 2005.

Guideline retrieval: Thirty-four relevant CPGs were identified.

Data extraction: Recommendations were extracted from CPGs and categorised into: assessment and diagnosis, pharmacological management, nonpharmacological management, complementary/alternative therapy, or surgery. The quality of the CPGs were assessed by two appraisers using the Appraisal of Guidelines Research and Evaluation (AGREE) instrument.

Data synthesis: Most recommendations for aspects of diagnosis and treatment of OA of the hip and/or knee were consistent among the CPGs included in this study. However, quality varied considerably, with few CPGs being “strongly recommended” according to the AGREE quality appraisal instrument.

Conclusions: Given the number of CPGs available relevant to OA, and the consistency of recommendations within them, and considering the time and resources required for CPG development, future efforts to guide management of OA of the hip and/or knee may be better directed towards adapting existing CPGs to the local context, implementing practices known to be effective, and facilitating research to answer important questions where there is little evidence.

Osteoarthritis (OA) is a degenerative joint disease and is one of the 10 most disabling diseases in developed countries.1 Self-report data suggest that 15% of Australians have arthritis, and much of this burden is OA.2 A major cause of pain, disability and use of health care resources among middle-aged and older people, the costs associated with ongoing care of patients with OA in terms of medications, days lost from work, and cost of long-term care facilities are high.3 There is no known cure for OA, but there is an abundance of evidence available to guide diagnosis and management, and many clinical practice guidelines (CPGs) exist at the local, national and international level.

The use of evidence-based CPGs to inform clinical practice is increasing throughout the world.4 Evidence-based CPGs have the potential to improve the quality of health care by promoting interventions of proven benefit and discouraging unnecessary, ineffective or harmful interventions.5 CPG developers are encouraged to employ evidence-based principles6,7 and undertake consultation with relevant stakeholders, and peer review, dissemination and implementation, and review and updating of the resultant CPG.6,7 CPG development groups should include consumer representatives as well as relevant clinical and methodological experts. The comprehensive nature of this process means that developing evidence-based CPGs can take time and be costly.

Despite this, national and international organisations are increasingly developing their own CPGs, potentially leading to multiple CPGs on the same topic, which may not be consistent in their recommendations or quality.7 Duplication of CPGs within the same clinical area has been demonstrated in type 2 diabetes,8 acute low back pain9 and lower-limb OA,10-12 where the recommendations are consistent among CPGs, but the evidence underpinning these recommendations varies.8 Methodological quality of CPGs is found to vary considerably. Overall quality and transparency of CPG development processes, and consistency in reporting is poor.9-12 While there have been descriptive overviews of CPGs for OA of the hip and/or knee, these reviews have not performed comprehensive searches and therefore only contain a subset of available relevant CPGs, and often conclude with a call for more CPGs.

Here, we present a comprehensive descriptive overview and critical appraisal of CPGs for diagnosis and management of OA of the hip and/or knee identified through a comprehensive search. We aim to encourage future CPG development approaches in OA (such as adapting existing CPGs) that consider the time- and resource-intensive nature of this process.

Methods

This descriptive overview of CPGs for OA was conducted within a larger study evaluating a collaborative method to promote best practice in management of OA of the hip and/or knee in Victorian general practices.13 One outcome of this larger study was a key messages table (available at http://www.cochrane.org.au/projects/osteo_guidelines.php) that comprised a summary of CPG recommendations. This project was funded by a grant from the Australian Government Arthritis and Musculoskeletal Quality Improvement Program.

Identification of guidelines

A systematic search was conducted of relevant databases and websites from 1966 to August 2005 (Box 1). Search terms included: osteoarthritis knee; osteoarthritis hip; practice guideline/s; guideline/s; and combinations of these terms.

Quality assessment

The quality of included CPGs was assessed independently by two appraisers using the Appraisal of Guidelines Research and Evaluation (AGREE) instrument.14 This instrument is a result of a collaboration of international researchers and policymakers from 13 countries which was formed in the mid 1990s to develop an internationally recognisable framework to assess the quality of CPGs. The resultant AGREE instrument is designed to assist guideline developers, users and those wishing to critically appraise CPGs, and consists of 23 items separated into six domains according to a dimension of guideline quality. We used the instrument and accompanying user guide (containing definitions and additional information for each item) in this overview to score each item in each domain. Using the AGREE protocol, we scored items on a four-point Likert scale with the following range: 4 (strongly agree); 3 (agree); 2 (disagree); 1 (strongly disagree). Domains were scored by summing all the item scores in a domain and standardising the total as a percentage of the maximum possible score for that domain. The AGREE instrument states that it is not possible to set thresholds for the domain scores to conclude that a guideline is good or bad. Instead, the domain scores can be used to help decide whether the guideline should be strongly recommended (if over half of the domains score > 60%), recommended (most of the domains score > 30%) or not recommended (most of the domains score < 30%). The most positive category was chosen in the case of a tie (ie, CPGs were recommended if half of the domains scored < 30% and half of the domains scored > 30%). Disagreements between appraisers were resolved through discussion until consensus was reached.

Results

Our search yielded a total of 291 potentially relevant articles. Of these, 193 were retrieved from databases and 98 from websites that specifically publish CPGs. Of the 291 potentially relevant articles, 242 were excluded after screening the titles and abstracts, leaving 49 to be assessed based on the full text version of the article. Of those 49 articles, 34 CPGs were identified for management of OA of the hip and/or knee and were included in this overview (Box 2).

Of the 34 CPGs included in our overview, 17 originated from the United States, seven from the United Kingdom, four from Europe, two each from Australia and Canada, and one each from South Africa and Hong Kong; their publication dates ranged from 1993 to 2005. There was a marked increase in the number of CPGs developed since 2000 (28 CPGs), with most being published in the UK and US. Fourteen CPGs were newly developed, three were adapted from existing CPGs, and eight were updates of existing CPGs. In the case of nine of the CPGs, it was unclear whether they were newly developed, adapted or updated.

CPGs varied considerably in format, coverage and scope. For example, some CPGs provided detailed information about indicators for diagnosis, while other CPGs were limited to a brief list of indicators only. Similarly, the scope varied, with some CPGs being limited to a specific management modality (pharmacological management) while others included more than one or all possible management modalities (pharmacological, nonpharmacological and surgery). Irrespective of the organisation that developed the CPG, population, coverage or scope, most CPGs had similar recommendations for aspects of diagnosis and treatment of OA of the hip and/or knee.

Quality assessment

The evaluation of the methodological quality of the CPGs included in our overview, by means of the AGREE instrument,14 is summarised in Box 2. Disagreements between appraisers were rare, and were only in the order of one item score, where the disagreement was between “agree” and “strongly agree” or “disagree” and “strongly disagree”, but not between “agree” and “disagree”.

The quality of CPGs was variable. The scope and purpose domain was clearly addressed in most of the CPGs (average score, 76%), as was clarity and presentation (77%). By contrast, the stakeholder involvement (35%), rigour of development (47%), applicability (18%) and editorial independence (30%) domains were frequently poorly addressed or omitted. While CPG development groups often comprised individuals from relevant professional groups and the target users were clearly described, only three CPGs reported seeking patients’ views and preferences, and only two CPGs were piloted among target users. The rigour of the development methods used was variable across CPGs. Health benefits, side effects and risks were frequently considered in formulating recommendations, and links were made between the recommendations and the evidence. However, external review was omitted in 21 CPGs, and updating procedures were omitted in 31 CPGs; 14 CPGs failed to address dissemination and implementation adequately, and 22 CPGs did not address applicability. Furthermore, declaration of financial support for the development process and independence from the funding body was poorly reported (not declared in 25 CPGs). Domain item scores for each CPG included in this overview are available from the authors on request.

Despite the considerable variation in quality among the CPGs, six CPGs were appraised as “strongly recommended”: (i) an Australian (Clinical Epidemiology and Health Service Evaluation Unit) evidence-based clinical pathway for best practice management of OA of the hip and knee;17 (ii) the Ontario Program for Optimal Therapeutics guideline;18 (iii) the American Pain Society guideline;19 (iv) the American Geriatric Society Panel on Persistent Pain in Older Persons guideline;20 (v) the American Geriatric Society Panel on Exercise and Osteoarthris guideline;21 and (vi) the MOVE consensus guideline.22

Discussion

There is substantial investment in the development and dissemination of CPGs to inform clinical practice and promote effective health care.4 Multiple CPGs on the same topic are being developed by various local, national and international organisations, and these CPGs are not necessarily consistent in either their recommendations or their quality.7 Where recommendations are consistent, the duplication of effort is time- and resource-intensive and potentially wasteful; where inconsistent, they can provide conflicting treatment recommendations.23 Ideally, the quality of CPGs, as assessed by a validated framework, should be consistent across CPGs regardless of topic or setting.

Our overview describes the number and content of CPGs for the diagnosis and management of OA of the hip and/or knee at the local, national and international level. Treatment recommendations included in these CPGs are consistent, but only a small number of the CPGs included were adapted from existing CPGs — most were newly developed.

Given the consistency among recommendations in existing CPGs, and the time-intensive and resource-intensive nature of CPG development, a possible approach for future guidance in this setting is the adaptation of existing high-quality CPGs to local contexts. This could form one step in an overall implementation strategy to ensure sustainable quality of care for patients with OA. Adaptating CPGs to local contexts, facilitated by key local knowledge translation experts who have experience in applying evidence to a local setting, also has the potential to lead to better uptake of CPGs. Adaptation of CPGs should be systematic and transparent, should involve stakeholders, and should report the key factors that influence decisions, including those in the original CPG, and the reasons for any modifications that are made.24 Steps often lacking in the original development process, such as piloting of CPGs among target users, patient input and review, auditing, external review, consideration of organisational barriers, cost implications and updating plans can be addressed in the adaptation process.

An example of a framework for identifying CPGs for local adaptation is ADAPTE.25 ADAPTE is useful for determining whether to adapt an existing CPG by means of a step-by-step process that promotes the use of evidence-based and rigorous development principles, and introduces concepts for adapting CPGs from their original setting to different contexts, whether cultural or organisational.25 Adaptation of CPGs is addressed in varying levels of detail in handbooks for preparing CPGs from the Council of Europe, the National Health and Medical Research Council (Australia), the National Institute for Clinical Excellence (UK), the New Zealand Guidelines Group and the Scottish Intercollegiate Guidelines Network.26

There have been cases where high-quality CPGs have been adapted to the local context and implemented successfully into clinical practice. One such model is the Australian-based Health for Kids (HFK) project that is designed to improve health care for children through best practice and partnerships among medical, nursing and allied health clinicians and consumers.27 The HFK model involves key local CPG development specialists adapting existing high-quality CPGs for local use, and has been implemented in the care of asthma and diarrhoea.28

In this overview, we have shown an example of a large number of CPGs addressing the same clinical issue. These CPGs were of varying quality, but consistent in the recommendations given. Given the time and resources required for evidence-based CPG development, we encourage future developers to first determine whether a high-quality evidence-based CPG already exists for the clinical topic of interest, and then to assess whether the CPG is suitable for adaptation to the local context. If a high-quality evidence-based CPG exists, it may be unnecessary to duplicate a search of the primary research and instead focus on adapting existing CPGs and addressing the poor uptake of CPGs by designing well funded and rigorous implementation and sustainability strategies for practices known to be effective, and facilitating research to address identified areas lacking evidence.

  • Marie L Misso1
  • Veronica J Pitt2
  • Kay M Jones3
  • Hayley N Barnes4
  • Leon Piterman5
  • Sally E Green6

  • Monash University, Melbourne, VIC.



Acknowledgements: 

We acknowledge Jako Burgers, Anne Parkhill, Tari Turner, Amanda Weeks and Silva Zavarsek for their contribution to the study and manuscript.

Competing interests:

Marie Misso, Veronica Pitt, Hayley Barnes and Sally Green are staff of the Australasian Cochrane Centre, funded by the Australian Department of Health and Ageing and supported by Monash University. Sally Green is a member of the Cochrane Collaboration Steering Group. The views expressed in the present paper represent those of the authors and are not necessarily the views or the official policy of the Cochrane Collaboration (unless otherwise stated and referenced).

  • 1. World Health Organization. Chronic diseases and health promotion: Chronic rheumatic conditions. http://www.who.int/chp/topics/rheumatic/en/ (accessed Jul 2008).
  • 2. Australian Bureau of Statistics. National health survey: first results, 1995. Canberra: ABS, 1996.
  • 3. Maetzel A, Li LC, Pencharz J, et al. The economic burden associated with osteoarthritis, rheumatoid arthritis, and hypertension: a comparative study. Ann Rheum Dis 2004; 63: 395-401.
  • 4. Grimshaw JM, Thomas RE, MacLennan G, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess 2004; 8 (6): iii-iv, 1-72.
  • 5. Woolf SH, Grol R, Hutchinson A, et al. Clinical guidelines: potential benefits, limitations, and harms of clinical guidelines. BMJ 1999; 318: 527-530.
  • 6. National Health and Medical Research Council. A guide to the development, implementation and evaluation of clinical practice guidelines. Canberra: NHMRC, 1999. http://www.nhmrc. gov.au/publications/synopses/cp30syn.htm (accessed Jul 2008).
  • 7. World Health Organization. Guidelines for WHO guidelines: Geneva: WHO, 2003. http://www.gradeworkinggroup.org/publications/WHO_EIP_GPE_EQC_2003_1.pdf (accessed Jul 2008).
  • 8. Burgers JS, Bailey JV, Klazinga NS, et al. Inside guidelines: comparative analysis of recommendations and evidence in diabetes guidelines from 13 countries. Diabetes Care 2002; 25: 1933-1939.
  • 9. van Tulder MW, Tuut M, Pennick V, et al. Quality of primary care guidelines for acute low back pain. Spine 2004; 29: E357-E362.
  • 10. Pencharz JN, Grigoriadis E, Jansz GF, et al. A critical appraisal of clinical practice guidelines for the treatment of lower-limb osteoarthritis. Arthritis Res 2002; 4: 36-44.
  • 11. Poitras S, Avouac J, Rossignol M, et al. A critical appraisal of guidelines for the management of knee osteoarthritis using AGREE criteria. Arthritis Res Ther 2007; 9: R126.
  • 12. Zhang W, Moskowitz RW, Nuki G, et al. OARSI recommendations for the management of hip and knee osteoarthritis, part I: critical appraisal of existing treatment guidelines and systematic review of current research evidence. Osteoarthr Cartil 2007; 15: 981-1000.
  • 13. Jones K, Piterman L. Promoting best practice in general practitioner management of osteoarthritis of the hip and knee: arthritis and musculoskeletal quality improvement pilot program. Aust J Prim Health 2007; 13 (2): 104-112.
  • 14. AGREE (appraisal of guidelines for research and evaluation) Collaboration. AGREE instrument. http://www.agreecollaboration.org/instrument/ (accessed Jul 2008).
  • 15. Eccles M, Freemantle N, Mason J. North of England evidence based guidelines development project: methods of developing guidelines for efficient drug use in primary care. BMJ 1998; 316: 1232-1235.
  • 16. Shekelle PG, Woolf SH, Eccles M, Grimshaw J. Clinical guidelines: developing guidelines. BMJ 1999; 318: 593-596.
  • 17. Brand C, Cox S. Evidence-based clinical pathway for best practice management of osteoarthritis of the hip and knee. Melbourne: Clinical Epidemiology and Health Service Evaluation Unit, Royal Melbourne Hospital, 2004.
  • 18. Holbrook AM (Chair) for Ontario Musculoskeletal Therapy Review. Ontario treatment guidelines for osteoarthritis, rheumatoid arthritis and acute musculoskeletal injury. Toronto: Queen’s Printer of Ontario, 2000.
  • 19. American Pain Society. Guideline for the management of pain in osteoarthritis, rheumatoid arthritis, and juvenile chronic arthritis. 2nd ed. Glenview, Ill: American Pain Society, 2002.
  • 20. AGS Panel on Persistent Pain in Older Persons. The management of persistent pain in older persons. JAGS 2002; 50 (6 Suppl): S205-S224.
  • 21. American Geriatrics Society Panel on Exercise and Osteoarthritis. Exercise prescription for older adults with osteoarthritis pain: consensus practice recommendations. A supplement to the AGS clinical practice guidelines on the management of chronic pain in older adults. JAGS 2001; 49: 808-823.
  • 22. Roddy E, Zhang W, Doherty M, et al. Evidence-based recommendations for the role of exercise in the management of osteoarthritis of the hip or knee — the MOVE consensus. Rheumatology 2005; 44: 67-73.
  • 23. Grilli R, Magrini N, Penna A, et al. Practice guidelines developed by specialty societies: the need for a critical appraisal. Lancet 2000; 355: 103-106.
  • 24. Schunemann HJ, Fretheim A, Oxman AD. Improving the use of research evidence in guideline development. 13. Applicability, transferability and adaptation. Health Res Policy Syst 2006; 4: 25.
  • 25. ADAPTE. ADAPTE framework. http://www.adapte.org/rubrique/adapte-framework.php (accessed Sep 2008).
  • 26. Turner T, Misso M, Harris C, et al. Development of evidence-based clinical practice guidelines (CPGs): comparing approaches. Implement Sci 2008. In press.
  • 27. Harris C, Turner T, Mazza D, et al. Guideline development process for the Health for Kids in the South East project. Aust Fam Physician 2008; 37: S2-S5.
  • 28. Health for Kids in the South East Project Team. Evaluation of evidence-based clinical paths for asthma, croup, gastroenteritis and bronchiolitis. Melbourne: Centre for Clinical Effectiveness, Southern Health, 2007.

Author

remove_circle_outline Delete Author
add_circle_outline Add Author

Comment
Do you have any competing interests to declare? *

I/we agree to assign copyright to the Medical Journal of Australia and agree to the Conditions of publication *
I/we agree to the Terms of use of the Medical Journal of Australia *
Email me when people comment on this article

Online responses are no longer available. Please refer to our instructions for authors page for more information.