Objective: To investigate the feasibility, achievement and acceptance of indicators of quality general practice in the RACGP Standards for general practices (third edition), using complaints registers as a case study.
Design, setting and participants: A purposive sample of convenience of 200 general practices (stratified according to location and size) participated in a field test of quality and safety proposals during an accreditation survey visit between October 2004 and February 2005. Included was a test of the proposal for a complaints register (a document where complaints made to the practice are recorded).
Results: Few practices used a formal complaints register (79/200; 39.5%), with large practices more likely (12/20; 60.0%) and very remote practices less likely (1/11; 9.1%) to use one. The proposal for complaints registers was rated feasible by 123 general practices (61.5%) and rated acceptable by 121 general practices (60.5%).
Conclusions: The proposal for complaints registers in general practice, while popular with policymakers, gained limited support when tested in Australian general practice. This shows the need for a balance between the expectations of policymakers, the need to increase performance by setting standards, and the practicalities of every-day general practice.
Since the early 1990s, the Royal Australian College of General Practitioners (RACGP) has been developing a set of Standards for general practices (the Standards).1 The RACGP Standards aim to:
engage general practices in quality improvement;
articulate the professional norms of high-quality general practice; and
focus on the structures and processes within the practice setting, rather than individual practitioner competence.
Because of the comprehensiveness of the Standards, it is important to ensure that they are appropriate in scope and implementation. Between October 2003 and June 2005, the RACGP undertook a comprehensive review of the Standards to ensure they reflected the norms of contemporary general practice. The review culminated in the publication of the third edition in July 2005.1
In July 2004, after considering the available Australian and international literature on quality and safety, as well as the feedback from extensive national consultation, the RACGP faced a dilemma. The suggested changes to the Standards were numerous and often divergent. General practitioners and their practice teams wanted the RACGP to have a robust method for determining which proposals were finally included. As a result, the RACGP engaged in a second round of consultations and a field test of the wide range of proposals.
The field test results formed one component, albeit an important one, in the RACGP’s decision-making process in rewriting the Standards. The RACGP also considered other feedback from the profession, and information from other sources. This included opinion of the legal issues associated with some proposals, including a complaints register.2
A complaints register is a document (electronic or otherwise) where complaints made to the practice are recorded. Although the proposal to include a complaints register had merit, and there had been a recent national project on complaints management in Australian health care,3 very little research had been undertaken in relation to the use of complaints registers within Australian or international general practice. The RACGP was uncertain whether general practices in Australia currently used complaints registers, whether complaints registers were feasible, and whether GPs and other members of the practice teams found them acceptable. As a result, the RACGP included a range of proposals relating to complaints management in the field test to ensure that decisions about their inclusion in the new Standards would be based on sound research evidence.
Here, we outline the results of the field tests as they apply to the proposal to include a formal complaints register as a requirement for Australian general practices. (The results of the entire field test are available from the RACGP.4)
The data collection component of the study was subcontracted to the organisations that assess general practices against the RACGP Standards for general practices accreditation: Australian General Practice Accreditation Limited (AGPAL) and GPA ACCREDITATION plus.
A purposive sample of convenience of 200 general practices across Australia was recruited. Practices scheduled to undertake a (re)accreditation survey visit between 21 October 2004 and 28 February 2005 were eligible to participate. The practices were stratified according to the rural, remote and metropolitan areas (RRMA) classification5 to ensure sufficient sampling from each category of location for robust statistical comparison. Recruitment of practices was performed by the accreditation organisations. The recruitment rate was 91.7%.
One GP and one member of the practice team in each practice rated the feasibility and acceptance of the complaints register proposal using a self-completed questionnaire. They rated on a scale of 1–5 whether the practice would like to achieve the proposal (measure of acceptance) and whether the practice would find it easy to achieve the proposal (measure of feasibility).
Data were analysed using SPSS, version 11 (SPSS Inc, Chicago, Ill, USA). Descriptive statistics and selected cross-tabulations were performed. A target of 75% of the sample was set to illustrate high levels of acceptance, feasibility and achievement. A difference of 10% from the mean was deemed to be the threshold to be reported as a considerable difference in cross-tabulations.
A total of 200 general practices participated in the field test. The RRMA distribution of the participating practices is given in the Box, as is the practice size distribution.
Results for the proposal to include a formal complaints register are also provided in the Box. Less than half the practices had a complaints register (79/200; 39.5%), with large practices tending to be more likely (12/20; 60.0%) and very remote practices (RRMA 7) tending to be less likely to have one (1/11; 9.1%). The proposal to include complaints registers in the new Standards was rated feasible and acceptable by over half the general practices (123 [61.5%] and 121 [60.5%], respectively).
The principal findings of the field test were that few Australian general practices recorded complaints on a register, and just over half the general practice professionals surveyed thought it was feasible and acceptable to do so.
Like all applied research, the field test had both strengths and weaknesses that need to be considered when reviewing the results. It was conducted in the context of an established accreditation system at a time when peak numbers of practices were scheduled to undertake re-accreditation against the Standards. Although this assisted in ensuring that the results reflected the “reality” of assessment during the normal accreditation process, rather than a theoretical or “manufactured” experiment of testing in the field, one limitation of the study is that results were reliant on the quality of the accreditation process. The purpose of the field test was to examine the new proposals themselves, not to evaluate the way in which the accreditation process worked as a test of these proposals. Therefore, issues relating to the accuracy of the accreditation process (survey visit) in assessing the proposals (such as issues of reliability, validity and consistency in surveying) were beyond the scope of the field test.
The field test was the first Australian study into the use, acceptance and feasibility of complaints registers in Australian general practice. A strength of the test is the confidence that the results can be generalised to the general practice population in Australia. A comparison was made between the sample and practices participating in the Practice Incentives Program (PIP) to determine the representativeness of the sample of accredited general practices in Australia. The Australian Government Department of Health and Ageing provided unpublished data for this purpose. By design, it was expected that the sample would be under-represented in RRMA 1 and 2 practices and over-represented in RRMA 3–7 practices. PIP data confirmed that the sample was broadly representative of Australian general practices, with some under-representation of RRMA 1 and 2 practices and some over representation of RRMA 3–7 practices, as expected. However, the PIP data comparison also suggested that the sample under-represented large practices (more than four full-time equivalent GPs).
The results of the field test had a number of implications for the RACGP’s decision-making process in revising the Standards, as well as for general practices in implementing the Standards. The proposal to include a complaints register in the Standards, while popular with policymakers and patients, gained limited support when tested in general practice. The basis for this limited support may lie in GPs’ concern about the legal implications of formalising complaints management. A legal opinion commissioned by the RACGP suggested that a written complaints register required for the Standards might, under certain circumstances (although rare and infrequent) be discoverable, admissible and probative in a range of legal proceedings.2
Thus, in its final decision making, the RACGP needed to balance the desire to reflect good practice in complaints management in the Standards with the evidence that a substantial number of GPs did not maintain a complaints register and contested the feasibility and acceptance of implementing complaints registers.
The implications for general practices are that the new Standards1 now place a greater emphasis on complaints management in general practice than previous editions of the Standards and include many elements of best practice, but do not include a complaints register. Criterion 2.1.2 and Criterion 4.1.1 in the Standards1 include requirements for complaints management, particularly in relation to having a “system” to manage complaints; that staff members are aware of the processes used for complaints management; that the practice has an identified leader in complaints resolution; and that the practice can describe an improvement made in response to a complaint. These requirements reflect a greater emphasis on complaints management in general practice, and will require improved complaints management for some general practices. The focus on making improvements is a realistic means of meeting the needs of patients — reducing the recurrence of the complaints.
While the RACGP has reduced the risks associated with discovery of recorded complaints during a legal proceeding (by not requiring a complaints register in the Standards), the field test results suggest that policymakers may need to consider the role of privilege (used to protect other quality improvement processes) or legal protection of the complaints register (as has been applied to an apology) to address the acceptance and feasibility of requiring the use of complaints registers in general practice.
The field test shows the importance of testing the acceptance, feasibility and current achievement levels of safety and quality proposals before requiring their implementation in general practice. The example of including complaints registers in the RACGP Standards demonstrates the need for an evidence base to provide a balance between the sometimes ambitious expectations of policymakers and the practicalities and realities of every day general practice in Australia. This study suggests that it is important in the future to assess the “quality” of safety and quality proposals in general practice before implementing them across the profession.
Results of the field test of a complaints register proposal, by the characteristics of the participating general practices. Data are number (%) of practices
RRMA = Rural, remote and metropolitan areas. RRMA 1 & 2 includes capital cities or metropolitan areas; RRMA 3 & 4 includes large and small rural centres; RRMA 5 & 6 includes very-small or remote centres; and RRMA 7 includes very remote areas.5 FTE = full-time equivalent.
- 1. Royal Australian College of General Practitioners. Standards for general practices (3rd ed). South Melbourne: RACGP, 2005. Available at: http://www.racgp.org.au/document.asp?id=17623 (accessed Jun 2006).
- 2. Milstein & Associates. Changes to college standard: advice re medical legal repercussions. Available at: http://www.racgp.org.au/downloads/pdf/standards/Changes_to_College_Standards_Advice_re_Medical_Legal_Repercussions.pdf (accessed Jun 2006).
- 3. Australian Council for Safety and Quality in Health Care. Better practice guidelines on complaints management for health care services. Sydney: ACSQHC, 2004.
- 4. Hutchinson R. Report of the field test of the draft Standards for General Practices (3rd ed). South Melbourne: Royal Australian College of General Practitioners, 2005.
- 5. Australian Government Department of Health and Ageing. The Rural, Remote and Metropolitan Area (RRMA) classification system. Available at: http://www.health.gov.au/internet/wcms/Publishing.nsf/Content/health-workforce-bmp-rrma.htm (accessed Jun 2006).
Publication of your online response is subject to the Medical Journal of Australia's editorial discretion. You will be notified by email within five working days should your response be accepted.