In the two years of the CSSP (2000–2002) we identified some important factors that influence whether evidence can be put into practice. In the main, these factors are qualitative and subjective. While this does not minimise their importance, it makes it more difficult to be categorical about ensuring that routine practice is based on evidence. Nevertheless, we contend that, unless these factors are addressed and resolved in a way that is locally relevant, it is unlikely that evidence will be implemented into practice in a sustainable way.
We were surprised at the degree to which training and education were required to enable staff to participate effectively in the CPI project. Education was required to overcome the concept of the randomised controlled trial (RCT), which is deeply entrenched in the clinical psyche and was a major hurdle to implementing CPI. In the RCT model, professional data collectors derive frequently voluminous, detailed and all-inclusive data to be analysed at the end of a study, while at the same time trying to minimise the impact on day-to-day clinical experience. On the other hand, CPI is a cyclical process carried out by the clinical team, who themselves design a small study to collect only data relevant to a utilitarian question about practice. The clinical team collects the data, immediately analyses them for their utility, modifies the questions or data collection process or behaviour and repeats the cycle.1,2 Thus, while the RCT is central to the acquisition of evidence, it provides many distractions to the introduction of CPI. To firmly entrench CPI in clinical practice, training needs to be directed at instilling the principles outlined in Box 1.
Training improved the skill of the team in recognising and understanding the nature of evidence and in identifying and demonstrating that evidence had been translated into practice. While this seems simple enough, it belies the underlying complexities and steps required to bring the staff to this level. Establishing this skill level took much of the two years of the CSSP at Monash — indeed, it was not possible to examine some of the other important structural, organisational and attitudinal factors that affect the implementation of best practice until this was achieved.
The distinction between management (which implies preserving the status quo) and leadership (which calls for responsiveness to change, creative problem-solving and the ability to motivate others) has been made before,3-5 and the differences were clearly observable in our study (Box 2). The hospital context, which bolsters clinical safety and efficiency through routine and reiteration, emphasises management. Management processes of this kind are well developed, and consequently educating staff about management was not the principal issue. Instead, management needed to be harnessed and redirected to the task of day-to-day collecting and reviewing of data for evidence of best practice. On the other hand, the process of creating change and of cleaving out administrative “space” and resources requires leadership. At some of our sites, both leadership and management were apparent by the end of the project. While desirable management characteristics were readily apparent and have been well described in the past, the qualities of good leadership are less tangible (although it was usually recognisable when present). We also suspect that leadership may be less susceptible to training and that good leaders may be best obtained by selective recruitment.
A central feature of CPI is that the questions asked, and thus the data collected, need constant modification. The organisational structure in which a clinical team operates is organic and constantly changing: hence, the activities of the team need constant monitoring and the factors being monitored may alter. Successfully implementing evidence into practice presupposes that data are readily available to provide rapid feedback on the effectiveness of the implementation. However, the natural inclination of clinical teams seems to be to revert to collecting large amounts of complex data for review at some distant time, with many of the questions posed not being directly relevant to the problem at hand. By contrast, a small number of apposite questions, asked in a timely manner and with results rapidly fed back, can lead to a prompt change in practice (Box 3). This usually presupposes access to a computer network.
Finally, the need for a small number of apposite questions implies the ability to ask the right questions. The confluence of management skill, an ability to translate research evidence into routine clinical behaviour and an understanding of the process of quality assurance are central — they are, in effect, the prerequisites for effective data collection and interpretation.
For the CPI process to become embedded in clinical practice, cultural change must occur. Staff require confidence to question their own practice and to experiment in ways that require long-term personal and institutional investment. This is only possible when the larger hospital administrative culture places sufficient store on implementing best practice to provide the necessary resources. These may include computer support or funded time for staff to devote to education and data collection. Perhaps most importantly, institutions must provide buffers against demoralising structural changes that cut across hard-won improvements in practice. Regrettably, we found little evidence that the commitment of institutional administrators to best practice matched the commitment of their clinical staff, despite the potential benefits to the institution of adopting a CSSP approach (Box 4).
1: Concepts required for clinical practice improvement
Aim for usefulness rather than perfection. The aim is to bring about a desired outcome by following a particular form of practice, and the purpose of the questions posed and data collected is to achieve this. Absolute compliance with the protocol is not necessary — compliance failure is as enlightening as adherence. Staff should be able to recognise that the data are useful to them in their daily practice.
Start small. Begin with simple questions and collect an amount of data that is manageable and not an imposition on staff. The purpose of beginning small is to (a) better understand the root causes of a problem, (b) obtain collaboration from staff, and (c) identify major issues that will require pressure for change. It is not necessary to begin with a system that collects all conceivable data at the outset — new data can be collected later, as needed.
Just do it. Small studies targeting key questions are all that is required to begin. Beginning is also part of educating staff about the nature of clinical practice improvement (CPI).
Expect to change the system. A central feature of CPI is constant modification of questions and data. The organisational structure in which a clinical team operates is organic and constantly changing; hence, the activities of the team need constant monitoring and the factors monitored may alter.
2: Newton’s first law of motion in the hospital
Every ward unit continues in a state of rest or uniform motion unless compelled by some external force to change that state of rest or uniform motion.
In hospitals, the management process provides a buffer against external forces, creating an environment in which routine and reiteration reduce the risk of error. However, this state also reduces responsiveness to change when new evidence indicates that old practices are no longer justified. Change requires leadership, which in turn requires an ability to analyse the root causes of a problem. Leadership can’t be exerted without being able to identify the core issues to be addressed.
The problems experienced by our study participants in adopting a clinical practice improvement (CPI) model are a good example of the difficulty of learning to analyse data and evidence in a new way. All site representatives struggled to identify the core information that could act as indicators of adherence to CPI.
3: Focusing questions to ensure that the data collected for CPI are manageable and relevant
The aim is to make data collection simple and relevant and to allow questions to be constantly and readily modified. A useful approach is to provide a generic questionnaire form (available on the Internet) consisting of a small number of questions involving Boolean (yes/no) logic and scalars (how long, how much, how big or of what value). A list of three to five of these questions is sufficient to obtain key indicators for many important management issues.
Our study showed that being constrained by the number of questions, being able to easily modify the questions, and having rapid feedback of new data encouraged teams to ask relevant questions based on a small amount of easily collected data.
CPI = clinical practice improvement.
4: Impact of the Clinical Support Systems Program
The hospitals participating in our study as part of the Clinical Support Systems Program (CSSP) were Monash Medical Centre (MMC), Frankston Hospital, Cabrini Hospital and Warragul Hospital. After a slow uptake of the clinical practice improvement (CPI) program, the average length of stay at each participating hospital for all stroke patients managed at that hospital fell by over 15% and showed a sustained downwards trend (A). A reduction in average length of stay is a useful indicator of the increased effectiveness of clinical and ward processes. While the project cannot claim sole responsibility for achieving this improvement in performance, it is clear that, with a focus on stroke management created by the project and the provision of the environment and mechanisms needed for staff to make improvements, results ultimately follow.
The fall in average length of stay seems to have been associated with adherence to clinical protocols (B). There was significant improvement in the application of protocols and, by implication, an improvement in the quality of service provided (assuming that application of best practice results in better patient outcomes). The average length of stay for stroke patients at MMC fell by 3 days (representing about $300 000 in savings) — this alone would offset the cost of implementing CPI.
- 1. Demming WE. Quality, productivity, and competitive position. Cambridge, Mass: Massachusetts Institute of Technology Center for Advanced Engineering Study, 1982.
- 2. Joss R, Kogan M. Advancing quality: total quality management in the National Health Service. Buckingham, UK: Open University Press, 1995.
- 3. Schein E. Organizational culture and leadership. 2nd ed. San Francisco: Jossey Bass, 1992.
- 4. The leaderlab model. Center for Creative Leadership, 2000. Available at: www.ccl.org/CCLCommerce/index.aspx (accessed Apr 2004).
- 5. Senge PM. The fifth discipline: the art and practice of the learning organization. London: Random House, 1990.
Abstract
As part of the Clinical Support Systems Program, the Monash University Consortium conducted a project to identify factors influencing the implementation of clinical evidence into routine hospital practice.
Training was required in the process of clinical practice improvement (CPI) and the nature of evidence.
One of the most helpful instruments for change was to point to active models of quality assurance as exemplars.
Staff can be trained to be good managers, but leadership is less susceptible to training and is better obtained by selective recruitment.
CPI requires rapid feedback on the effectiveness of the implementation. Access to this information and the confluence of management skill, an ability to translate research evidence into routine clinical behaviour and an understanding of the process of quality assurance are central.
Effective CPI is only possible when the larger hospital administrative culture is committed to providing the necessary resources.