Method: Analysis of the meeting interaction involved three steps: recording the meeting, annotating the verbal interactions of the meeting, and calculating various metrics from the annotations. Each annotation represented one utterance by one person. Annotations were assigned start and end times and an associated behaviour from four mutually exclusive behaviour categories. We used software and algorithms developed for this process at the Australian e-Health Research Centre.
Health care networks can take many forms. Generally they are groups of health professionals from different services and perspectives who come together to improve health care. They are increasingly being used to bring about health service improvement. In Queensland, health care networks were formally established after the Queensland Health Systems Review in 2005,1 with the aim of engaging clinicians in health care decision making. Health care networks continue to be an important policy intervention in the 2010 National Health and Hospitals Network agreement.2 As improvements in the health care system depend on the success of network teams, health systems need to consider how network teams work to achieve their goals.
The way in which decision making is approached is recognised as an important function of leadership in health care.3 This increasing interest in the process of decision making as a function of leadership is evidence of a much larger shift in leadership thinking among the academic community.4-6 Although we traditionally associate leadership in health care with individuals, the complexity of change required in the health care system means a new paradigm of leadership is required.7 Relational approaches to leadership offer an alternative view; rather than being the function of an individual, leadership is considered to be an outcome of the processes that emerge when individuals interact.5 Relational processes are being recognised as just as influential in changing peoples’ attitudes, behaviours and actions as the direct influence of a formal leader.4
Within this relational leadership paradigm sits sense-making approaches to leadership;6-8 leaders are considered as sense makers who interact in ways that enable the nature of a problem to be understood by all those involved. The Cynefin (pronounced Ku-nev-in) framework (Box 1)9 is one example of a sense-making approach, and has been identified as useful for understanding decision making in the complex environment of health care delivery.
According to Fulop and Mark, approaches that rely on technical expertise and professional experience are favoured in health care, and health care leaders are therefore at risk of being trapped in the mindset of past successes.3 To avoid this, health care leaders should pause before deciding on a course of action, consider the decision-making context in which they are operating, and choose the most appropriate approach.3
In the light of this advice and with the aim of making a connection between the Cynefin framework and the day-to-day decision making of a network team, we have developed a process for real-time capture and analysis of a network team’s interactions. This can be used to create a “meeting interaction fingerprint” that describes the interaction patterns of a network team engaged in decision making. The fingerprint consists of metrics summarising different aspects of a network team’s interactions.
The metrics from different meeting fingerprints would enable the interaction patterns that emerge during a meeting to be compared. We would like the fingerprint to capture similarities and differences in interaction patterns between teams, or meetings or decisions, which emerge during decision making. In combination, all possible metrics could be likened to a DNA chain. Just as variation in a specific part of a DNA chain can result in different traits, variation in individual metrics could indicate different decision-making processes.
We are studying the decision-making processes of seven service improvement teams. The teams are part of two health care networks in Queensland. They are subgroups of formally recognised networks with the remit of supporting and improving services across a defined geographical area. Their membership is made up of six to eight clinicians and administrators from rural, regional and remote health services across the network. Membership of a service improvement team is voluntary and each team has a nominated chairperson. The teams have a 12-month time limit and are responsible for developing, implementing and evaluating a service improvement project. Here, we demonstrate a meeting fingerprint for one meeting of one of the teams.
The meeting was recorded using two camcorders (Sony Handycam, Sony Australia, Sydney, NSW) and eight iPods (iPod nano, third generation, Apple Australia, Sydney, NSW) equipped with an amplifier (Belkin TuneTalk, Belkin Australia, Tuggerah, NSW) and lapel microphones (Sony ECM-C115 lapel microphone, Sony Australia).
The meeting was annotated into verbal utterances, and the utterances for each individual were identified. An utterance was an episode of speech by one person, with a start and end time without any contribution from another person. Verbal utterances were then subdivided and categorised, using Bales interaction process analysis, into four mutually exclusive behaviour categories: “positive” (shows solidarity, shows tension release and agrees); “negative” (shows antagonism, shows tension and disagrees); “asks” (asks for orientation, asks for opinion and asks for suggestion); and “gives” (gives orientation, gives suggestion and gives opinion).10
Manual meeting interaction analysis is time consuming (eg, it takes between 10 and 15 minutes to annotate one minute of video recording). To improve the efficiency and scalability of the annotation process, the CSIRO (Commonwealth Scientific and Industrial Research Organisation) Australian e-Health Research Centre has developed new, automated, speech processing technologies that enable the annotation of speech utterances (excluding the assignment of a behaviour) to be completed in less than 1 minute. The automated analysis includes automated utterance annotation, time synchronisation of files from multiple cameras and iPods, and software for simultaneously reviewing annotations and recordings. With automatic utterance annotation from iPod recordings, the iPods act as an ad-hoc microphone array, and the software uses relative travel times of speech to each iPod to identify the speaker location and segment the audio.
Of the large number of possible metrics that can be derived from the annotations, we calculated a subset of these that we were interested in. Two types of metrics were calculated from the annotations of the meeting: aggregate statistics, which measure the magnitude of occurrence of the various types of annotations; and sequence metrics, which measure the order of annotations. The count of behaviour annotations will be greater than the count of speaker annotations, as an utterance assigned to one speaker may be expanded into multiple annotations representing different behaviours.
A meeting fingerprint with the metrics that we are interested in is shown in Box 2. The total number of utterances was 1031, which increased to 1274 utterances after subdividing into behaviours. The total duration of utterances in hours, minutes and seconds was 1:27:04, while the total duration of utterances subdivided into behaviours was 1:22:53 (a shorter duration because of removed silences). The total duration of the meeting was 1:20:25. Overlapped speech caused the total duration of the utterances to be greater than the length of the meeting.
The sections in Box 2 each represent a different metric. The metrics are grouped into aggregate statistics and sequence metrics. On the left of Box 2, sections i, iv, vii, viii, xi and xii describe speaker-related metrics. On the right of Box 2, sections iii, vi, ix, x, xiii and xiv describe behaviour-related metrics. Raw sequence metrics (sections vii, ix, xi and xiii) are to the left of the normalised sequence metrics (sections viii, x, xii, and xiv) that eliminate most effects except that of the sequence order. The upper aggregate statistics, sections i, ii and iii, are aggregations of utterances of all time lengths. The lower aggregate statistics, sections iv, v and vi, are aggregations of utterances with particular durations. A logarithmic time scale was chosen for the durations to give a histogram bin size proportional to the duration, and very short utterances were removed to produce a smooth curve.
We have demonstrated a process for generating a meeting interaction fingerprint and given some examples of metrics that could be included. The fingerprint presented here describes the interactions over a whole meeting, leading up to a critical decision point. However, as meetings often include multiple decision points, it is also likely that different parts of meetings may have different fingerprints. A video of the fingerprint could also be made, in which each frame shows the fingerprint calculated from data up to the time of that frame.
Aggregate statistic metrics (counts of various types of annotations) give an insight into the meeting interaction. For instance, there are several dominant speakers (Box 2, i) and the short utterances tend to be positive behaviours (Box 2, vi). The aggregate statistic metric plots could be subdivided further to show the contributions of the different stages of the meeting.
Sequence metrics (the order in which types of annotations occur) can be divided into the macroscopic and microscopic order of the annotations. The upper sequence metrics (Box 2, vii, viii, ix and x) describe the macroscopic order of the annotations. The time elapsing between speakers’ utterances or speakers’ behaviours that re-occur give an insight into whether these are evenly or unevenly distributed throughout the meeting. The lower sequence metrics (Box 2, xi, xii, xiii and xiv) describe the juxtaposition of annotations. These could be regarded as describing the microscopic order of the annotations.
By manually reviewing the automated annotations, we have found a high correspondence between manual and automated annotations. Review of the places where the automated and manual meeting annotations differ shows that, for the metrics reported here, the automated process is generally correct. This automated analysis process then allows for all meetings that have been recorded to be annotated. The fingerprints of each meeting can then be normalised and plots presenting per-speaker information could be sorted so that speakers making the largest contribution are placed first.
Once all meetings have been annotated, similarities and differences between interaction patterns associated with manually identified decision-making processes can be made. We expect that this will prove useful for describing the interaction patterns that emerge during decision making.
Decision making in the different decision-making domains in the Cynefin framework may manifest as different modes of interacting and networking.11 For example, issues with strong scientific evidence, such as the promotion of proper hand-washing techniques to prevent infection, would fall into the simple domain. In the simple domain, the solution is evident; therefore, networking is more formal and targets specific individuals or groups for the purpose of communication. In the complicated domain, there is the requirement for expert debate; for example, to determine the appropriate information to be included in a treatment plan intended for general practitioners. Consequently, the network of individuals involved, although again targeting those with technical expertise, is more widespread than in the simple domain. The complex domain requires “out of the box” thinking, which is achieved by bringing together individuals or groups, all experimenting with possible solutions, to share their discoveries. Issues such as how to respond to unexplained variation in patient outcomes would fall into this domain. In the complex domain, networking and high levels of interaction are encouraged. In the chaotic domain (eg, immediately following a serious adverse event), it is important that order is restored to the system as quickly as possible; therefore, networking is once again restricted to a few individuals who decide on an immediate course of action. The intention of a meeting interaction fingerprint is to describe the differences in the interaction patterns typical of the different approaches to decision making.
Our process for annotating meetings allows for the automatic generation of a partial fingerprint (ie, one that includes the metrics that do not rely on behaviour). To date, conducting a program of research of this type on a large scale has been constrained by the time taken to manually annotate meetings. The technology developed for this project by the Australian e-Health Research Centre has given the processing power to routinely “fingerprint” meetings.
We have demonstrated the concept of generating a meeting interaction fingerprint. Future publications will report on the reliability of the annotation process, both manual and automatic. Most importantly, we will report on the results of comparing the meeting interaction fingerprints of different network teams engaged in decision making. The fingerprint concept will be expanded to include qualitative (narrative) as well as quantitative analysis.
1 The Cynefin framework9
The Cynefin framework is a sense-making tool for decision making, that identifies five types of decision-making contexts in which leaders can find themselves. These are described as five domains: simple, complicated, complex, chaotic and disordered. Decision making in each of the domains involves a different approach:
in the simple domain, a best practice solution can usually be found;
in the complicated domain, there is often no single obvious solution, but experts when brought together can usually agree on “best practice”;
in the complex domain, an experimental approach is required in which a diversity of opinions is sought, and a high level of interaction and different types of communication between stakeholders are encouraged;
in the chaotic domain, there is little time to explore possible solutions, and quick action is required with an immediate review of the impact; and
in the disordered domain, there are competing agendas and disagreement among leaders. Disorder is best dealt with by breaking the issues down into their component parts and assigning each part to the appropriate domain.
- 1. Queensland health systems review. Final report. Brisbane: Queensland Health, September 2005. http://www.health.qld.gov.au/health_sys_review/final/qhsr_final_report.pdf (accessed Dec 2010).
- 2. Australian Government. A national health and hospitals network for Australia’s future. Canberra: Commonwealth of Australia, 2010. http://www.yourhealth.gov.au/internet/yourhealth/publishing.nsf/Content/nhhn-report-toc (accessed Dec 2010).
- 3. Fulop L, Mark AL. Leadership in healthcare as a solution-oriented practice. In: Proceedings of the 24th Annual Australian and New Zealand Academy of Management Conference. Managing for unknowable futures; 2010 Dec 8-10; Adelaide, SA.
- 4. Hosking DM. Not leaders, not followers. In: Shamir B, Pillai R, Bligh M, Uhl-Bien M, editors. Follower-centred perspectives on leadership: a tribute to the memory of James R Meindl. Greenwich, Conn: Information Age Publishing, 2007: 243-263.
- 5. Uhl-Bien M. Relational leadership theory: exploring the social processes of leadership and organising. The Leadership Quarterly 2006; 17: 654-676.
- 6. Grint K. Leadership: limits and possibilities. Basingstoke, Hampshire, UK: Palgrave/Macmillan, 2005: 115-117.
- 7. Fulop L, Day GE. From leader to leadership: clinician managers and where to next? Aust Health Rev 2010; 34: 344-351.
- 8. Peck E, Dickinson H. Performing leadership. Basingstoke, Hampshire, UK: Palgrave/Macmillan, 2009: 75-78.
- 9. Snowdon DJ, Boone ME. A leader’s framework for decision making. Harvard Business Rev 2007; 85(11): 68-76.
- 10. Bales RF. Interaction process analysis: a method for the study of small groups. Cambridge, Mass: Addison-Wesley Press, 1950: 59.
- 11. Snowdon DJ. The origins of Cynefin, 2010. http://www.cognitive-edge.com/ceresources/articles/100825%20Origins%20of%20Cynefin.pdf (accessed Dec 2010).