MJA
MJA InSight
MJA Careers
Supplement

Bringing evidence to bear on policy processes: the challenge of the Australian Primary Health Care Research Institute

Nicholas J Glasgow, John E Marley, Linda J Kristjanson, Janette A Donovan, Sally J Hall, Mark F Harris, David M Lyle, Elizabeth J Kerr and Frith Rayner
Med J Aust 2008; 188 (8): 46

The Australian Primary Health Care Research Institute (APHCRI) is part of the Primary Health Care Research Evaluation and Development (PHCRED) strategy.1 The Institute was the most recent element of the PHCRED strategy to be established, beginning research activities following the appointment of the Research Advisory Board (RAB) in November 2003. APHCRI’s mission is to “provide national leadership in improving the quality and effectiveness of primary health care through the conduct of high quality priority-driven research and the support and promotion of best practice”.2 APHCRI has a specific focus on the links between primary health care evidence and policy. Its activities not only fund research programs, but also seek to build capacity within the research community and policy community to facilitate the adoption of evidence into policy.

There are two important interdependent features in the APHCRI model adopted to fulfil this mission. Firstly, it is a “virtual” institute operating a “hub and spoke” model. The Institute comprises staff based at the Australian National University (the “hub”) and, within different streams of research activities, multiple “spokes” undertaking commissioned programs of research. Secondly, the Institute has a specific pool of funds to commission research — it both funds research and undertakes research. The Institute’s RAB sets the research priorities, oversees the independent assessment of applications for funding and determines the successful spokes. Groups compete to be commissioned within a particular stream of research, but, once successful, collaborate as part of the virtual institute.

In this article, we provide an overview of the approach APHCRI has taken to bring research evidence to bear on policy formation. We also reflect on lessons learned through the process of conducting our fourth research stream.

Linkage and exchange, APHCRI style

Influencing policy with research evidence is not a simple “linear” proposition. Health policy draws on many information inputs apart from research evidence, including political realities.3 APHCRI has adapted the Canadian Health Services Research Foundation “linkage and exchange” approach4 in order to make its research products more useful to policymakers.

APHCRI links together four groups of participants to exchange knowledge from their different perspectives. These are:

  • Policymakers and decisionmakers in both the federal and state/territory spheres;

  • Providers of primary health care services and the various organisations with which they are associated;

  • Researchers; and

  • Users of primary health care services, and the various organisations with which they are associated.

Members of these four groups serve on the RAB. APHCRI’s research priorities are iterated with policy advisers and the RAB to ensure they are relevant to policy. Expert review committees, convened to assess applications within the different streams, include members with expertise across these groups. The assessment criteria for applications within streams reflect the emphasis on policy and provider expertise in addition to more usual academic criteria.

Stream 4

APHCRI organises its research programs in “streams”. Each stream has a particular focus and may have several spokes or individuals working within it (the numbers of the streams denote the chronological order in which they were announced).

The Institute’s Stream 4 program (with $1.8 million funding in total) further sharpened the linkage and exchange focus. It aimed to increase both the capacity of researchers to respond to policy priorities and the capacity of policy advisers to utilise research evidence. A list of policy-relevant topics, approved by the RAB, was identified in consultation with the Australian Government Department of Health and Ageing. A total of 12 spokes were commissioned to address these topics, using a common methodology to address two broad questions:

  • What do we know about the topic?; and

  • What are the possible options for the Australian context?

The first question focuses the systematic review that is synthesised by the research team. The second question requires the researchers to use the results of the review to develop evidence-based recommendations for ways forward for Australia’s primary health care system. The policy options include consideration of funding arrangements (existing and alternative), delivery arrangements and governance arrangements reflecting system-level perspectives.

An overview of the steps, timelines and major activities that comprised Stream 4 is presented in the Box. Within this supplement, we summarise key findings from each spoke on the topics as follows:

Systematic review and interactions between policy advisers and researchers

The disparate nature of much of the published literature required careful consideration of the appropriate methods for systematically reviewing and synthesising such evidence. Stream 4 drew heavily on the Journal of Health Services Research and Policy supplement, “Synthesizing evidence for management and policy-making”,18 to underpin its approaches, particularly the narrative synthesis approach of Mays and colleagues.19

Additional funding was provided to Stream 4 participants to allow their engagement in structured meetings in Canberra on four separate occasions during the 12 months of the program. These served five main purposes:

  1. Facilitating agreement on methodological issues (eg, common approaches to searching for primary health care literature or economic literature, assignment of quality criteria to diverse literature, “stopping” rules to allow a decision to be made that enough material has been obtained);
  2. Minimising duplication of effort through sharing of material and, where more than one spoke was working on a topic, agreeing on how the work of one spoke would complement the work of others on that topic;
  3. Facilitating interactions with policy advisers to provide provisional results to them and to test emerging options for their policy relevance;
  4. Allowing access to international experts to ensure a high standard of review (eg, Nicholas Mays, Professor of Health Policy at the London School of Hygiene and Tropical Medicine, delivered a workshop on systematic review methods); and
  5. Ensuring that progress against stated milestones was achieved so that the results were delivered in a timely fashion.
Presentation of results

The research teams were asked to present their results (preliminary and final) in different ways through the program. The intent was twofold — addressing the concern of policymakers that research results are often not delivered in a timely fashion, and presenting the final reports in a manner that was easily accessible. The structured research components in Canberra allowed presentation of early results, as did the meetings between the individual spokes and their reference groups and interactions with key stakeholders. Throughout this interaction, the independent nature of the process was maintained by careful attention to the scientific method involved in synthesis and critical internal and external review. The final reports were prepared for web-based presentation using the “1:3:25” approach,5 with one page summarising the key take-home messages, three pages providing an overview, and the longer report containing all the information, including full references and appendices where appropriate.

What have we learned?

Most spokes found the systematic review process very demanding. A number of participants had experience with the Cochrane approach to systematic reviews, but did not find this suitable for the kind of literature being surveyed. The volume of potential literature identified in the searches was very large, and making decisions about when to stop searching and how to adjudicate the relevance and weight that should be given to retrieved material was a challenge throughout the process. The result for most spokes was a greater proportion of the 12 months being spent on the review process than had been anticipated at the outset, with a relatively lesser proportion of time iterating potential options with key stakeholders.

Researchers are accustomed to writing for research audiences. Most spokes found the production of the one- and three-page summaries of the options for non-research audiences challenging. Researchers tended to default into research writing mode — for example, qualifying statements in the summary documents to convey the sense of uncertainty around them rather than stating the implications for policy less ambiguously.

While policy advisers played a significant role in priority setting for the research program, participation by policy advisers in the structured sessions in Canberra was more variable. Senior policy advisers usually had unanticipated demands being made on their time and thus were unable to attend. More junior policy advisers were hesitant to offer critical comments from a policy perspective on the material being discussed, and had to balance the commitment of being present for the full day against the other requirements of their roles. Engagement with senior policy advisers was more successful when the separate spokes arranged to meet with individuals outside the structured sessions.

Locating the material on the Internet has made it easily accessible to Australian audiences and, to some extent, international audiences. The number of hits suggested the resources have been useful, and, interestingly, it seems the full reports are most often visited rather than the one- or three-page summaries. Anecdotal accounts suggest that the recent Australian Government intervention in the Northern Territory has resulted in much use being made of the reports by McDonald et al15 and Humphreys et al,13 although this can not be corroborated through analysis of APHCRI website activity. Anticipating future policy challenges in an explicit and timely fashion allows for a repository of relevant research information to be developed.20

What has followed Stream 4?

Because of the largely positive experiences associated with the conduct of Stream 4, the RAB has continued to support the development of the linkage and exchange approach. Stream 6 is repeating the Stream 4 process, with a single focus on addressing the primary health care workforce shortage. Stream 7 provided opportunities for researchers involved in Stream 4 to compete for new linkage and exchange travelling fellowships, allowing Australian primary health care researchers to visit world-renowned international academic primary health care institutions in relevant comparator countries and consider their Stream 4 work in the context of these international settings. On their return, they will provide written reports of their findings and participate in a briefing to policy advisers in Canberra.

What has been the impact of Stream 4?

Improving the quality and effectiveness of primary health care requires the adoption of evidence into policy and practice. Has APHCRI’s research been taken up in policy? As Nutley et al report,21 direct or instrumental use of research findings to shape policy is unusual. Research evidence is only one source of information that policymakers draw upon. APHCRI does not expect to demonstrate direct links between its research programs and subsequent policy. However, it does expect to contribute to the policy processes through use of its research to assist with conceptualisation of issues and to mobilise support for key reforms. Conceptual use is illustrated by the provision of succinct summaries of relevant information or provision of new ways of framing issues or gaining further insights into the strengths and weaknesses of different options — all illustrated in the articles of this supplement. APHCRI’s Stream 4 program has contributed to debate and raised public discussion of crucial issues confronting Australia’s health system. An example of this is the contribution APHCRI’s Stream 4 program and related activities have made to mobilising support for discussion of the health system reforms necessary to meet the challenges posed by chronic disease.

Conclusion

APHCRI’s development of the linkage and exchange approach through its Streams 4, 6 and 7 has been positively received by the primary health care research community. The RAB will continue to develop and implement refinements to this approach, with a view to enhancing the uptake of evidence in policy.

Steps, timelines and major activities within Stream 4, Australian Primary Health Care Research Institute (APHCRI)

Steps and timeline

Major activity focus


Setting priority research topics through iteration with Department of Health and Ageing then decision by RAB (Mar–Apr 2005)

Clarifying national policy relevance and prioritising issues


Call for responses addressing selection criteria (May 2005)

Publication of opportunity in national press, through established email networks and on APHCRI website


Optional information workshop for potential applicants (Jul 2005)

Overview of APHCRI and Stream 4, with workshop presentation freely accessible on the Internet


Assessment of responses by the ERC against selection criteria, with recommendations to the RAB (Aug 2005)

Independent assessment of all applications initially, then meeting of the ERC to arrive at consensus scores and recommendations to the RAB


Decision making by the RAB and commissioning of spokes (Aug 2005)

Further discussion of applications, and identification of any issues needing clarification before announcement of decisions


First research program component meeting in Canberra (Sep 2005)

  • Discussion of overall program

  • Identification and management of potential commonalities/synergies between spokes

  • Development of shared approach to:

    • systematically identifying relevant black literature;

    • classifying studies;

    • assessing strength of evidence; and

    • synthesising results


Research activities

  • Scoping literature

  • Refining questions

  • Establishment of reference groups and stakeholder lists


Second research program component meeting in Canberra (Oct 2005)

  • Further specifying research questions

  • Discussing initial mapping exercise

  • Selecting studies

  • Re-running the searches


Research activities

  • In-depth searches

  • Discussions with reference groups and stakeholders


Additional research program component meeting in Canberra (Jan 2006)

  • Master class in systematic review methods with Nicholas Mays, Professor of Health Policy at the London School of Hygiene and Tropical Medicine


Third research program component meeting in Canberra (May 2006)

  • Key findings against key questions

  • Report on key stakeholders engaged thus far and to be engaged

  • Key learnings from the review process to date

  • 1:3:25* report approach


Research activities

  • Formatting draft reports into a 1:3:25* template and development of options


Fourth research program component meeting in Canberra (Sep 2006)

  • Technical problems with template discussed

  • Provisional options presented and commented on by participants


1:3:25* reports finalised and submitted to APHCRI (Oct–Nov 2006)

  • Editing

  • Standardising presentations

  • Iterating with authors to ensure sense not changed


1:3:25* reports published on APHCRI website (Nov 2006)


ERC = Expert Review Committee. RAB = Research Advisory Board. * The 1:3:25 approach uses one page to summarise the key take-home messages, three pages to provide an overview, and a longer report to give all the information, including full references and appendices where appropriate.5

Received 
30 Sep 2007
accepted 
31 Jan 2008
Nicholas J Glasgow, MD, FRACGP, Professor and Director1
John E Marley, MD, FRCP(Edin), FRACGP, Professor2
Linda J Kristjanson, RN, MN, PhD, Professor and Deputy Vice-Chancellor, Research and Development3
Janette A Donovan, BA, GradDipPubHealth, Consumer Representative4
Sally J Hall, RN, GradCertClinMan, Research Manager, Academic Unit of General Practice4
Mark F Harris, DRACOG, FRACGP, MD, Executive Director and Professor of General Practice5
David M Lyle, MB BS, PhD, FAFPHM, Professor and Head6
Elizabeth J Kerr, BA(Off Mgt), GradDipAET, Institute Manager1
Frith Rayner, BA, GradDipJournalism, Program Coordinator, Communications and Policy Liaison1
1 Australian Primary Health Care Research Institute, Australian National University, Canberra, ACT.
2 Faculty of Health Sciences, University of Queensland, Brisbane, QLD.
3 Curtin University of Technology, Perth, WA.
4 Australian National University, Canberra, ACT.
5 Centre for Primary Health Care and Equity, University of New South Wales, Sydney, NSW.
6 Broken Hill University Department of Rural Health, University of Sydney, Broken Hill, NSW.
Competing Interests: 

The Secretariat of APHCRI book and pay for Janette Donovan’s travel to and from meetings of the RAB.

Reference Text: 
Primary health care research, evaluation and development strategy: phase 2 (2006–2009) strategic plan. Canberra: Australian Government Department of Health and Ageing, 2006. http://www.phcris.org.au/phcred/PHCRED%20Strategic%20Plan%202006-2009.pdf (accessed Feb 2008).
Reference Order: 
1
PubMed ID: 
Reference Text: 
Australian Primary Health Care Research Institute [website]. http://www.anu.edu.au/aphcri (accessed Mar 2008).
Reference Order: 
2
PubMed ID: 
Reference Text: 
Buse K, Mays N, Walt G. Making health policy. Maidenhead, UK: Open University Press, 2005.
Reference Order: 
3
PubMed ID: 
Reference Text: 
Lomas J. Using “linkage and exchange” to move research into policy at a Canadian foundation. Health Aff (Millwood) 2000; 19: 236-240.
Reference Order: 
4
PubMed ID: 
10812803
Reference Text: 
Lavis J, Davies H, Oxman A, et al. Towards systematic reviews that inform health care management and policy-making. J Health Serv Res Policy 2005; 10 (3 Suppl 1): 35-48.
Reference Order: 
5
PubMed ID: 
Reference Text: 
Cranston JM, Crockett AJ, Moss JR, et al. Models of chronic disease management in primary care for patients with mild-to-moderate asthma or COPD: a narrative review. Med J Aust 2008; 188 (8 Suppl): S50-S52. <eMJA full text>
Reference Order: 
6
PubMed ID: 
Reference Text: 
Dennis SM, Zwar N, Griffiths R, et al. Chronic disease management in primary care: from evidence to policy. Med J Aust 2008; 188 (8 Suppl): S53-S56. <eMJA full text>
Reference Order: 
7
PubMed ID: 
Reference Text: 
Jackson CL, Nicholson C, Doust J, et al. Seriously working together: integrated governance models to achieve sustainable partnerships between health care organisations. Med J Aust 2008; 188 (8 Suppl): S57-S60. <eMJA full text>
Reference Order: 
8
PubMed ID: 
Reference Text: 
Mitchell GK, Tieman JJ, Shelby-James TM. Multidisciplinary care planning and teamwork in primary care. Med J Aust 2008; 188 (8 Suppl): S61-S64. <eMJA full text>
Reference Order: 
9
PubMed ID: 
Reference Text: 
Powell Davies G, Williams AM, Larsen K, et al. Coordinating primary health care: an analysis of the outcomes of a systematic review. Med J Aust 2008; 188 (8 Suppl): S65-S68. <eMJA full text>
Reference Order: 
10
PubMed ID: 
Reference Text: 
McDonald J, Harris MF, Cumming J, et al. The implementation and impact of different funding initiatives on access to multidisciplinary primary health care and policy implications. Med J Aust 2008; 188 (8 Suppl): S69-S72. <eMJA full text>
Reference Order: 
11
PubMed ID: 
Reference Text: 
Naccarella L, Southern D, Furler J, et al. Primary care funding and organisational policy options and implications: a narrative review of evidence from five comparator countries. Med J Aust 2008; 188 (8 Suppl): S73-S76. <eMJA full text>
Reference Order: 
12
PubMed ID: 
Reference Text: 
Humphreys JS, Wakerman J, Wells R, et al. “Beyond workforce”: a systemic solution for health service provision in small rural and remote communities. Med J Aust 2008; 188 (8 Suppl): S77-S80. <eMJA full text>
Reference Order: 
13
PubMed ID: 
Reference Text: 
Griffiths KM, Christensen H. Depression in primary health care: from evidence to policy. Med J Aust 2008; 188 (8 Suppl): S81-S83. <eMJA full text>
Reference Order: 
14
PubMed ID: 
Reference Text: 
McDonald EL, Bailie RS, Rumbold AR, et al. Preventing growth faltering among Australian Indigenous children: implications for policy and practice. Med J Aust 2008; 188 (8 Suppl): S84-S86. <eMJA full text>
Reference Order: 
15
PubMed ID: 
Reference Text: 
Hearn LA, Miller MR, Campbell-Pope R. Review of evidence to guide primary health care policy and practice to prevent childhood obesity. Med J Aust 2008; 188 (8 Suppl): S87-S91. <eMJA full text>
Reference Order: 
16
PubMed ID: 
Reference Text: 
Glasgow NJ, Wells R, Butler J, Gear A. The effectiveness of competency-based education in equipping primary health care workers to manage chronic disease in Australian general practice settings. Med J Aust 2008; 188 (8 Suppl): S92-S96. <eMJA full text>
Reference Order: 
17
PubMed ID: 
Reference Text: 
Synthesizing evidence for management and policy-making. J Health Serv Res Policy 2005; 10 (3 Suppl 1): 1-56. http://www.ingentaconnect.com/content/rsm/jhsrp/2005/00000010/b10303s1;jsessionid=16prf4scejw9w.alexandra (accessed Aug 2007).
Reference Order: 
18
PubMed ID: 
Reference Text: 
Mays N, Pope C, Popay J. Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J Health Serv Res Policy 2005; 10 (3 Suppl 1): 6-20.
Reference Order: 
19
PubMed ID: 
Reference Text: 
Maynard A, Bloor K. Primary care and health care reform: the need to reflect before reforming. Health Policy 1995; 31: 171-181.
Reference Order: 
20
PubMed ID: 
10142614
Reference Text: 
Nutley S, Davies H, Walter I. Evidence based policy and practice: cross sector lessons from the UK. Working Paper 9. London: ESRC UK Centre for Evidence Based Policy and Practice, Research Unit for Research Utilisation, 2002.
Reference Order: 
21
PubMed ID: