Connect
MJA
MJA

Self-audit as part of a research governance framework for health research

Bradley R Crammond, Anna V Parker, Megan Brooks, Marina Skiba and John J McNeil
Med J Aust 2011; 194 (6): 310-312.
Published online: 21 March 2011

Clinical research has been an area of increasing activity for hospitals, universities and research institutions in Australia, rising from 1960 trials in 2000 to 3208 trials in 2009 (Therapeutic Goods Administration, personal communication). To manage the increasing risks for both patients and institutions, strategies to better monitor research governance are now receiving greater attention.

The National Health and Medical Research Council (NHMRC) National statement on ethical conduct in human research1 requires that individual institutions take responsibility for monitoring clinical research conducted at their institutions. One of the most common methods used to monitor research activities is the annual progress report required by human research ethics committees (HRECs). The report includes information about compliance with the approved protocol, as well as maintenance and security of records. Some institutions also undertake random inspections of research sites, data or consent documentation.

Developing the self-audit

To promote attention to ethical conduct and compliance with principles of good research practice,2 the Department of Epidemiology and Preventive Medicine (DEPM) at Monash University and the Alfred Research and Ethics Unit (Alfred) in Melbourne introduced research audits in 2004. The DEPM and the Alfred collaborated to introduce a variety of monitoring strategies, including annual progress reports and detailed audits.3 Other components of the DEPM program included appointing a research governance officer, establishing a research governance committee, developing a manual of clinical and public health good research practice, and introducing an induction program for new staff.

However, although the detailed audits were effective, they proved time consuming and resource intensive (a detailed audit takes about 3 days to complete, and each year it is possible to examine only about 10% of the total number of active projects).

To improve the institutional oversight of research, the DEPM and the Alfred subsequently adopted a tiered approach to audits. Detailed audits were supplemented with short audits that can be completed in less than a day and a self-audit (Box 1), which is completed by researchers themselves, rather than by governance or ethics staff. The self-audit highlights to researchers the issues they need to keep in the forefront of their minds while conducting research and helps them identify any areas where they may not be meeting the expected standards.

The strategy is based on the premise that researchers are keen to conduct their research to the highest standard, provided they know what is required of them. Other health services have since adopted self-audits, and the Victorian Managed Insurance Authority has incorporated them into its Research governance toolkit, which it developed to assist all Victorian public entities conducting clinical research.3

To evaluate whether the self-audit has been an effective tool for highlighting and promoting good research practice, the DEPM and the Alfred designed a short, online questionnaire. The questionnaire was sent out on 3 July 2008, with responses received over the following 8 weeks. Our article reports the findings.

Completing and evaluating the self-audit
The DEPM

An email was sent to all DEPM staff, inviting those directly responsible for the conduct of a research project to complete the self-audit form. In most cases, the most appropriate person to complete the form was the principal investigator. If the principal investigator was not able to complete the form due to time or other constraints, he or she was encouraged to ask someone else with relevant project knowledge to complete the self-audit. The email also included a link to an online version of the form.

To ensure that the audit was being completed by an appropriate person, the respondent’s name and responses were emailed to the research governance officer after it was completed.

At the same time, the respondent was redirected to an online survey where he or she was invited to complete an evaluation questionnaire of the self-audit.

The Alfred

The Alfred adopted a slightly different approach. The Alfred Hospital Ethics Committee amended its annual progress report to include a question asking the researcher to complete the self-audit, attach a copy of the audit to the annual report at the time of submission, and file a copy in their study files.

An invitation to complete the online evaluation questionnaire was emailed to research personnel with current projects and advertised in the Alfred Ethics Committee Newsletter.

The evaluation

The evaluation was conducted and analysed anonymously using an online survey tool (SurveyMonkey, Palo Alto, Calif, USA). The evaluation questionnaire sought information in three main areas:

  • whether the text and substance of the self-audit were easy to understand and appropriate to the research being performed

  • whether participants believed the audit would result in their making changes to their projects or whether it would improve the ethical quality of their research

  • whether the participants thought the self-audit required follow-up to be useful and whether they discussed the audit with the principal investigator.

Ethics approval

Approval for our research was obtained from the Standing Committee on Ethics in Research Involving Humans at Monash University and the Alfred Ethics Committee.

Evaluation findings

At the time of the evaluation, there were 165 projects being conducted at the DEPM, overseen by 56 principal investigators. Twenty-seven principal investigators or their delegates (48%) completed the self-audit and, of those, 24 completed the evaluation (43% of principal investigators; 89% of those who completed the audit).

At the Alfred, invitations were distributed to about 340 researchers with active projects. Thirty-three researchers completed the evaluation but many of these were responsible for multiple projects.

Results of the evaluation are shown in Box 2. Across both sites, most respondents (92% at the DEPM and 97% at the Alfred) believed that the self-audit promoted good research practice.

One of the aims of the self-audit is to encourage people to think about new ethical issues. However, fewer than half of the participants (25% at the DEPM and 40% at the Alfred) identified new ethical issues as a result of using the self-audit. Our results may have been influenced by selection bias among respondents. Those who were already interested in good research practice were more likely to have completed the questionnaire, making it more likely that the evaluation records an interest in the area but less likely that it raises new issues.

Discussion

Auditing research is one means by which irregularities can be prevented or discovered before damage is caused.4 This role is recognised by the NHMRC national statement and equivalent international documents (Box 3).

Ongoing HREC oversight of projects has increased in the past decade, with many institutional HRECs now requiring annual progress reports. However, there remains an important and separate institutional role in ensuring that the research conducted conforms to high standards of good research practice.7 The self-audit, completed by principal investigators and project coordinators, provides feedback to researchers concerning the adequacy of their research procedures. It is part of a series of research governance initiatives designed to make an explicit statement to researchers about the standards expected in the institution and to fulfil a preventive role in alerting all those involved in clinical research to the dangers of poor research practice.

It is envisaged that the most important role of the self-audit will be in helping to create a culture that is conducive to the responsible conduct of research. A 2008 study reported alarmingly low familiarity of researchers with the major research governance documents,4 including the NHMRC national statement and the Australian code for the responsible conduct of research.8 The authors suggested that a mandatory training program in research governance along with research auditing would improve researcher knowledge. Indeed, the Australian code requires institutions to train researchers in responsible research conduct.

Although the self-audit cannot address the breadth and depth of issues possible in a formal training course, it can provide a simple checklist encompassing the most important governance principles for researchers to follow. In so doing, it can help to educate researchers about their research responsibilities while at the same time performing an auditing role.

To date, the key difficulty in implementing the self-audit has been encouraging researchers to complete the audit without intervention or follow-up from governance staff. This has been achieved at the Alfred by including the self-audit form with the documents that the ethics committee requires annually to ensure ongoing project approval. The DEPM plans to introduce a similar compulsory system.

Although the self-audit is not intended to replace more comprehensive audits, it appears to be a promising means of reminding researchers of their obligations and encouraging self-monitoring of research activities without requiring the prohibitive resources needed by more extensive audits.

1 Self-audit process at the Department of Epidemiology and Preventive Medicine and the Alfred Research and Ethics Unit

  • The self-audit is designed principally as an educational tool, providing a quick check for researchers to determine if their research activities meet the expected high standard of good research practice. It also serves as an aid to the research governance officer by identifying projects or departments that warrant a more detailed assessment.

  • The self-audit asks researchers whether their research could easily be continued without them, whether they are meeting their legal and ethical obligations, and whether data and other important information are being stored appropriately. If a researcher is not able to answer “yes” to a question, he or she must seek assistance from the research governance officer.

  • A member of the study team completes the self-audit form annually. This person needs to have intimate knowledge of all aspects of the project. He or she is instructed to complete and sign the form and then discuss the results with the principal investigator (if the principal investigator is not completing the audit himself or herself) and remedy any deficiencies identified. The principal investigator must then sign the form and return it to the research governance officer.

2 Evaluation questionnaire results: participants answering “yes”, by question

DEPM (n = 24)

Alfred (n = 33)


Are the audit questions easy to understand?

20 (83%)

31 (94%)

Are the audit questions relevant to promoting good research practice?

22 (92%)

32 (97%)

Do you plan to make changes to your research practice in response to the audit?

10 (42%)

10 (30%)

Does the audit improve the ethical quality of your research?

15 (63%)

18 (55%)

Did the audit lead you to think of ethical issues you had not previously considered?

6 (25%)

13 (39%)

Did you need to discuss the questions on the audit with your principal investigator?

0

13 (39%)

Does the self-audit require follow-up from ethics staff to be useful?

11 (46%)

11 (33%)

Do you rate your knowledge of ethics compliance as good or excellent?

21 (88%)

27 (82%)


DEPM = Department of Epidemiology and Preventive Medicine.
Alfred = Alfred Research and Ethics Unit.

3 Examples of audit requirements in Australia, the United Kingdom and the United States

National Health and Medical Research Council — National statement on ethical conduct in human research1

5.1.17 Institutions should have in place an auditing process to confirm that:

(a) research in their institution is being reviewed at the levels of review their criteria require

UK Department of Health — Research governance framework for health and social care5

5.2 ... Mechanisms to monitor the quality of clinical work, such as inspection, audit, risk management and staff appraisal, can assist in the monitoring of research governance. Coherent systems are needed to monitor performance, to identify best practice and shortfalls, to enhance public confidence and help to prevent adverse events

US Office of Management and Budget Circular A-133 — Audits of states, local governments, and other non-profit organizations6

§ ___ .200 Audit requirements

(a) Audit required. Non-Federal entities that expend ... $500 000  ... or more in a year in Federal awards shall have a single or program-specific audit conducted for that year

Received 24 May 2010, accepted 1 September 2010

  • Bradley R Crammond1
  • Anna V Parker2
  • Megan Brooks1
  • Marina Skiba1
  • John J McNeil1

  • 1 School of Population Health and Preventive Medicine, Monash University, Melbourne, VIC.
  • 2 Research and Ethics Unit, Alfred Health, Melbourne, VIC.


Competing interests:

John McNeil is Chair of the Alfred Ethics Committee.

  • 1. National Health and Medical Research Council; Australian Research Council; Australian Vice-Chancellors’ Committee. National statement on ethical conduct in human research. Canberra: NHMRC, 2007.
  • 2. Poustie S, Taylor DM, Forbes AB, et al. Implementing a research governance framework for clinical and public health research. Med J Aust 2006; 185: 623-626. <MJA full text>
  • 3. Victorian Managed Insurance Authority. Research governance toolkit. Melbourne: VMIA, 2010. http://www.vmia.vic.gov.au/~/media/Content-Documents/Risk-Management/Clinical-Risk-and-Trials/Clinical-Trials/Guidebooks-Tools-and-Templates/research-governance-toolkit.ashx (accessed Feb 2011).
  • 4. Babl FE, Sharwood LN. Research governance: current knowledge among clinical researchers. Med J Aust 2008; 188: 649-652. <MJA full text>
  • 5. UK Department of Health. Research governance framework for health and social care. 2nd ed. London: Department of Health, 2005.
  • 6. US Office of Management and Budget Circular A-133. Audits of states, local governments, and other non-profit organizations. Washington, DC: OMB, 2003.
  • 7. Walsh MK, McNeil JJ, Breen KJ. Improving the governance of health research. Med J Aust 2005; 182: 468-471. <MJA full text>
  • 8. National Health and Medical Research Council; Australian Research Council; and Universities Australia. Australian code for the responsible conduct of research. Canberra: NHMRC, 2007.

Author

remove_circle_outline Delete Author
add_circle_outline Add Author

Comment
Do you have any competing interests to declare? *

I/we agree to assign copyright to the Medical Journal of Australia and agree to the Conditions of publication *
I/we agree to the Terms of use of the Medical Journal of Australia *
Email me when people comment on this article

Responses are now closed for this article.