Design: Cost–benefit analysis of teaching in private general practice across three levels of training — undergraduate medical training, prevocational training and general practice vocational training — using data from a 2007 survey of general practitioners in South Australia.
Results: The net financial outcome of teaching varied across the training levels. Practices incurred a net financial cost from teaching medical students that was statistically significantly different from zero. With respect to vocational training and teaching junior doctors, there were small net financial benefits to practices, although the mean estimates were not statistically significantly different from zero.
Conclusions: This study shows a net financial cost for practices teaching medical students, while at the prevocational and vocational training levels, adequate levels of subsidies and income generated by the trainees help offset the costs of teaching. Our results suggest that a review of subsidies for undergraduate teaching is necessary, particularly as the demand for teaching practices will increase substantially over the next 5 years.
As in many countries, Australia has increasing numbers of medical students,1 prevocational junior doctors and postgraduate vocational trainees who will place increasing demands on existing training posts in hospitals and general practice. For general practice, the increasing demand is likely to have a significant impact. Unlike public hospital settings, teaching in general practice uses private businesses within a fee-for-service arrangement, which affects the care provided, as well as space, staff resources and general practitioners’ time and income.2-4 This makes it different to other teaching environments, such as public hospitals, where supervisors are salaried.
In Australia, about 1305 of 9600 general practices are involved in teaching medical students, and 1933 are involved in general practice vocational training.5 With an estimated 3000 medical students, 900 interns and over 600 registrars requiring placement in general practice each year by 2012, training positions will fall well short of requirements unless more teaching sites are recruited or larger numbers of trainees are placed at each site.5
Some financial compensation for the impact of teaching on general practices and general practice supervisors is provided through various government schemes, but some of these subsidies have been documented as being inadequate.6,7 Many subsidies have been based on historic costing that may not reflect all aspects of teaching.
Some research has been undertaken on the financial costs of teaching in general practice, but mainly with a focus on the impact that medical students have on a supervisor’s productivity.3,8-11 None of these studies have placed a dollar value on this impact. Some studies have investigated the costs associated with an entire training program, but not at a practice level.12-15 To our knowledge, no studies have looked at the financial benefits associated with teaching in general practice, nor investigated the costs and benefits of teaching in private general practice across the whole medical training continuum. We aimed to identify the financial costs and benefits associated with teaching medical students, junior doctors (postgraduate year 1 and postgraduate year 2 or 3) and general practice registrars in private general practice.
Data on the time associated with teaching were obtained from a 2007 questionnaire-based survey of GPs who taught medical students, junior doctors and general practice registrars within their private general practices. Details of the survey method are provided elsewhere.6 Briefly, the Adelaide to Outback GP Training Program (AOGP)16 surveyed 325 GPs in 102 urban, outer metropolitan, rural and remote practices located in its training region. A response rate of 60% was achieved from general practice supervisors and 75% from general practices. A general practice supervisor is a GP who undertakes the primary teaching role in a practice (hereafter referred to as a “supervisor”). Also evaluated was the traditional model of teaching, involving one-on-one teaching by a supervisor within a practice — for example, a supervisor teaching a fourth-year medical student, junior doctor at postgraduate year 1 level (intern), or general practice registrar at basic level.
A cost–benefit analysis was performed on all levels of training in private general practice within the AOGP training region in South Australia and in association with the Discipline of General Practice at the University of Adelaide. Applying a practice perspective, costs were calculated by quantifying the resources used to teach and assigning a specific unit cost to these resources. The benefits associated with teaching were calculated using data collected on the income generated by trainees and the teaching subsidies received. For the medical student level, the analysis only applied to students placed in general practice through the undergraduate medical course provided by the University of Adelaide. The length of these placements varied with the year of study.
The study was overseen by a steering group, consisting of supervisors with experience in teaching at all levels investigated, who validated the methods and results. The study was approved by the University of Adelaide Human Research Ethics Committee.
Financial costs were grouped into four categories: administrative activities; teacher upskilling; direct teaching activities; and other costs, such as infrastructure (Box 1).
Administrative costs included staff and supervisor administration time. The time supervisors spent at events focused on upskilling their teaching skills was determined by obtaining the mean estimated time spent on these activities from the supervisor questionnaire. Direct teaching activities included preparation time for teaching and the additional time that teaching added to a supervisor’s session. The latter included formal teaching and “corridor” or opportunistic teaching. Costs associated with GP time spent on teaching activities were based on the Australian Medical Association method of calculating a GP’s salary, which is based on the number of practice services per hour and includes a bulk-billing fee. This approach provided an annual income after expenses that fell between lower and higher rates reported by others.18 By adapting the approach further to include gap fees, it accommodated practices that were entirely bulk-billing as well as those that applied bulk-billing to specific patient groups only.
Infrastructure costs included “room rental forgone” and accommodation costs (rental rate). Room rental forgone is the cost of a fully equipped consulting room with administrative support that is used for teaching rather than renting to another medical consultant. An average rate for the rent of a room to a medical consultant (GP or other specialist) was obtained from a sample of rural and urban practices. This cost applied to all teaching levels, although for medical students, only a proportion of practices placed students in their own consulting room, therefore a weighting was applied based on the data from the survey. For some practices (rural), cost of renting a house for the junior doctor and general practice registrar was included and based on an average weekly rental rate.
Financial benefits included in the analysis were: subsidies for teaching (teaching allowance and practice subsidy); teacher upskilling subsidy; rental subsidy; and the practice component of income generated by the trainee, where applicable (Box 1). The rate of these teaching subsidies and payments varied between the training levels, and the funds were provided by different programs. For example, the Practice Incentives Program payments for teaching were dependent on the number of sessions the medical students spent in the practice. For junior doctors and general practice registrars placed in a rural practice, a rental subsidy is provided by the AOGP.
To attribute an income for the junior doctors and general practice registrars, data on the mean number of patients seen during a placement were obtained from data collected weekly during each placement by the AOGP. To account for the different workloads of full-time and part-time registrars, weighted estimates were generated based on number of sessions worked, with all data converted to full-time equivalent. All junior doctors worked full-time. These data were then converted to a dollar value using the formula for costing supervisor time. For both junior doctors and general practice registrars, the practice retained a proportion of the income that these trainees generated. These estimates were determined using information from the questionnaire and input from the steering group. For junior doctor training, the Prevocational General Practice Placements Program funds 100% of the intern salary but none of the post-intern salary.
Means for each financial cost and benefit category were estimated within each of the eight stages of the three teaching levels, which were then summed to estimate the net financial outcome of teaching. To analyse the uncertainty around the mean estimates, confidence intervals were estimated for parameters describing the time spent teaching and administering students and the number of patients seen per week by vocational trainees. Bootstrap samples were generated to determine the confidence intervals, selecting at the respondent level rather than the individual item level, because the times for the various activities are likely to be correlated.
The robustness of the results of the cost–benefit analysis was tested by one-way sensitivity analysis as well as probabilistic sensitivity analysis. The one-way analysis involved setting the input values for each teaching time parameter and the mean number of patients seen per week to their lower and upper 95% confidence interval values.
For the probabilistic sensitivity analysis, given the nature of the parameters — bounded by zero and with a positive skew — a log-normal distribution was first used to generate the confidence intervals. However, due to a large proportion of zero values and missing data, a normal distribution was used to avoid non-convergence of bootstrap samples. The defined distributions for these parameters informed a Monte Carlo analysis of each financial model, in which 1000 sets of values for the input parameters were sampled and used to calculate separate estimates of the financial outcomes for each trainee stage. The 2.5th and 97.5th percentile values from the 1000 estimates were used as the 95% confidence limits about the mean estimate of net financial outcome for each trainee stage.
The weekly financial costs and benefits associated with teaching medical students, junior doctors and general practice registrars in general practice (by stage within each of these training levels) are shown in Box 2.
For teaching medical students, there was a net financial cost for practices that was statistically significantly different from zero. For teaching junior doctors and general practice registrars, there were small net financial benefits to a practice, although the mean estimates were not statistically significantly different from zero. Within each of the three training levels, the net cost or net benefit varied according to the stage of training. For example, at the medical student level, the greatest significant net financial cost to practices for teaching was for fourth-year medical students (− $1385 per week; 95% CI, − $1746 to − $1100) and the least was for sixth-year medical students (− $630 per week; 95% CI, − $907 to − $400). The largest net financial benefit came from teaching interns ($484 per week; 95% CI, − $536 to $1026), although this estimate was not statistically significantly differently from zero.
The means and 95% CIs generated by the probabilistic sensitivity analysis for the net costs and benefits of teaching at different training levels are shown in Box 3. For all medical student stages, we can be certain that the net financial outcomes are negative. For general practice registrars, there is a high probability that the net financial outcomes are positive. The most uncertainty lies around the net financial outcome of teaching postgraduate year 1 and postgraduate year 2 or 3 trainees, where the net financial outcomes are more evenly distributed between positive and negative values.
The one-way sensitivity analysis indicated that for all stages of medical students, interns and general practice registrars, the net financial outcomes were most sensitive to the estimates of additional time per session. However, the effects on the net financial outcomes of applying the estimated 95% confidence limits for the additional time parameters were not large, indicating that the mean results are robust. For postgraduate year 2 or 3, the estimated net financial outcomes were highly sensitive to additional time per session.
These results show that the net financial outcome in a general practice of one supervisor teaching one type of trainee varies between training levels (medical student, junior doctor or general practice registrar). There was a significant net financial cost for teaching medical students, but there were net financial benefits for teaching junior doctors and general practice registrars.
Not surprisingly, the greatest cost driver at all levels was direct teaching activities. This finding complements those of other studies,3,9,11,19,20 which have found that teaching added time to patient consultations and total time spent at work, and reduced the number of patients seen by the supervisor. How a practice organises its teaching may reduce the impact that teaching has on consultation length. It has been reported that the use of a parallel consulting model for students does not significantly increase a rural GP’s consultation length.21 However, only a few studies have determined a monetary value for teaching time, included other activities related to teaching (such as teacher upskilling and administrative time) or looked beyond medical student training.3,10,11 Two studies found few differences in practice income if students were present or not,10,20 while another found that third-year medical students, in terms of supervisor time, cost $73.80 per day.11 To our knowledge, our study is the first to determine the financial costs and benefits associated with teaching in private general practice across the whole training continuum. Our costing analysis shows that interns were the most costly to teach when compared against all the training stages and levels investigated. However, these high costs were offset by commensurate benefits (subsidies and income generated by interns) for the practice. In contrast, at the medical student level, the benefits received by practices (subsidies) did not offset the costs.
These results have implications for the recruitment and retention of general practices as teaching environments, particularly for medical student teaching. Our results provide evidence for a review of these arrangements and an exploration of methods for recognising the accurate teaching costs for GPs, most specifically undergraduate medical teaching. One possible option for this level of teaching is to introduce a graduated subsidy rate, reflecting the varied costs of teaching within the stages of medical student training, as occurs at the prevocational and vocational training levels. Our results may also have implications for the involvement of other private health providers in teaching — for example, similar issues may arise in private hospitals and private allied health practices, particularly in light of the Health Workforce Australia funding for clinical training placements.22
Our study has a number of limitations. The data relate only to teaching in practices based in the AOGP region of SA. However, the profile of the practices that participated in the study suggests they are representative of practices across Australia. Additionally, the rates and subsidies were based on those used by the AOGP, and these may vary between training regions. The costing analysis was based on data self-reported by supervisors, so estimated time spent on teaching activities may be overestimated or underestimated, and this is a key limitation of our study. The scope of our study did not allow for formal triangulation of the results using a time-and-motion method, where these times are documented by observation. However, the results were reviewed by the study’s steering group and, based on their experience, assessed as valid. Data on numbers of patients seen by registrars were checked through a review of practice records for a number of registrars. We also reviewed the data to identify outlier results and, where possible, confirmed these with the respondent. Finally, coefficients of variation for the individual items assessing time were calculated and there were no major discrepancies regarding differences in variability across training levels.
This study only investigated the costs associated with medical undergraduate training and did not include graduate-entry medical students, therefore direct comparisons between universities may not be possible. Also, no distinction was made for subgroups within medical undergraduate training, such as rural cohorts where placement length differs and additional subsidies are available to practices. Even so, the financial model developed for this analysis could be adjusted to take these variations into account. Similarly, the number of practices involved in prevocational doctor training is limited at a national level, and linked to funding available for this training. However, SA has the largest number of intern positions in this program, with all the Prevocational General Practice Placements Program practices in the AOGP region involved in such training since at least 2005. Finally, other intangible benefits or costs of teaching for the supervisor were not included in the analysis. While these were identified in the survey,6 it was not possible to place a monetary value on these for this study, but this is worthy of consideration in the future.
Our study has shown there is a marginal net financial gain to general practices involved in prevocational and vocational training but not for those involved in undergraduate medical student training. A major contributor to these results appears to be the low level of teaching subsidies paid to practices for teaching medical students. It may be time to review the subsidies paid to teach medical students to ensure that, at the very least, a cost-neutral financial outcome is achieved. It may also be worthwhile exploring the net financial outcome of different models of teaching within general practice.
1 Details of unit costs and benefits and data sources used to analyse net financial outcomes of teaching in private general practice, 2007
2 Mean costs and benefits of teaching in private general practice per week at 2007 prices, by training level and stage*
- 1. Joyce CM, Stoelwinder JU, McNeil JJ, Piterman L. Riding the wave: current and emerging trends in graduates from Australian university medical schools. Med J Aust 2007; 186: 309-312. <MJA full text>
- 2. Walters LS, Worley PS, Prideaux D, et al. The impact of medical students on rural general practitioner perceptors [sic]. Rural Remote Health [internet] 2005; 5: 403. Epub 2005 Nov 11.
- 3. Vinson DC, Paden C, Devera-Sales A. Impact of medical student teaching on family physicians’ use of time. J Fam Pract 1996; 42: 243-249.
- 4. Usatine RP, Nguyen K, Randall J, Irby DM. Four exemplary preceptors’ strategies for efficient teaching in managed care settings. Acad Med 1997; 72: 766-769.
- 5. Pearce R, Laurence CO, Black LE, Stocks N. The challenges of teaching in a general practice setting. Med J Aust 2007; 187: 129-132. <MJA full text>
- 6. Laurence CO, Black LE. Teaching capacity in general practice: results from a survey of practices and supervisors in South Australia. Med J Aust 2009; 191: 102-104. <MJA full text>
- 7. Australian Association of Academic General Practice. PIP general practitioner teaching payment: is it time for improvement? Submission to the Red Tape Taskforce. AAAGP, 2003.
- 8. Worley PS, Kitto P. Hypothetical model of the financial impact of student attachments on rural general practice. Rural Remote Health [internet] 2001; 1: 83. Epub 2001 Mar 2.
- 9. Grayson MS, Klein M, Lugo J, Visintainer P. Benefits and costs to community-based physicians teaching primary care to medical students. J Gen Intern Med 1998; 13: 485-488.
- 10. Fields SA, Toffler WL, Bledsoe NM. Impact of the presence of a third-year medical student on gross charges and patient volumes in 22 rural community practices. Acad Med 1994; 69 (10 Suppl): S87-S89.
- 11. Ricer RE, Van Horne A, Filak AT. Costs of preceptors’ time spent teaching during a third-year family medicine outpatient rotation. Acad Med 1997; 72: 547-551.
- 12. Murray E, Jinks V, Modell M. Community-based medical education: feasibility and cost. Med Educ 1995; 29: 66-71.
- 13. Pugno PA, Gillanders WR, Lewan R, et al. Determining the true value of a family practice residency program. Fam Pract Manag 2000; 7: 39-42.
- 14. Blewett LA, Smith MA, Caldis TG. Measuring the direct costs of graduate medical education training in Minnesota. Acad Med 2001; 76: 446-452.
- 15. Carlisle R. Financing and budgeting of community-based family medicine residency programs. South Med J 2006; 99: 576-578.
- 16. Adelaide to Outback GP Training Program [website]. http://www.adelaidetooutback.com.au (accessed Aug 2009).
- 17. Sullivan C. 2007 salary survey — the results. Melbourne: Australian Association of Practice Managers, 2007. http://www.qip.com.au/uploads/files/C60000EB_Lunch_Seminar_Session_-_AAPM_Salary_Survey.pdf (accessed Oct 2008).
- 18. Kron J. Working hard for the money. Aust Doctor 2007; 27 Jun: 39-42. http://www.australiandoctor.com.au/news/3a/0c04d33a.asp (accessed Oct 2008).
- 19. Australian Government Department of Health and Ageing. General practice in Australia: 2004. Canberra: DoHA, 2005.
- 20. Vinson DC, Paden C. The effect of teaching medical students on private practitioners’ workloads. Acad Med 1994; 69: 237-238.
- 21. Walters L, Worley P, Prideaux D, Lange K. Do consultations in rural general practice take more time when practitioners are precepting medical students? Med Educ 2008; 42: 69-73.
- 22. Health Workforce Australia. Programs. http://www.hwa.gov.au/programs (accessed Jun 2010).
Publication of your online response is subject to the Medical Journal of Australia's editorial discretion. You will be notified by email within five working days should your response be accepted.