Changes in pathology test ordering by early career general practitioners: a longitudinal study

Pathology testing is an integral part of clinical practice, for screening, diagnosing and monitoring disease. Clinicians have access to an ever expanding selection of pathology tests, and the volume of ordering has grown markedly in recent years.1,2 The increasing range reflects technological advances and increased understanding of disease processes, and this may underpin an appropriate expansion of test ordering. But inappropriate ordering is a major concern in contemporary health care;1,3-5 not only does it increase the costs of care,6 it can also lead to overdiagnosis and overtreatment.7,8

General practice generates a greater proportion of pathology tests than other specialities and disciplines; in Canada, for example, 58% of total pathology expenditure was generated by primary care physicians.9 Inappropriate pathology testing is a particular problem in general practice.2,10 Compared with other areas of medical practice, the prevalence of definable disease is lower and that of undifferentiated disease correspondingly greater, with consequently lower pre-test probabilities of disease; this reduces the positive predictive value of individual tests and increases the proportion of false-positive test results. This, in turn, leads to overdiagnosis and inappropriate additional testing.8 Further, even true-positive results can be problematic if the findings prove to be incidental,8 or if the evidence for the treatment of the diagnosed condition is unclear.11 Overtreatment, with attendant waste of limited health care resources and exposure of patients to iatrogenic harm (including psychological harm), may ensue.8

Australian data indicate that general practitioners often do not adhere to evidence-based consensus guidelines when ordering pathology tests.2 In Canada, GPs are also much more likely than specialist physicians to order non-evidence-based testing at periodic health examinations.10 Differences in testing rates between countries,12 regions,6,13 individual general practices,14 and individual GPs10,13 are marked, and seem to be more attributable to practitioner preferences than to demographic or other factors.10,15

Early career GPs and GPs-in-training are of particular interest in this respect, as they are at a career stage in which they establish patterns of clinical behaviour that may be long lasting. They have moved from a hospital environment with a high prevalence of serious illness to one of low prevalence. They often have difficulty tolerating diagnostic uncertainty and selecting investigations for undifferentiated disease.16 It is therefore unsurprising that Australian general practice registrars order more tests than more established colleagues,17 especially given that registrars have the same test ordering rights as senior GPs.

Time and experience in a general practice environment might be expected to attenuate the volume of test requests by general practice registrars. To test whether this is the case, we analysed longitudinal data on their test ordering behaviour in Australia.

Methods

We conducted a longitudinal analysis of ten rounds of data collection (2010–2014) for the Registrar Clinical Encounters in Training (ReCEnT) study. ReCEnT is an ongoing, multicentre, cohort study of general practice registrars from five of 17 regional training providers (RTPs), from all Australian states except Western Australia.

ReCEnT documents the clinical experiences of general practice registrars, including pathology test ordering; its methodology has been described previously.18 Briefly, registrars record on paper forms the details of 60 consecutive consultations during the midpoint of each of their three compulsory general practice training terms (ie, every 6 months for full-time registrars, every 12 months for part-time registrars). Some registrars at one of the five RTPs (RTP 1) also recorded data during an optional fourth training term. Only office-based consultations, not home visits or nursing home visits, are recorded. As data collection is intended to reflect a normal week of general practice, consultations in a specialised clinic (eg, vaccination clinic) are excluded. Registrar and practice demographic characteristics for each collection period (training term) are also documented. Data collection and reflection about feedback reports on their individual results are integral to the registrar education program, and registrars may also consent to their data being used for research purposes.

Analyses

Each RTP contributed two to ten rounds of data collection, depending on when they started participating in ReCEnT.

The outcome variable was the number of pathology tests ordered per consultation. Registrars record all pathology tests ordered in each consultation, to a maximum of 12 individual tests; if more than 12 are ordered, participants are asked to record the 12 most clinically relevant. The main independent variable of interest was our measure of time for this analysis, the registrar training term (Terms 1–4).

Other covariates included:

• registrar variables: age, sex, full-time or part-time status, country of primary medical training (Australia v other).

• patient variables: age, sex, Indigenous status, non-English-speaking background status, the patient being new to the practice, the patient being new to the registrar.

• practice variables: Australian Standard Geographical Classification–Remoteness Area (ASGC-RA) classification19 (major city, inner regional, outer regional, remote, very remote) and socio-economic status (Socio-Economic Indexes for Areas [SEIFA] Index of Relative Socio-economic Disadvantage decile20 of the postcode of the practice), billing policy (bulk-billing v no bulk-billing).

• consultation variables: duration of the consultation, whether imaging tests were ordered, the number of problems managed, whether a chronic disease was managed in the consultation. Problems and diagnoses were coded according to the International Classification of Primary Care, version 2 (ICPC-2);21 whether a diagnosis or problem was a chronic disease was coded according to a classification derived from the ICPC-2.22 We also recorded whether the registrar sought in-consultation advice or information from their supervisor or other sources, including specialists, books or electronic resources.

Statistical analysis

The outcome was a count variable with 13 response levels (0–12 tests ordered). A negative binomial model was fitted in SAS 9.4 (SAS Institute) to account for overdispersion in the outcome distribution. In a second analysis (for comparison with the negative binomial model), a zero-inflated negative binomial model was fitted in Stata 13.1 (StataCorp). In addition to allowing for overdispersion, this analysis also models variables with large numbers of zero counts. The assumption is that excess zero counts are generated by a process distinct from that which generates non-zero values, yielding two distributions that are modelled independently. These models produce two sets of parameter estimates: one set of estimates for the count outcome, and one set for predicting membership in the “certain zero” category. Parameter estimation was performed within the generalised estimating equations (GEE) framework to account for repeated measures on registrars.

In order to assess potential temporal changes in the number of tests ordered, a time variable was included (training term). Variables were selected for inclusion in the multivariable model by modelling each covariate, adjusted for time, to test for a univariate association with the outcome. Covariates for which P < 0.20 and with a non-negligible effect size (ie, not close to 0) in the univariate analysis were considered for inclusion in the final multivariable model. Covariates which had a small effect size and were no longer significant (P > 0.05) in the multivariable model were tested for removal from the model; if removal did not cause a change in effect size greater than 10% or a change in statistical significance, the covariate was removed from the final model.

The GEEs accounted for the repeated measures on registrars, but sensitivity analyses tested for the following types of clustering:

• clustering by registrar only;

• clustering by registrar nested within practice; and

• clustering by registrar nested within term.

The results of these analyses were compared with the primary model to assess their impact on conclusions based on the primary model.

Ethics approval

Ethics approval was granted by the Human Research Ethics Committee of the University of Newcastle (reference, H-2009-0323).

Results

A total of 876 registrars (response rate [ie, proportion who consented to their data being used for research purposes], 96.4%) contributed 2017 registrar–rounds of data, including the details for 114 584 individual consultations. The demographic characteristics of the participating registrars and practices are presented in Box 1; the most frequently ordered tests are listed in Box 2.

No pathology tests were ordered in 87 885 consultations (77%); at least one was ordered in 26 699 consultations (23%; Box 3). The associations between individual practitioner, practice and patient characteristics and the number of tests ordered (adjusted for time) are included in the online Appendix, table 1.

Nesting by registrar within practice and registrar within term did not cause any substantial changes from clustering by registrar alone. The effect size for each covariate varied between the different models by less than 2% (data not shown). It was concluded that clustering by registrar within the time covariate (registrar term) was adequate for the final multivariable model.

In the negative binomial model, the number of pathology tests ordered by registrars increased significantly with time, by 11% for each additional training term (95% confidence interval [CI], 8–15%; P < 0.001). The covariates “registrar full-time or part-time status”, “registrar sex” and “country of primary medical qualifications” had small effect sizes and were no longer significant in the model, and were removed from the multivariable model; as this did not substantively alter it, these covariates were not included in the final model (Box 4, Box 5). In the zero-inflated negative binomial analysis, the effect of time was smaller but still significant, with an estimated increase in the number of pathology tests ordered by registrars per training term of 4% (95% CI, 1–7%; P = 0.016; (Appendix, table 2).

Discussion

Time and experience in the typical general practice environment of relatively low prevalence of serious disease (compared with the hospital environment) had been expected to attenuate the rate of test ordering during general practice registrar training. It might also be expected that specific training in rational test ordering would have been included in GP vocational training, and thereby also reduced ordering rates. Contrary to our expectations, however, general practice registrars ordered more pathology tests as their time in training increased. Given their costs and the implications for patient wellbeing and safety, the 11% increase in ordering for each additional training term was highly significant in clinical as well as in statistical terms. The smaller increase (4% per term) found in our secondary, zero-inflated negative binomial analysis was also highly clinically significant.

Our finding that ordering increased as registrars moved through training appears to conflict with evidence that general practice registrars in Australia order more tests than established GPs.17,23 One relevant factor is the degree of autonomy or responsibility of the registrar in Australia. Although they are vocational trainees, general practice registrars have a large degree of clinical independence, including when ordering tests. They have recourse to advice from clinical supervisors (in an apprenticeship-like model), but this assistance decreases markedly with each training term.24 In our analysis (as in a previous cross-sectional analysis of this population17), seeking information (including supervisor advice) was associated with lower levels of test ordering. It is possible that the uncertainty inherent to increasingly independent decision making by more senior registrars (ie, with less supervisor assistance) leads to more ordering, so that the relationship of rational test ordering (ie, targeted, evidence-based ordering) with career stage may exhibit a J-shaped curve, with a peak in overall ordering during late vocational training and early post-vocational training practice before again declining with greater experience.

More experienced registrars may treat patients with more complex disease, especially more complex chronic disease, and this might be associated with an appropriate increase in the ordering of tests. Earlier analyses of ReCEnT data suggest, however, that this is not the case.25,26 Similarly, more experienced registrars may adhere more closely to clinical guidelines and protocols, but the guidelines are as likely to recommend parsimony in pathology testing as to prompt appropriate ordering of tests.

Strengths and limitations of our study

Our longitudinal approach allowed us to assess temporal changes in test ordering during training. The large number of registrars who delivered detailed contemporaneous records of individual clinical consultations provided us with fine-grained data with which we could adjust our analyses for several demographic, clinical and educational variables. The study population drawn from five of 17 regional vocational training providers in different geographic settings across five of six states increases the potential generalisability of our results to other Australian registrars, and the participation rate was higher than for other studies of GPs.27

The principal limitation of our study was that we did not have data on the appropriateness of individual test ordering decisions. Tests were ordered in 23% of consultations, and in 10% of consultations only one was ordered; of these, at least some (for example, Pap smears) will have been appropriate. We interpreted our findings, however, in the light of the volume of literature describing a high prevalence of inappropriate testing.5,10,15

A further limitation is that registrars recorded a maximum of 12 individual pathology tests ordered per consultation (ie, we may have underestimated the number of requested tests); but, as this maximum was recorded for only 1.4% of consultations, this will not have had a large effect on our results.

Implications for practice and policy

An 11% increase in test ordering per term of training has implications for health care costs.6 It may also have implications for patient wellbeing and safety if, as seems likely, higher pathology testing rates entail increased inappropriate testing, elevating the probability of overdiagnosis and overtreatment.1,4,5,8 Even were the effect to decline as early career GPs gained experience and became more comfortable with diagnostic uncertainty, its impact would persist for a considerable period.

Targeted, multifaceted interventions that encourage rational test ordering should incorporate several components. It would be appropriate to better support registrars during this difficult period in their clinical development. Educational interventions in other settings have been effective in reducing test ordering by changing the practice system or environment, to encourage behavioural change in clinicians.28 Specific approaches could include developing and applying clinical practice guidelines that modify the behaviour of clinicians and patients and the organisation of health care processes.3 Effective interventions in primary care have included guideline dissemination and education,29,30 modifying laboratory request forms,29 small group quality improvement meetings,30 and feedback on test request rates.30 Educating GPs about estimating the pre-test probability of disease when considering a test, and on how to respond to patient expectations,31 may also be useful. As seeking in-consultation information or advice is associated with lower rates of ordering, structural changes to the supervisory model, including support for greater supervisor in-consultation advice during later training terms, may be indicated.

Conclusion

We found that test ordering by general practice registrars increased during vocational training. While the lack of information about the appropriateness of individual tests restricts the interpretation of this finding, the increase causes concerns about overtesting. Registrars need to be supported during a difficult period in the development of their clinical practice patterns.

Box 5 – Least squares mean estimates of number of pathology tests ordered per consultation, by registrar term*

---

* Modelled as the logarithm of the count, with least squares mean estimates (and 95% confidence interval endpoints) exponentiated for plotting on the original count scale, by term. The numbers of registrars for each term were 783, 682, 477 and 75 respectively; the total number of tests were 50 287, 35 253, 29 419 and 3495 respectively.