Justin Beilby and Chris Silagy
(I) Referees' comments
Referee 1 (Mr Roy Harvey, Principal Research Fellow, Centre for Health Services Development, University of Wollongong)
Overall Assessment
The subject matter is probably worthy of an article but I'm not sure that even with a re-write this article would be suitable for publication. It may be better to expand the search and re-focus the paper to "strategies for changing prescribing and ordering of diagnostic tests".
[Authors' response: The aim of this paper is to look at whether providing information on costs would change behaviour. We have added two new references (5,6) (second paragraph in the introduction) to the paper. These references look at the general area of "strategies for changing prescribing and ordering of diagnostic tests" and we did not want to repeat this work.]
General Comments
I am surprised that only 17 articles were found by the literature search, and, am surprised how few of these relate to GPs - only 6 by my count (based on Table 1). Even with these 6, there were a number of confounding factors that make the attribution of the changes observed attributable to the cost information difficult.
[Authors' response: We have redone our search and believe that we have found all relevant articles. We also aimed to include all strategies that included costs (see hypotheses - Methods section.) As we found it is unlikely that providing costs alone is adequate. This finding is important to proving or disproving the hypotheses.]
The term "efficiency" was used several times. Its use raises issues about the role of clinicians in controlling costs that the paper should explore if it is to be used. These issues are wrapped up in the assumption that reducing prescribing and test ordering was a contribution to efficiency. Similarly, the paper assumed that issues relating to increasing costs of pharmaceutical costs, and costs of pathology and diagnostic imaging were the same - I don't think they are.
[Authors' response: There was some confusion over the term efficiency. As a consequence we decided to refocus the paper on behaviour change with resultant cost savings in order to remove this confusion. As such we have changed the title to:"Providing costing information to general practitioners - will this intervention change behaviour and create cost savings?"]
In the Discussion section a number of additional references about changing prescribing practices are discussed. These presumably did not meet the search criteria.
[Authors' response: The additional references in the Discussion (references 39,40 - Paragraph 6 in Discussion) do not include strategies involving feedback of costing information. We have however, expanded this paragraph to include a brief summary of the intervention used. We feel this provides a useful contrast.]
1) Focus of article
i) appropriateness of changes: Sentence 2 of para 1 of the Introduction says that the increases in pathology, diagnostic imaging and community pharmaceutical costs (PBS costs?) are "largely general practitioner driven". The paper takes it for granted that reducing the rate of use, or the rate of increase is a good thing to do. The paper implies that these increases are not appropriate - some justification should be given as to the appropriateness or otherwise of the level of costs and these increases.
[Authors' response: This is an important point. We have added paragraph 1 to the Introduction which describes in more detail the increase in diagnostic imaging and pathology ordering and the substantial variation in prescribing. We believe this provides justification for concerns around the rising rates of test ordering and prescribing.]
ii) factors driving increases. The "growth" in PBS costs has been mainly due to increases in prices of PBS drugs - not to a large increase in prescribing volume. The increases in diagnostic imaging and pathology have resulted from increases in the quantity of services ordered mainly by GPs. There are two different issues (raised in (i) above) that are not explored in the paper but that are relevant to the types of strategy. First, if a GP prescribes the highest cost drug it may provide a patient with the highest gain in health, but not necessarily the most cost effective gain if lower cost drugs are available that provide (almost) equal benefit. This raises the important issue of the role of individual clinicians in making judgements about the relative cost-effectiveness of drugs, and indirectly, in becoming involved in cost control measures. Second, issues relating to ordering diagnostic imaging and pathology tests is concerned with the appropriateness of ordering. If the issues of kickbacks is ignored then encouraging the appropriate ordering of these tests is essentially an educational matter. The relevance of "providing information about the costs for (services)" to these quite different circumstances should be teased out in the article.
[Authors' response: We have added a comment in paragraph 2 of the Discussion and paragraph 1 of the Introduction about the issue of increases in PBS costs being due to the use of more expensive medications as well as increases in rates of prescribing.
The issue of appropriateness of ordering has been mentioned in paragraph 1 in the Introduction. The issue of the role of clinicians in being directly involved in cost control methods and assessment of cost effectiveness is discussed in two places - paragraph 3 in the Introduction and the last paragraph of the Discussion. This is an important point and the reviewer was correct in asking us to tease this out.]
2) Efficiency. In a number of places the article uses the term "efficiency" when it really should say "reduction in prescribing or ordering of tests" or something similar. In the Conclusion in the Abstract "efficiency" is used to mean reduction in costs, while in the Introduction it talks about wanting to "facilitate debate on efficiency" but does not discuss what it considers "efficiency" to be. It is an important issue and, as suggested above (1, i) leads to consideration of the important issues of the role of clinicians in cost control vs their roles to act in their patients' interest.
[Authors' response: See second point and reply under General Comments above.]
3) Conclusion (in Abstract). This asserts that "the review has indicated . . . ... in all service areas". In view of the fact that only 6 of the articles were relevant to GPs, and as a number of the situations in which there was provision of cost information there was also a wide range of other activity (mentoring, sharing of costs, etc), I think that the evidence to support the conclusion is not strongly based, especially in view of the other factors included in the multifaceted studies.
[Authors' response: While we agree that the search strategy only found 6 articles that were relevant to GPs we would argue that successful interventions aimed at other primary care doctors are likely to be useful for GPs (see paragraph 1 - Discussion).]
4) Assumption in Introduction. Paragraph 2 of the Introduction says "it is hard to see this (more efficient practice) occurring if they do not know or understand the costs generated as a consequence of their decisions". Again this is an issue that could be further explored. Is it necessary for all clinicians to become aware of all cost issues - the relative cost-effectiveness of all the drugs, pathology tests, etc - that they might be able to prescribe or order?. Also, the issues of pharmaceuticals and pathology and diagnostic imaging need to be distinguished.
Pharmaceuticals. Some of the US managed care models, and the PBS, consider factors of effectiveness and cost effectiveness when deciding to list items on their formularies or on the PBS. If GPs trust these processes (the scientific skills and ethical skills of the groups that make recommendations to list drugs) they should be able to focus on clinical issues of appropriateness for the patient, safety efficacy etc. Alternatively, if appropriately qualified groups develop prescribing guidelines, then is it enough for a GP to follow the Guidelines or does the GP need to be provided with cost and cost-effectiveness information as well?
Diagnostic imaging and pathology tests. The issues here are different to those raised in prescribing. Appropriate ordering of these tests should be determined by the value of the information they provide in diagnosing or monitoring a patient's health. Again, can information be provided in Guidelines without the GP knowing the detailed costs.
[Authors' response: We have discussed the first issue in the last paragraph in the Discussion. We agree that providing costing information on all services to all GPs may not be an effective use of resources. Secondly, in the fifth paragraph of the Discussion, we have mentioned the need to distinguish between prescribing and diagnostic imaging and pathology ordering.]
5) Magnitude of effect. There is a section on Magnitude of effect. The paper gives no indication of what was the magnitude of the +++ to + scale that it developed.
[Authors' response: The issue about the Magnitude of effect created confusion with all reviewers. We have decided to remove Table 2 and the paragraph relating to it. We feel removal of this material does not detract from the review and allows the qualitative element of the paper to be more focused.]
6) Discussion. "Efficiency" is used (para 1) instead of cost reduction.
[Authors' response: See second point and reply under General Comments above.]
The Discussion introduced a wide range of additional material. In paragraph 3 a number of papers which do not fit search criteria for this paper are introduced but not discussed at length.
[Authors' response: We have expanded reference 39,40 (see paragraph 6, Discussion). The original paragraph refers to the Academic detailing model which seems self explanatory. As such we have not expanded these references.]
The Discussion stated that "pharmaceutical costs contribute 9.6% of the health care budget" but did not say that only about half of this was for prescribed drugs, and that some of this was specialist not GP prescribing. Diagnostic imaging and pathology constituted about 27% (1994-95) of Medicare Benefits payments and hence make up about 5% of total health expenditure. So it is not clear the prescribing is the first issues of concern.
[Authors' response: We have added a comment (Discussion para 4) on the impact of pathology and diagnostic imaging ordering - i.e. "4.7% of the health care budget".]
I did not find the last paragraph very helpful - more research needs to be done on "GP remuneration, managed care and fund holding, improved doctor-patient communication and changes to the malpractice legislation". Providing some specific focus would be more helpful.
[Authors' response: We have removed this last paragraph and added a new one - see also response to Assumption in Introduction above.]
7) Clarity: Paragraph 1 is unclear: the first sentence refers to an increase in "benefits paid" - for GP consultations or all GP consultations and ordered services?
[Authors' response: In the first sentence of the Introduction we have added "paid for GP consultations".]
This paper seeks to undertake a systematic review of the literature on providing costing information to general practitioners and whether this intervention will improve efficiency. The abstract however gives the impression that this article contains empiric rather than review data.
Scientific content and basis of manuscript
This study is a qualitative, systematic review of the literature from 1980 to 1995 in computerised databases and citations on the objectives of the study. All of the papers cited are quite old, the most recent is 1990.
[Authors' response: We have rerun our search strategy and believe we have included all relevant papers. A comment has been added in the first paragraph of the Discussion emphasising this point.]
The authors whilst defining the study criteria have not identified a systematic method for the review nor a reference group. It is not clear from the Method what were the inherent biases of the group undertaking the study.
[Authors' response: We believe that systematic method is described - see the first and second paragraph of the Method. We are unclear what "inherent biases of the group" means but our belief was that costing information alone was unlikely to change behaviour. However we do not believe this needed to be stated within the paper.]
The objectives as stated in the Method are different from the objectives stated in the Abstract and those identified in the last paragraph of the Introduction. As a result the reader is left with a great deal of uncertainty as to the real aims of the researchers. The authors should make it clear whether they are looking at efficiency or effectiveness. They should also clarify their assumption that general practitioner behaviour changes lead to changed health outcomes.
[Authors' response: We have already discussed this issue in response to Referee 1. The objectives in the Method, the second paragraph into the Introduction and the objectives stated in the Abstract all focus on the effect of providing information about costs of pharmaceutical pathology and diagnostic imaging to general practitioner/primary care physicians to change behaviour.]
The methodology of the paper is also marred by an inappropriate mix of qualitative and quantitative strategies. Whilst the authors suggest that the paper is essentially qualitative, Table 2 attempts to quantify the information by providing a derived measure called "magnitude of effect". There is limited discussion of this derived indicator and certainly no justification for its inclusion.
[Authors' response: We have removed Table 2 and the information related to the Table from the Results (see also response to Referee 1)]
The continued growth in costs of pharmaceuticals and diagnostic test should be of concern to all who participate in the health care system. It is important that rational scientific approaches to promoting appropriate use of pharmaceuticals and diagnostic tests be tested. The undesirable alternatives include further administrative restrictions on prescribing and test ordering or non-selective capping of these budgets. This paper is a good succinct systematic review of the evidence that feedback to clinicians on volume and costs may reduce the number of drugs and tests ordered. The effect appears to be larger when multi-faceted approaches are used.
The methodology for the review is scientifically adequate. The source of the papers is adequately described. The authors have placed a quality filter on the types of information they have included by restricting it to specific study types. Non-randomised experimental designs are included.
However, other than the study design, the paper contains limited discussion on the quality of the studies. In particular, in relation to the generalisability of the results. It would be useful to know the participation rates in the studies. No comment is made on whether study design influenced the size of the observed effect although this can be judged from the tabled data.
[Authors' response: A comment has been added in the Results section dealing with the quality of the studies. "The number of participants for each study varied from 24 (27) to 435 (17). The participation rates were over 90% in all studies" - first paragraph Results section.]
Each study is scored on a three-point scale as to the size of the effect (Table 2) but nowhere are the criteria for substantial, moderate or minor effect size defined.
[Authors' response: See responses to this point in Referee 1 and Referee 2 sections above]
A concern of many clinicians is that such strategies may result in inappropriately under testing or prescribing. While the authors indicate they looked for papers that examined impact on health outcomes, no comment is made on this issue. It is very likely that none of the papers reported on this, in which case this point should be commented on in the Discussion.
[Authors' response: A paragraph has been added in the Discussion expanding on the important point -- impact on health outcomes(second last paragraph).]
The paper is easy to read and the two tables, although large, are necessary. The paper should be of broad interest given that similar approaches have been proposed in Australia.
[Authors' response: We agree with the comments but have removed Table 2 (see above).]
Six of the studies reviewed involve general practitioners and the rest "hospital doctors". As the modus operandi of doctors in these two environments are markedly different the Committee wants the literature search expanded to involve all health care professionals (MDs) or to focus on GPs only.
[Authors' response: The article is now focused on general practitioners alone]
Article - Register to be notified of new articles by e-mail - Papers for review list - ©MJA 1997