Funding Australia's basic biomedical research of 1993 and 1994

The meshing of two databases of scientific publications -- the Wellcome Trust's Research Outputs Database, and the Research Evaluation and Policy Project's database of Australian publications -- allows a detailed analysis of the funding agencies providing external (as opposed to intramural) support for Australia's basic biomedical research. This analysis shows the success Australian researchers are having in attracting funding from overseas, and the high citation rates achieved by publications with external funding.

Linda Butler

MJA 1999; 171: 629-633

Introduction - Characteristics of basic biomedical research funding - Comparison of funded and unfunded publications - The impact of funded research - Conclusions - References - Authors' details
Make a comment - Register to be notified of new articles by e-mail - Current contents list - More articles on Economics

Data analysing this question, particularly Australian Bureau of Statistics (ABS) data, are aggregated at a national or sectoral level. There is a paucity of data on the performance of individual funding bodies, and funding for basic fundamental research and applied clinical observation is rarely distinguished.

In 1998, the Research Evaluation and Policy Project (REPP) undertook a consultancy for the National Health and Medical Research Council (NHMRC), examining funding acknowledgements and citation performance of Australian biomedical research publications appearing in journals indexed by the Institute for Scientific Information (ISI).² The ISI indices have been shown to provide very good coverage of the basic research output of scientists in universities and research institutes.³ In contrast, ISI's coverage of the output of scientists concentrating on the clinical application of basic research, particularly in hospitals, is less robust, as a considerable proportion of this research is published in the "grey literature" -- non-peer-reviewed journals, monographs, reports, and the like.

The REPP study, with its use of ISI data, focuses more precisely than other studies on funding of basic research. In addition to information in its own extensive bibliometric database, REPP drew on the Wellcome Trust's Research Outputs Database (ROD) covering the funding acknowledgements of ISI-indexed articles.

The "funding acknowledgements" in research publications (which are usually explicit but sometimes implicit) are to external sources of funding (as opposed to intramural funding obtained through the internal recurrent budget of the authors' affiliated institutions).

Here, I examine some of the main findings of the REPP report on funding acknowledgements and performance,² and contrast Australian funding sources with those available to biomedical researchers in the United Kingdom.⁴ The methodology is described below.

Number of funding acknowledgements
The distribution of articles according to number of funding acknowledgements was compared for Australian and similar UK publications (Figure 1). Australian publications mirror UK publications, although a slightly larger proportion of Australian publications did not list any funding source (38.4% v 33.8%), and a smaller proportion listed only one. Of those that did acknowledge external funding, 60% listed more than one source.

Funding sectors
The ROD database identifies 11 funding sectors, which have been aggregated into three major sectors to allow direct comparison with published UK data. Table 1 shows the distribution of acknowledged funding across these major sectors and the comparative UK data.

Researchers in Australia are more reliant on government sources for external funding than their UK counterparts. A closer analysis of the industry sector shows that the difference between Australia and the United Kingdom lies principally in pharmaceutical industry funding: 11.5% of UK publications acknowledge support from the pharmaceutical industry, while, in Australia, the comparative figure is only 6.1%, of which only 20% comes from Australian companies.

Funding countries
The countries of the funding bodies acknowledged in Australian publications are shown in Table 2 (only countries acknowledged in at least 50 Australian publications are included); 17% of Australian publications with funding acknowledgements did not list an Australian funding body.

To examine this in more detail, Figure 2 shows the proportion of publications with funding sources from more than one country (classified according to whether the list of countries included or excluded Australia), and those that received all their funding from a single country.

Only 56% of publications with funding acknowledgements relied entirely on Australian funding bodies. In some cases, funding from another country would be the result of Australian researchers collaborating with colleagues from that country, but this is not invariably the situation. For example, in the case of the United Kingdom, an examination of addresses shows that fewer than half the funding acknowledgements can be directly associated with collaborative effort. The impetus for obtaining funds from Switzerland appears to be an indication of Australia's lack of a sizeable pharmaceutical industry and its reliance on other countries as a source of funds in this area: 55% of all acknowledgements to Swiss funding bodies (109 of 193 publications) are to pharmaceutical companies.

Funding bodies
The funding bodies assisting biomedical researchers in Australia are many and varied. Table 3 lists the most commonly acknowledged Australian and international bodies.

This Table indicates that more than three-quarters of all basic research publications acknowledge external sources of funding, while only 30% of publications in the more applied journals do so. Funding for the latter research is most likely from the recurrent budgets of hospitals and universities.

At first glance, it would appear that the medical research institutes are particularly successful at attracting external funding. However, the first section of the Table shows that the level of external funding is closely linked with the level of the research undertaken, and medical research institutes focus heavily on basic research. Conversely, the hospital sector focuses primarily on more applied research and a lower proportion of funded publications would be expected.

The funding rates attributable to different levels of research also need to be borne in mind when analysing data on ASRC subfields. Lower funding rates would be expected for fields at the more applied end of the research spectrum, such as clinical sciences and health services research.

Funding sector
To gauge the relative impact of research funded by the different Australian sectors, the measure of actual and expected citation rates is used (Figure 3). The 11 sectors identified by the ROD database are shown (with the exception of "Other").

Figure 3 shows that two Australian sectors demand closer attention. The pharmaceutical industry sector is funding research that is appearing in very high impact journals, but these publications are achieving citation levels significantly below the expected level. The non-profit sector, in contrast, is achieving a level of citation above the expected level. All other sectors are aligned closely to the diagonal line, denoting actual citation rates very close to expected rates. Level of research and field of research will influence a sector's position along the x-axis in this graph, as both greatly influence the expected citation rate.⁹ But, at face value, all except the pharmaceutical industry sector have been successful in identifying research projects that are also judged well by the journal community.

Funding bodies
In Figures 4 and 5, the impact of the support by research funding bodies is shown. Figure 4 includes all international funding bodies mentioned in at least 100 Australian publications. This result could be anticipated, as it is unlikely that Australian researchers would succeed in winning grants ahead of researchers in the funding body's own country unless they had proved their research to be at the forefront of their field.

Figure 5 shows the strong performance of most major Australian biomedical funding bodies. The performance of NHMRC-funded publications is analysed in detail in the full report.²

Highly cited publications
In Table 5 the proportion of publications in the top 1% and 5% bands acknowledging external funding is contrasted with the proportion of total publications that acknowledge funding in each biomedical subfield. Table 5 also lists the number of citations required to place a publication in these bands.

For all subfields, over three-quarters of the publications with the highest citation impact acknowledge external funding. In all instances, the proportion acknowledging funding is higher for these high impact journals than for all publications in the subfield. In Clinical Sciences the discrepancy is particularly large; only 53.1% of all publications acknowledge external funding sources, but 81.7% of the top 5% of the most highly cited publications (and 88.9% of the top 1%) contain funding acknowledgements.

These data appear to be an endorsement of peer review in both the grant-giving and journal-publishing systems. Funding bodies are identifying and supporting those projects which "succeed" in the judgement of other journal authors.

Without knowledge of the scale of an individual grant in relation to the actual costs of the project, it is difficult for funding agencies (or studies of data from acknowledgements) to draw definite conclusions from publication acknowledgements, as they are an imperfect guide to the relationship between funding and outcomes. Nevertheless, carefully studied and classified patterns of acknowledgement provide a general map of the nexus between the evaluation processes of grant agencies and the outcomes of their decisions.

Data sources listing individual publications arising from a specific grant may prove a more reliable guide; however, such data sources are rare and usually limited to small funding agencies. REPP is investigating one such source -- the information supplied on applications for NHMRC large grants -- and, in time, these data may be able to verify how accurate funding acknowledgements data are as a surrogate for precise attribution of journal publications to funding sources.

Finally, granting agencies have a stake of their own in aligning themselves with "winners". All granting agencies have boards, ministries and other bodies scrutinising how they spend their funds, and there are strong imperatives for granting bodies not only to create "winners", but also to get involved with existing "winners". Accordingly, while the good relationship between the success of researchers in securing external grants and their visibility in the international journal literature may reflect the success of peer selection processes, this may also be evidence of shrewd investment by granting bodies.

1. Wills PJ (Chairman). Health and Medical Research Strategic Review. The virtuous cycle -- working together for health and medical research. Canberra: Department of Health and Aged Care, 1999: 28.
2. Butler L, Biglia B, Bourke P. Australian biomedical research: funding acknowledgments and performance. Canberra: NHMRC, 1998.
3. Bourke P, Butler L. Monitoring research in the periphery. Canberra: Research Evaluation and Policy Project, Australian National University, 1996. (Monograph Series No. 3.)
4. Dawson G, Lucocq B, Cottrell R, Lewison G. Mapping the landscape: national biomedical research outputs 1988-95. London: Wellcome Trust, 1998. (Policy Report No. 9.)
5. Jeschin D, Lewison G, Anderson J. A bibliometric database for tracking acknowledgements of research funding. In: Koenig M, Bookstein A, editors. Fifth International Conference of the International Society for Scientometrics and Informetrics; 1995 Jun 7-10; Rosary College, River Forest, Ill, USA. Medford, NJ: Learned Information Inc, 1995.
6. Australian Bureau of Statistics. Australian Standard Research Classification 1993. Canberra: ABS, 1993. (Catalogue No. 1297.0.).
7. Lewison G, Dawson G, Anderson J. The behaviour of biomedical scientific authors in acknowledging their funding sources. In: Koenig M, Bookstein A, editors. Fifth International Conference of the International Society for Scientometrics and Informetrics; 1995 Jun 7-10; Rosary College, River Forest, Ill, USA. Medford, NJ: Learned Information Inc, 1995.
8. National Board of Employment, Education and Training. Quantitative Indicators of Australian Academic Research. Canberra: AGPS, 1994. (Commissioned Report No. 27.)
9. Narin F. Evaluative bibliometrics: the use of publications and citation analysis in the evaluation of scientific activity. Cherry Hill. NJ: CHI Research Inc, 1976.

Reprints: Ms L Butler, Research Evaluation and Policy Project, Research School of Social Sciences, The Australian National University, ACT 0200.
linda.butlerATanu.edu.au

©MJA 1999
Make a comment

Home | Issues | eMJA shop | Terms of use | Classifieds | More... | Contact | Topics | Search

The Medical Journal of Australia    eMJA

Readers may print a single copy for personal use. No further reproduction or distribution of the articles should proceed without the permission of the publisher. For permission, contact the Australasian Medical Publishing Company.
Journalists are welcome to write news stories based on what they read here, but should acknowledge their source as "an article published on the Internet by The Medical Journal of Australia <http://www.mja.com.au>".
<URL: http://www.mja.com.au/> © 1999 Medical Journal of Australia.
We appreciate your comments.