Skin cancer is the most common cancer in Australia: over 250 000 people are diagnosed with non-melanoma skin cancer (NMSC) and over 8000 with melanoma annually.1,2 This extremely high incidence makes skin cancer the most costly of all cancers to treat in Australia.3,4
Diagnosing skin cancer can be difficult. In primary care settings, sensitivity of clinical examination for diagnosing skin cancer has been reported to range from 40% to 80%.5,6 Diagnostic accuracy for pigmented lesions can be considerably lower. In one Australian study, the “number needed to treat” (ie, number of pigmented lesions excised for each confirmed melanoma) was reported to be about 29.7
General practitioners in Australia provide primary care and act as gatekeepers to specialist services, traditionally diagnosing and managing most skin cancers without referral.8,9 This has changed recently, with the rapid emergence of “skin cancer medicine” as a subspecialty within primary care, particularly in Queensland.10 This change has been controversial, mainly around issues such as diagnostic performance and appropriate management within skin cancer clinics.11,12 To date, only one small study of skin cancer clinics has examined diagnostic accuracy13 and another has reported on billing data.14
To address this gap in the evidence base, we undertook a prospective study of the casemix of patients with skin lesions presenting to primary care practitioners working in skin cancer clinics and in general practice. Focusing specifically on excised or biopsied skin lesions, our aim was to compare the diagnostic accuracy of clinicians working in the two settings. We did not address the issues of false negative results after a skin examination or of the adequacy of excision or recurrence of skin cancer after excision.
Our study, conducted in 2005, involved mainstream GPs and skin cancer clinic doctors in south-eastern Queensland. Ethical approval was obtained from the Behavioural and Social Sciences Research Ethical Review Committee of the University of Queensland.
GPs. Two hundred GPs in south-eastern Queensland were randomly selected from the Australasian Medical Publishing Company (AMPCo) database. They were sent a letter inviting participation in the study, an information sheet, a consent form and a reply-paid envelope. Non-responders received a reminder letter 2 weeks later, then a telephone call after a further 2 weeks if there was still no response.
Skin cancer clinic doctors. Using telephone listings, advertisements, the AMPCo database and the Internet, we identified 51 potentially suitable skin cancer clinics in the south-eastern Queensland area. Those eligible for our study were contacted using the same method as for GPs. Doctors working within skin cancer clinics are primarily vocationally trained GPs who have elected to subspecialise in skin cancer medicine, either in addition to or instead of general practice.
We collected data on age, sex, year of graduation, location of training, number of years worked in skin cancer clinics or as a GP, number of sessions per week, Royal Australian College of General Practitioners fellowship status, additional training in skin cancer (such as seminars, workshops, in-house courses within skin cancer clinics, dermoscopy courses), and equipment used to aid diagnosis (dermatoscopes, digital imaging).
To ensure sufficient numbers of lesions for analysis, we collected data from GPs over two 8-week periods (a total of 16 weeks). As the volume of skin examinations within skin clinics was known to be higher than in general practice,13 we collected data from skin cancer clinic doctors over two 4-week periods (a total of 8 weeks). Data were collected on a rolling basis during March–May and September–November.
For lesions excised or biopsied, doctors provided a clinical diagnosis and used five-point scales to rate both the likelihood of malignancy (1 [“very unlikely”] to 5 [“very likely”]) and the degree of patient pressure to excise (1 [“no pressure”] to 5 [“strong pressure”]). The case report form was matched with the histopathology report for each excised or biopsied lesion. Histopathology information included procedure date, body site and histological diagnosis. Case report forms and, where appropriate, histopathology forms were collated by trained research assistants at the practice and allocated a unique number. Multiple lesions from a single patient were numbered separately. To ensure completeness and accuracy of the data, the study team regularly visited the practices.
For analysis, clinical and histological diagnoses were categorised into broad groups: melanoma; basal cell carcinoma (BCC); squamous cell carcinoma (SCC), including intraepidermal carcinoma or Bowen’s disease (SCC-in-situ) and keratoacanthoma; solar keratosis; dysplastic naevus; benign naevus; other pigmented lesions (lentigines, ephelides and seborrhoeic keratosis); other benign lesions (skin tags, dermatofibroma, and cysts); and other malignant lesions. Where multiple diagnoses were recorded for a single lesion, malignant diagnoses were accorded pre-eminence over pre-malignant or benign diagnoses.
We examined the frequencies of clinical and histological diagnoses and made standard bivariate comparisons between GPs and skin cancer clinic doctors. In our analysis of diagnostic accuracy, we included only those lesions for which both a clinical and a histological diagnosis (the latter being the “gold standard”) were available (97.5% of all excisions or biopsies). We compared clinical with histological diagnoses separately for the major diagnostic groups, using measures including sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV).15 We also calculated the number needed to excise (NNE), defined as the ratio of all lesions (both benign and malignant) excised to the number of malignant lesions excised (see Box 1 for explanation of terms).
All analyses took into account the potentially changed variance associated with the sample design, specifically the possibility of correlation between assessments by the same clinician, and assessments by clinicians in the same practice. Logistic regression models were used to estimate each measure of diagnostic accuracy (Box 1). Initially, a constant-only logistic model was fitted, restricting it to either skin clinic doctors or GPs. The parameter estimate for the constant was then transformed (eβo/[1+eβo]) to estimate the diagnostic accuracy for that doctor group. This is equivalent to fitting a risk difference model (using the identity link function).
An additional model (including both GP and skin cancer clinic doctors) with a variable indicating doctor type was then used to assess whether doctor type was significantly associated with diagnostic accuracy. Statistical significance was assumed at the 0.05 level. All analyses were performed using Stata statistical software, version 9.2 (StataCorp, College Station, Tex, USA).
Of the 200 GPs originally selected, seven were ineligible (four could not be traced, and three were no longer in practice). Of the remaining 193 eligible GPs, 39 refused, 47 did not respond and 107 consented. Three withdrew before data collection, leaving 104 participating GPs (54% of the original sample).
Of the 51 skin cancer clinics initially identified, 15 were ineligible for our study (four were no longer in business, nine were part of a general practice and two were staffed by dermatologists). Of the 36 remaining eligible clinics, six refused, one did not respond and two initially consented but withdrew before data collection. The final group consisted of 27 skin cancer clinics (75% participation rate), representing 50 doctors.
There were no apparent differences in demographic or other characteristics between participating and non-participating skin cancer clinics or doctors, except female GPs were significantly more likely to participate than male GPs (P < 0.001).
Skin cancer clinic doctors were significantly younger, on average, than GPs (mean, 45 years v 50 years, respectively; P = 0.002); were predominantly male (84.0% in skin cancer clinics v 57.7% in general practice; P < 0.001); and were more likely to have undertaken additional training (including in-house training) in skin cancer diagnosis (P < 0.001). Skin cancer clinic doctors worked fewer sessions per week — an average of 6.7 sessions (median, 7.0), compared with 8.0 sessions (median, 8.0) among GPs (P = 0.002). Compared with GPs, skin cancer clinic doctors were significantly more likely to use dermatoscopes (P < 0.001) and digitised imaging (P < 0.001) (Box 2).
During the study period, GPs conducted 8790 skin examinations over a total of 1305.5 full-time equivalent (FTE) weeks (mean, 6.7 examinations per week). Skin cancer clinic doctors conducted 19 965 skin examinations over 236.5 FTE weeks (mean, 84.4 examinations per week). Patients of skin cancer clinics were more likely to be male than female and were younger, on average, than patients consulting GPs. Skin examinations performed in skin cancer clinics were more likely to involve the whole body than those done by GPs (Box 3).
A total of 11 403 skin lesions were excised or biopsied from 28 755 skin examinations. Over 16 weeks, GPs performed 3231 excisions or biopsies (mean, 2.5 procedures per week): 2391 excisions (74.0%), 807 biopsies (25.0%), 23 re-excisions (0.7%) and 10 procedures of unspecified type (0.3%). Over 8 weeks, skin cancer clinic doctors performed 8172 excisions or biopsies (mean, 34.6 procedures per week): 3832 excisions (46.9%), 4212 biopsies (51.5%), 70 re-excisions (0.9%) and 58 procedures of unspecified type (0.7%). For about a quarter of lesions, patients exerted significant pressure to excise (Box 3).
Clinical and histological diagnoses were available for 11 116 lesions (3175 for GPs and 7941 for skin cancer clinic doctors). The 3175 skin lesions excised/biopsied by GPs included 743 BCCs (23.4%), 704 SCCs (22.2%), 308 solar keratoses (9.7%), 340 benign naevi (10.7%) and 49 melanomas (1.5%). A similar distribution was seen for the 7941 skin lesions excised/biopsied by skin cancer clinic doctors, which included 2701 BCCs (34.0%), 1274 SCCs (16.0%), 1129 solar keratoses (14.2%), 709 benign naevi (8.9%) and 103 melanomas (1.3%).
Measures of sensitivity and specificity varied across diagnoses (Box 4). The highest sensitivities were achieved for BCC, with GPs correctly diagnosing 79% (95% CI for sensitivity, 0.75–0.82) and skin cancer clinic doctors 89% (95% CI for sensitivity, 0.87–0.90). Sensitivity for a diagnosis of SCC was lower than for BCC for both groups (0.69 and 0.67 for GPs and skin cancer clinic doctors, respectively). For SCCs and BCCs combined (NMSC), over 90% of lesions were correctly diagnosed by both groups (sensitivity 0.92 for GPs and 0.94 for skin cancer clinic doctors) (Box 4). Sensitivity for diagnosing melanoma was significantly higher for skin cancer clinic doctors compared with GPs (0.60 v 0.29, respectively) (P < 0.01), although estimates were based on small numbers of lesions (49 for GPs and 103 for skin cancer clinic doctors). Measures of specificity were similar between GPs and skin cancer clinic doctors for each of the major diagnostic groups. Specificities of 0.98 were recorded for both melanoma and benign naevi.
GPs and skin cancer clinic doctors excised a similar number of lesions per malignancy (NNE for all malignant lesions, 2.1 [95% CI, 1.9–2.3] and 1.9 [95% CI, 1.8–2.1], respectively). Skin cancer clinic doctors excised significantly fewer lesions per BCC than GPs (NNE for BCC, 2.9 [95% CI, 2.7–3.2] and 4.3 [95% CI, 3.8–4.8], respectively; P = 0.05). However, GPs excised fewer lesions per SCC than skin cancer clinic doctors (NNE for SCC, 4.5 [95% CI, 3.9–5.1] and 6.2 [95% CI, 5.8–6.7], respectively; P < 0.01). There was no significant difference between GPs and skin cancer clinic doctors in the number of pigmented lesions excised per melanoma (NNE for melanoma, 20.7 [95% CI, 14.4–27.0] and 19.0 [95% CI, 14.9–23.1], respectively).
For both GPs and skin cancer clinic doctors, the NNE for melanoma fell significantly with increasing confidence that the lesion was malignant (NNE for both groups combined was 4.1 for lesions rated “very likely to be malignant” compared with 98.9 for lesions “not at all likely to be malignant”). In over a third of cases in which the likelihood of malignancy was rated as nil or very low, patient insistence was cited as the main reason for excision.
Skin cancer is a major public health issue in Australia and there are formidable challenges in providing clinical services to the population. In recent years, there has been a significant increase in skin cancer-related procedures such as diagnostic biopsies and skin flap repairs.4 The recent emergence of skin cancer clinics in primary care has provided an additional option for patients concerned about skin lesions, but questions have been raised about the clinical performance of doctors working in this area.11,12 To our knowledge, ours is the first large-scale prospective study to compare diagnostic accuracy between mainstream GPs and skin cancer clinic doctors. Our key finding was that diagnostic accuracy is similar for these two groups of doctors.
Strengths of our study were its large sample size, prospective design, random selection of GPs, and inclusion of a wide representation of skin cancer clinics. A limitation was the low response rate from GPs. While it was comparable to response rates in other studies auditing skin lesions,7,16 we can not be sure whether our sample was truly representative. Although we found no evidence of selection bias on the basis of age, sex or simple measures of clinical training, we had no information about the clinical interests of non-participating doctors, and so could not exclude the possibility that doctors with a particular interest in skin cancer medicine were over-represented among mainstream GP participants. There were no significant differences between participating and non-participating skin cancer clinics. While participants may have achieved increased diagnostic accuracy knowing that their performance was being scrutinised (the Hawthorne effect),17 this effect would have been similar for both groups.
Further, it is possible that patients attending general practices differ from those attending skin cancer clinics in ways that may affect diagnostic accuracy. For example, if patients in either setting are at higher risk of any skin cancer or of melanoma, a higher sensitivity and specificity for diagnosis of these cancers would be expected.
We observed that the average number of skin examinations, excisions and biopsies carried out by GPs in our study was similar to the number performed in previous studies in similar settings.5,18,19 Further, in our study, the distribution of histological diagnoses of lesions in both general practice and skin cancer clinics was similar (46% of lesions in general practice and 50% of those in skin cancer clinics were NMSC). These figures were also comparable to those of other studies.5,20 Our results are consistent with previous studies of diagnostic accuracy in skin cancer.7,21
Our finding that whole-body skin examinations were more common in skin cancer clinics than in general practice is significant, given the importance of early detection of melanoma. Melanoma detection rates can be improved sixfold by examining the whole body rather than just a part of it.22 We found that sensitivity for diagnosis of melanoma was significantly higher in skin cancer clinic doctors than in GPs. Skin cancer clinic doctors identified 60% of melanomas clinically, which was similar to the detection rate in one previous Australian study16 and higher than the detection rate in an overseas study.23
The PPV for melanoma was relatively low for both GPs (0.18) and skin cancer clinic doctors (0.25). However, this is not surprising, as doctors tend to be cautious in diagnosing melanoma, preferring to risk excising a benign lesion than to miss a potentially fatal cancer.24,25 About 30% of histologically confirmed melanomas were given a clinical diagnosis of dysplastic naevus.
The sensitivity of a diagnosis of malignancy was very high for doctors in both settings (0.91 for GPs and 0.94 for skin cancer clinic doctors) and did not alter after adjusting for years of clinical experience. Overall, about 70% of lesions diagnosed clinically as malignant were found to be malignant on histology (ie, the PPV was fairly high).
These results are similar to those found in other Australian studies,5,13,20 and higher than studies in countries where skin cancer incidence is lower.6,26 One interpretation is that Australian GPs develop good diagnostic skills given the high incidence of skin cancer. Similar positive correlations with increased caseloads have been reported for other diagnostic and surgical procedures.27,28 Additionally we found that, for both groups, the NPV was very high (0.92) for lesions that the doctor gave a low to very low likelihood of malignancy.
We found no significant differences between GPs and skin cancer clinic doctors regarding the number of lesions excised per malignancy. Moreover, the number of pigmented lesions excised per melanoma (21 for GPs versus 19 for skin cancer clinic doctors) was similar to the number found in previous studies (showing NNEs in the order of 22–29).7,21
The question arises as to whether doctors should aim for a high NNE (increasing the number of benign lesions excised) or a low NNE (increasing the risk that some melanomas will be missed). A number of factors can influence the decision of whether to excise a pigmented lesion. In our study we found that the number of lesions excised per melanoma fell as confidence in the likelihood of malignancy increased. We also found that a proportion of clearly benign lesions were excised after patient insistence, as reported by others.29
In conclusion, we found that doctors in both primary care settings (mainstream general practice and skin cancer clinics), achieved high sensitivity and specificity for diagnosing NMSC. Although skin cancer clinic doctors achieved significantly higher sensitivity for diagnosing melanoma, this was based on a small number of melanomas. Specificity for diagnosing melanoma and overall clinical accuracy for diagnosing skin cancer was similar between the two groups. These findings provide large-scale data to address the question of diagnostic accuracy of excised skin lesions within a rapidly growing area of primary care. However, the question of what clinical and patient-related factors influence decisions about management of suspicious skin lesions in the primary care setting is still to be answered.
1 Formulas used to calculate measures of diagnostic accuracy for all excised or biopsied skin lesions
2 Demographic characteristics of participating general practitioners and skin cancer clinic doctors in south-eastern Queensland, 2005*
3 Characteristics of patients undergoing a skin examination, by type of primary practice*
4 Diagnostic accuracy for selected clinical diagnoses, by type of primary practice*†
Received 14 February 2007, accepted 31 May 2007
- Philippa H Youl1
- Peter D Baade1
- Monika Janda2
- Christopher B Del Mar3
- David C Whiteman4
- Joanne F Aitken1
- 1 Viertel Centre for Research in Cancer Control, The Cancer Council Queensland, Brisbane, QLD.
- 2 Institute of Health and Biomedical Innovation, Faculty of Health, Queensland University of Technology, Brisbane, QLD.
- 3 Faculty of Health Sciences and Medicine, Bond University, Gold Coast, QLD.
- 4 Cancer and Population Study Group, Queensland Institute of Medical Research, Brisbane, QLD.
We thank the doctors who participated in our study, and Carla Shield and Joan Short for data collection. Our study was funded by National Health and Medical Research Council Project Grant No. 339100. Monika Janda is supported by an NHMRC Public Health Training Fellowship.
None identified.
- 1. Staples MP, Elwood M, Burton RC, et al. Non-melanoma skin cancer in Australia: the 2002 national survey and trends since 1985. Med J Aust 2006; 184: 6-10. <MJA full text>
- 2. Australian Institute of Health and Welfare; Australasian Associations of Cancer Registries. Cancer in Australia 2001. Canberra: AIHW, 2004. (AIHW Cat. No. CAN 23.)
- 3. Australian Institute of Health and Welfare. Health system expenditures on cancer and other neoplasms in Australia, 2000–01. Canberra: AIHW, 2005. (AIHW Cat. No. HWE 29.)
- 4. Medicare Australia. Medicare Benefits Schedule (MBS) item statistics reports. http://www.medicareaustralia.gov.au/providers/health_statistics/statistical_reporting/medicare.htm (accessed Jul 2007).
- 5. Raasch BA. Suspicious skin lesions and their management. Aust Fam Physician 1999; 28: 466-471.
- 6. Whited JD, Hall RP, Simel DL, Horner RD. Primary care clinicians’ performance for detecting actinic keratoses and skin cancer. Arch Intern Med 1997; 157: 985-990.
- 7. English DR, Del Mar C, Burton RC. Factors influencing the number needed to excise: excision rates of pigmented lesions by general practitioners. Med J Aust 2004; 180: 16-19. <MJA full text>
- 8. Del Mar CB, Lowe JB. The skin cancer workload in Australian general practice. Aust Fam Physician 1997; 26 Suppl 1: S24-S27.
- 9. Charles J, Knox S, Britt H. Skin excisions in Australian general practice. Aust Fam Physician 2006; 35: 474-475.
- 10. Wilkinson D, Dick M-L, Askew DA. General practitioners with special interests: risk of a good thing becoming bad? Med J Aust 2005; 183: 84-86. <MJA full text>
- 11. Swan N. Skin cancer clinics [radio transcript]. The Health Report 2003; 15 Sep. http://www.abc.net.au/rn/talks/8.30/helthrpt/stories/s946796.htm (accessed Jun 2007).
- 12. Chia AL, Shumack S. Skin cancer clinics in Australia: workload profile and performance indicators from an analysis of billing data [letter]. Med J Aust 2006; 185: 239-240. <MJA full text>
- 13. Moffatt CR, Green A, Whiteman DC. Diagnostic accuracy in skin cancer clinics: the Australian experience. Int J Dermatol 2006; 45: 656-660.
- 14. Wilkinson D, Askew DA, Dixon A. Skin cancer clinics in Australia: workload profile and performance indicators from an analysis of billing data. Med J Aust 2006; 184: 162-164. <MJA full text>
- 15. Greenhalgh T. How to read a paper. Papers that report diagnostic or screening tests. BMJ 1997; 315: 540-543.
- 16. Burton RC, Howe C, Adamson L, et al. General practitioner screening for melanoma: sensitivity, specificity, and effect of training. J Med Screen 1998; 5: 156-161.
- 17. Fletcher RH, Fletcher SW, Wagner EH. Clinical epidemiology: the essentials. 3rd ed. Baltimore: Williams and Wilkins, 1998: 129-156.
- 18. Del Mar CB, Balanda KP. Examination of skin naevi in Australian general practice: dissonance between care and need? Aust N Z J Public Health 1998; 22: 397-399.
- 19. Lowe JB, Balanda KP, Del Mar CB, et al. General practitioner and patient response during a public education program to encourage skin examinations. Med J Aust 1994; 161: 195-198.
- 20. Lathlean S. Skin cancer in general practice in South Australia. A five year study. Aust Fam Physician 1999; 28 Suppl 1: S28-S31.
- 21. English DR, Burton RC, Del Mar CB, et al. Evaluation of aid to diagnosis of pigmented skin lesions in general practice: controlled trial randomised by practice. BMJ 2003; 327: 375-378.
- 22. Rigel DS, Friedman RJ, Kopf AW, et al. Importance of complete cutaneous examination for the detection of malignant melanoma. J Am Acad Dermatol 1986; 14: 857-860.
- 23. Cassileth BR, Clark WH, Lusk EJ, et al. How well do physicians recognize melanoma and other problem lesions? J Am Acad Dermatol 1986; 14: 555-560.
- 24. Carli P, Nardini P, Crocetti E, et al. Frequency and characteristics of melanomas missed at a pigmented lesion clinic: a registry-based study. Melanoma Res 2004; 14: 403-407.
- 25. Morton CA, Mackie RM. Clinical accuracy of the diagnosis of cutaneous malignant melanoma. Br J Dermatol 1998; 138: 283-287.
- 26. Jonna BP, Delfino RJ, Newman WG, et al. Positive predictive value for presumptive diagnoses of skin cancer and compliance with follow-up among patients attending a community screening program. Prev Med 1998; 27: 611-616.
- 27. Beam CA, Conant EF, Sickles EA. Association of volume and volume-independent factors with accuracy in screening mammogram interpretation. J Natl Cancer Inst 2003; 95: 282-290.
- 28. Esserman L, Cowley H, Eberle C, et al. Improving the accuracy of mammography: volume and outcome relationships. J Natl Cancer Inst 2002; 94: 369-375.
- 29. Del Mar CB, Green AC. Aid to diagnosis of melanoma in primary medical care. BMJ 1995; 310: 492-495.
Abstract
Objective: To measure and compare the casemix and diagnostic accuracy of excised or biopsied skin lesions managed by mainstream general practitioners and doctors within primary care skin cancer clinics.
Design, setting and participants: Prospective comparative study of 104 GPs and 50 skin cancer clinic doctors in south-eastern Queensland, involving 28 755 patient encounters. The study was conducted in 2005.
Main outcome measures: Prevalence of each type of skin lesion; sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) for the clinical diagnosis against histology; number needed to excise or biopsy (NNE) for a diagnosis of skin cancer.
Results: GPs excised or biopsied 3175 skin lesions (mean 2.5/week) including 743 basal cell carcinomas (BCCs) (23.4%), 704 squamous cell carcinomas (SCCs) (22.2%) and 49 melanomas (1.5%). Skin cancer clinic doctors excised or biopsied 7941 skin lesions (mean 34/week), including 2701 BCCs (34.0%), 1274 SCCs (16.0%) and 103 melanomas (1.3%). Overall, sensitivity for diagnosing any skin cancer was similar for skin cancer clinic doctors (0.94) and GPs (0.91), although higher for skin cancer clinic doctors for BCC (0.89 v 0.79; P < 0.01) and melanoma (0.60 v 0.29; P < 0.01). The overall NNE was similar for skin cancer clinic doctors (1.9; 95% CI, 1.8%–2.1%) and GPs (2.1; 95% CI, 1.9%–2.3%). This did not change after adjusting for years of clinical experience.
Conclusions: GPs and skin cancer clinic doctors in Queensland treat large numbers of skin cancers and diagnose these with overall high sensitivity. The two groups diagnosed skin cancer with similar accuracy.