Connect
MJA
MJA

Evaluation of the PAPNET system in a general pathology service

Annabelle Farnsworth, Fay M Chambers and Colin S Goldschmidt
Med J Aust 1996; 165 (8): 429-431.
Published online: 9 September 1999
Research

Evaluation of the PAPNET system in a general pathology service

Annabelle Farnsworth, Fay M Chambers and Colin S Goldschmidt

MJA 1996; 165: 429-431

Abstract - Introduction - Methods - Results - Discussion - References - Disclaimer of conflict of interest - Authors' details
- - More articles on Obstetrics & gynaecology and women's health


Abstract Objective: To compare the results of an automated rescreening device (PAPNET) with manual screening of Papanicolaou (Pap) smears.
Design: All normal or technically unsatisfactory smears and a random sample of abnormal smears on manual screening were submitted for PAPNET rescreening.
Setting: Large general pathology laboratory in Sydney between January and September 1995.
Results: Of 54 658 PAP smears classified on manual screening as normal, 266 were reclassified as abnormal after PAPNET screening (32 atypical squamous cells of uncertain significance, 217 low-grade squamous intraepithelial lesions and 17 high-grade intraepithelial lesions). Of the random sample of 1022 smears classified on manual screening as abnormal, all high-grade intraepithelial lesions (122 smears) and squamous cell carcinomas (2 smears) were also detected by PAPNET, and 112 were reclassified as normal by PAPNET (14 atypical squamous cells of uncertain significance and 98 low-grade squamous intraepithelial lesions). Histological follow-up confirmed 15 of the 17 smears classified as high-grade intraepithelial lesions on PAPNET screening and detected a further seven that had been classified by PAPNET as either atypical squamous cells of uncertain significance or low-grade squamous intraepithelial lesions.
Conclusions: When used as a quality-control measure in a general pathology laboratory, the PAPNET automated screening system detects higher numbers of abnormal PAP smears than manual screening.


Introduction Papanicolaou (Pap)-smear examination has been shown to lower mortality and morbidity from cervical cancer,1 but the screening test itself is imperfect. False negative results (when a smear is reported as normal but has been taken from a woman with a high-grade lesion in her cervix) have been reported to be between 10%-20%.2

The failure of Pap smears to detect some high-grade abnormalities is the result of either sampling or laboratory error.2 Sampling error occurs when the person taking the smear fails to collect the abnormal cervical cells because of either poor technique or the small size or localised nature of the lesion.3 Laboratory error is usually caused by the screening cytologist failing to detect the abnormal cells on the slide.

Screening of Pap smears is a labour-intensive task requiring a high degree of skill and concentration. Even in laboratories using well-recognised quality control measures and staffed by highly trained cytologists, many high-grade abnormalities remain undetected by routine manual screening. These may also be missed on a manual rescreen.3

PAPNET is an automated interactive system for analysis of Pap smears which has been shown to detect abnormalities that were repeatedly missed on manual screening.4,5 We introduced the PAPNET system into our laboratory in January 1995 as a quality-control measure for rescreening of all negative (normal) smears. Here, we compare the detection of abnormalities by PAPNET with those detected by manual screening in our laboratory, and report on the histological follow-up of the PAPNET cytopredictions and the work practices of cytologists using this system.


Methods Our laboratory is a large general practice in Sydney with a referral base covering metropolitan and country areas of New South Wales. During the period of our survey (23 January to 30 September 1995), all Pap smears were screened manually using routine methods. Quality assurance measures used in the study (e.g., rescreening and follow-up of abnormal smears) were routine procedures used in our laboratory. All smears considered "normal" (negative) and "technically unsatisfactory" on routine manual screening, together with a random sample of smears considered "abnormal" on routine screening during the same period, were submitted for PAPNET analysis. Smears were considered technically unsatisfactory when a confident report could not be given because of scanty material, or obscuring of cellular detail by blood, inflammation, cellular degeneration or air drying. The abnormal smears were randomly selected from the recent file of atypical smears. The PAPNET rescreening was undertaken blind to the results of manual screening. (See Box 1 for a description of the PAPNET technology.)

Interim reports were generated for smears considered normal and technically unsatisfactory, and, after the slide review was completed in the laboratory, a final report was issued incorporating the PAPNET findings. Reporting terminology was based on the Bethesda system of reporting cervical/vaginal cytological diagnoses.6

A record was kept of the number of cases each cytologist was able to review per day with PAPNET. The number of slides that required manual rechecking and complete rescreening was also recorded. This involved the cytologists using the light microscope to recheck individual abnormal cells that were tagged on the PAPNET monitor. If these cells were considered to be significantly atypical, the whole slide was rescreened. If any degree of abnormality was found after rescreening, the slide was submitted to the cytopathologist for checking.

To test the specificity of PAPNET, histological follow-up was performed for all patients who had an additional lesion predicted by PAPNET and were recommended to have colposcopy for this lesion. As follow-up of cytological predictions can take many months, the process was expedited and, where possible, histology results are given. In some cases, follow-up was still pending or the referring doctor had decided to follow-up by repeated observation and Pap smears. These are designated as "follow-up to come". In some cases, follow-up was unavailable; the referring doctor had been unable to contact the patient, the patient had moved or had decided to proceed no further with management. These cases are designated as "lost to follow-up".


Results During the 10-month period of this survey, 60 317 Pap smears were submitted to the laboratory for examination. These smears are categorised in Box 2 (cytopredictions). The results fall within suggested National Cervical Screening Programme standards for reporting categories of technically satisfactory smears.

PAPNET screening was performed on 54 658 smears classified on manual screening as negative (or within normal limits) and 1819 smears classified as technically unsatisfactory. Of the 3840 smears classified on manual screening as abnormal, 1022 smears (27%) were randomly selected and submitted for PAPNET screening: 807 low-grade squamous intraepithelial lesions, 122 high-grade squamous intraepithelial lesions, 91 atypical squamous cells of uncertain significance, and two cases of squamous cell carcinoma.

Manual screening compared to PAPNET screening
Of the smears classified as negative on manual screening, 266 were reclassified as abnormal after PAPNET rescreening, including 17 reported as high-grade squamous intraepithelial lesions (Box 3). This represents a 7% increase in abnormal smears after PAPNET rescreening. All of the 122 smears reported as high-grade squamous intraepithelial lesions and both squamous cell carcinomas detected on manual screening were also detected by PAPNET (Box 3). Fourteen (15%) of the 91 smears reported as atypical squamous cells of uncertain significance and 98 (12%) of the 807 reported as low-grade squamous intraepithelial lesions were reclassified as negative on PAPNET rescreening (Box 3).

Histological follow-up of PAPNET cytopredictions
Histology confirmed an abnormality in 28% of smears reported as atypical squamous cells of uncertain significance and 61% reported as low-grade squamous intraepithelial lesions (Box 4). Of the 17 smears reported as high-grade squamous intraepithelial lesions on PAPNET rescreening, 15 (88%) were confirmed on histology (Box 4). Three of the 32 cases reported as atypical squamous cells of uncertain significance and four of the 31 reported as low-grade squamous intraepithelial lesions (where colposcopy was recommended) were also shown to have a high-grade lesion on histological biopsy. In total, there were an extra 22 histologically confirmed high-grade lesions not detected on manual screening, of which 15 were detected by PAPNET.

Cytologists' work practices
In our laboratory, cytologists manually screened between 25 and 50 slides (mean, 42; standard deviation [SD], 6) per day. The large SD reflected the range of experience among the laboratory's cytologists, who alternate between gynaecological and non-gynaecological cytology. PAPNET rescreening was included in this rotation.

After training, cytologists performing PAPNET rescreening were able to screen between 100 and 160 slides (mean, 126; SD, 15) per day. This included loading and unloading the tapes, viewing and assessing the images on the PAPNET screen, documentation and generating patient reports.

The number of cases submitted for limited rechecking varied between 10%-90%, depending on the cytologist. The number requiring complete rescreening by a senior cytologist varied between 0 to 10 per cytologist per day and averaged five per day. The number of cases submitted to a cytopathologist by each cytologist averaged one per day.


Discussion Pap smears remain the highest-volume pathology test that is not automated. Attempts have been made to automate this complex process for nearly 40 years, but it is only recently that advances in the development of automated systems have occurred which offer practical advantages in a laboratory.

Although the detection rate of abnormalities on manual screening was well within normal standards in our study, PAPNET rescreening was able to detect an increased number of abnormal Pap smears. This increase was not associated with a loss of specificity, as often occurs with increased detection by manual screening.3

Many studies have shown the ability of PAPNET rescreening to detect a significant number of abnormal Pap smears when compared with manual screening, but these have been predominantly smaller studies using archived material.8,9 Our study was undertaken as part of the routine workflow in a laboratory processing a large number of smears and is the first Australian study to compare the two screening systems.

The reported increase in detection of abnormalities in other studies has been as high as 16%,10 and the company which developed PAPNET (Neuromedical Systems Inc., New York, USA) claims increases of up to 30%. The increase in our study was much smaller (7%), with most detected abnormalities being low-grade lesions. However, we believe this increase was worthwhile given the reasonably high rate of detection on manual screening and the large volume of routine smears processed.

Boon et al. reported the results of a large series of smears that were not double-screened, but had been either manually screened or PAPNET screened. Higher detection rates of abnormalities were found in the PAPNET group. Our study is the first report of a large series in which all negative smears were screened both manually and by PAPNET.

Our study showed that the PAPNET system could be feasibly integrated into a routine working laboratory situation. It was a valuable educational and quality assurance tool. Cytologists found it easy to use and were able to review slides at three to four times their manual-screening rate. From our experience, cytologists did not become complacent in their manual screening, but rather became more cautious knowing that all the negative smears would be examined by a computerised device. The additional work for cytopathologists and senior cytologists was not great.

PAPNET is only one of a number of automated cytology systems currently available as an additional test for routine Pap-smear screening. We did not undertake a comparison of other automated systems or their cost-effectiveness. Further studies on the cost-effectiveness of PAPNET testing and its efficacy for use in public health screening programs are needed. It may be that automated testing technology will replace current screening methods in the future.



Disclaimer of conflict of interest
This work was conducted independently. Neither Douglass Hanly Moir Pathology nor any of the individual authors has any financial or other association with the suppliers of the PAPNET service, Neuromedical Systems Inc.


References
  1. Mitchell HS, Giles GG. Cancer diagnosis after a report of negative cervical cytology. Med J Aust 1996; 164: 270-273.
  2. Koss LG. The Papanicolaou test for cervical cancer detection. A triumph and a tragedy. JAMA 1989; 261: 737-743.
  3. Boscha MC, Rietveld-Scheffers PEM, Boon ME. Characteristics of false negative smears tested in the normal screening situation. Acta Cytol 1992; 36: 711-716.
  4. Koss LG, Hin E, Schreiber K, et al. Evaluation of the PAPNET cytologic screening system for quality control of cervical smears. Am J Clin Pathol 1994; 101: 220-229.
  5. Boon M, Kok CP. Neural network processing can provide means to catch errors that slip through human screening of Pap smears. Diagn Cytopathol 1993; 9: 411-416.
  6. The Bethesda system of reporting cervical/vaginal cytologic diagnoses: revised after the second National Cancer Institute Workshop, April 29-30, 1991. Acta Cytol 1993; 37: 115-124.
  7. Royal College of Pathologists of Australasia. Performance Standards for Australian Laboratories Reporting Cervical Cytology. Sydney: RCPA Quality Assurance Programs, 1995.
  8. Rosenthal DL, Acosta D, Peters RK. Computer-assisted re-screening of clinically important false negative cervical smears using the PAPNET testing system. Acta Cytol 1996; 40: 120-126.
  9. Sherman ME, Mango LJ, Kelly D, et al. PAPNET analysis of reportedly negative smears preceding the diagnosis of a high grade squamous intraepithelial lesion or carcinoma. Mod Pathol 1994; 7: 578-581.
  10. Elgert PA, Suhrland MJ, Re E, et al. Prospective quality control of cervical smears using the PAPNET system [abstract No. 5]. In: Abstracts of the Scientific Session of the 42nd Annual Scientific Meeting of the American Society of Cytology; 1994 Nov 1-6; Chicago. Acta Cytol 1994; 38: 793-805.
  11. Boon ME, Kok LP, Beck S. Histologic validation of neural network assisted cervical screening: comparison with the conventional procedure. Cell Vision 1995; 2: 23-27.

(Received 10 Jan, accepted 14 Aug 1996)


Authors' details Douglass Hanly Moir Pathology, 95 Epping Road, North Ryde, NSW.
Annabelle Farnsworth, MB BS, FRCPA, Cytopathologist.
Fay M Chambers, MB BS, FRCPA, Cytopathologist.
Colin S Goldschmidt, MB BCh, FRCPA, Cytopathologist.

Reprints: Dr A Farnsworth, Douglass Hanly Moir Pathology, 95 Epping Road, North Ryde 2113.




Journalists are welcome to write news stories based on what they read here, but should acknowledge their source as "an article published on the Internet by The Medical Journal of Australia <http://www.mja.com.au>".
<URL: http://www.mja.com.au/>
We appreciate your comments.


1: The PAPNET system.

PAPNET consists of a scanning apparatus and a review station.4 In Australia, the cervical slides are sent to Hong Kong for the initial scanning. The scanning apparatus includes an automated microscope with an attached video camera which transmits images to a primary classifier. Selected images are then passed into a neural net computing unit which chooses the 128 highest-ranking images for storage on an optical disk or digital tape, which is sent back to Australia for display at the review station. The review station is located in the cytology laboratory where manual screening takes place. The 128 recorded images are displayed in panels on a high-resolution colour monitor and are reviewed by a cytologist. If an abnormality is suspected, the original slide is manually checked and, if necessary, manually rescreened.

Back to text

2: Categories of Pap smears on manual screening (23 January to 30 September 1995).

CytopredictionNumber of smears

Negative*54 658 (90.6%)
Low-grade squamous intraepithelial lesions 2974 (4.9%)
High-grade squamous intraepithelial lesions 432 (0.7%)
Atypical squamous cells of uncertain significance434 (0.7%)
Technically unsatisfactory1819 (3%)
Total60 317 (100%)

* Cellular appearance was within normal limits.
Back to text

3: PAPNET rescreening of PAP smears classified as normal and abnormal on manual screening.

CytopredictionNormal smears: manual screeningPAPNET rescreeningAbnormal smears: manual screeningPAPNET rescreening

Negative*54 65854 359--112
Atypical squamous cells of uncertain significance--329177
Low-grade squamous intraepithelial lesions--217807709
High-grade squamous intraepithelial lesions--17122122
Squamous cell carcinoma--022
Technically unsatisfactory18191852----

* Cellular appearance was within normal limits. The 112 negative smears on PAPNET screening had been classified on manual screening as atypical squamous cells of uncertain significance (14 smears) and low-grade squamous intraepithelial lesions (98 smears).
Back to text

4: Histological follow-up of PAPNET cytopredictions.

Histological follow-up

PAPNET-detected abnormalitiesNegative/inflammation*Low-grade squamous intraepithelial lesionsHigh-grade squamous intraepithelial lesionsFollow-up to comeLost to follow-upTotal

Atypical squamous cells of unknown significance66371032
Low-grade squamous intraepithelial lesions21544631
High-grade squamous intraepithelial lesions 01150117

*Cellular appearance was within normal limits.
Back to text

Received 20 November 2018, accepted 20 November 2018

  • Annabelle Farnsworth
  • Fay M Chambers
  • Colin S Goldschmidt


Correspondence: 

Author

remove_circle_outline Delete Author
add_circle_outline Add Author

Comment
Do you have any competing interests to declare? *

I/we agree to assign copyright to the Medical Journal of Australia and agree to the Conditions of publication *
I/we agree to the Terms of use of the Medical Journal of Australia *
Email me when people comment on this article

Responses are now closed for this article.