Evaluation of the PAPNET system in a general pathology service
Annabelle Farnsworth, Fay M Chambers and Colin S Goldschmidt
MJA 1996; 165: 429-431
Disclaimer of conflict of interest -
More articles on Obstetrics & gynaecology and women's health
Objective: To compare the results of an automated rescreening device
(PAPNET) with manual screening of Papanicolaou (Pap) smears.|
Design: All normal or technically unsatisfactory smears and a random
sample of abnormal smears on manual screening were submitted for
Setting: Large general pathology laboratory in Sydney between
January and September 1995.
Results: Of 54 658 PAP smears classified on manual screening as
normal, 266 were reclassified as abnormal after PAPNET screening (32
atypical squamous cells of uncertain significance, 217 low-grade
squamous intraepithelial lesions and 17 high-grade
intraepithelial lesions). Of the random sample of 1022 smears
classified on manual screening as abnormal, all high-grade
intraepithelial lesions (122 smears) and squamous cell carcinomas
(2 smears) were also detected by PAPNET, and 112 were reclassified as
normal by PAPNET (14 atypical squamous cells of uncertain
significance and 98 low-grade squamous intraepithelial lesions).
Histological follow-up confirmed 15 of the 17 smears classified as
high-grade intraepithelial lesions on PAPNET screening and
detected a further seven that had been classified by PAPNET as either
atypical squamous cells of uncertain significance or low-grade
squamous intraepithelial lesions.
Conclusions: When used as a quality-control measure in a general
pathology laboratory, the PAPNET automated screening system
detects higher numbers of abnormal PAP smears than manual screening.
Papanicolaou (Pap)-smear examination has been shown to lower
mortality and morbidity from cervical cancer,1 but the
screening test itself is imperfect. False negative results (when a
smear is reported as normal but has been taken from a woman with a
high-grade lesion in her cervix) have been reported to be between
The failure of Pap smears to detect some high-grade abnormalities is
the result of either sampling or laboratory error.2 Sampling error
occurs when the person taking the smear fails to collect the abnormal
cervical cells because of either poor technique or the small size or
localised nature of the lesion.3 Laboratory error is usually
caused by the screening cytologist failing to detect the abnormal
cells on the slide.
Screening of Pap smears is a labour-intensive task requiring a high
degree of skill and concentration. Even in laboratories using
well-recognised quality control measures and staffed by highly
trained cytologists, many high-grade abnormalities remain
undetected by routine manual screening. These may also be
missed on a manual rescreen.3
PAPNET is an automated interactive system for analysis of Pap smears
which has been shown to detect abnormalities that were repeatedly
missed on manual screening.4,5 We introduced the PAPNET
system into our laboratory in January 1995 as a quality-control
measure for rescreening of all negative (normal) smears. Here, we
compare the detection of abnormalities by PAPNET with those detected
by manual screening in our laboratory, and report on the histological
follow-up of the PAPNET cytopredictions and the work practices of
cytologists using this system.
Our laboratory is a large general practice in Sydney with a referral
base covering metropolitan and country areas of New South Wales.
During the period of our survey (23 January to 30 September 1995),
all Pap smears were screened manually using routine methods.
Quality assurance measures used in the study (e.g., rescreening and
follow-up of abnormal smears) were routine procedures used in our
laboratory. All smears considered "normal" (negative) and
"technically unsatisfactory" on routine manual screening,
together with a random sample of smears considered "abnormal" on
routine screening during the same period, were submitted for PAPNET
analysis. Smears were considered technically unsatisfactory when a
confident report could not be given because of scanty material, or
obscuring of cellular detail by blood, inflammation, cellular
degeneration or air drying. The abnormal smears were randomly
selected from the recent file of atypical smears. The PAPNET
rescreening was undertaken blind to the results of manual screening.
(See Box 1 for a description of the PAPNET technology.)
Interim reports were generated for smears considered normal and
technically unsatisfactory, and, after the slide review was
completed in the laboratory, a final report was issued incorporating
the PAPNET findings. Reporting terminology was based on the Bethesda
system of reporting cervical/vaginal cytological
A record was kept of the number of cases each
cytologist was able to review per day with PAPNET. The number of slides
that required manual rechecking and complete rescreening was also
recorded. This involved the cytologists using the light microscope
to recheck individual abnormal cells that were tagged on the PAPNET
monitor. If these cells were considered to be significantly
atypical, the whole slide was rescreened. If any degree of
abnormality was found after rescreening, the slide was submitted to
the cytopathologist for checking.
To test the specificity of PAPNET, histological follow-up was
performed for all patients who had an additional lesion predicted by
PAPNET and were recommended to have colposcopy for this lesion. As
follow-up of cytological predictions can take many months, the
process was expedited and, where possible, histology results are
given. In some cases, follow-up was still pending or the referring
doctor had decided to follow-up by repeated observation and Pap
smears. These are designated as "follow-up to come". In some cases,
follow-up was unavailable; the referring doctor had been unable to
contact the patient, the patient had moved or had decided to proceed no
further with management. These cases are designated as "lost to
During the 10-month period of this survey, 60 317 Pap smears were
submitted to the laboratory for examination. These smears are
categorised in Box 2 (cytopredictions). The results fall within
suggested National Cervical Screening Programme standards for
reporting categories of technically satisfactory smears.
PAPNET screening was performed on 54 658 smears classified on manual
screening as negative (or within normal limits) and 1819 smears
classified as technically unsatisfactory. Of the 3840 smears
classified on manual screening as abnormal, 1022 smears (27%) were
randomly selected and submitted for PAPNET screening: 807 low-grade
squamous intraepithelial lesions, 122 high-grade squamous
intraepithelial lesions, 91 atypical squamous cells of uncertain
significance, and two cases of squamous cell carcinoma.
Manual screening compared to PAPNET screening|
Of the smears classified as negative on manual screening, 266 were
reclassified as abnormal after PAPNET rescreening, including 17
reported as high-grade squamous intraepithelial lesions (Box 3).
This represents a 7% increase in abnormal smears after PAPNET
rescreening. All of the 122 smears reported as high-grade squamous
intraepithelial lesions and both squamous cell carcinomas detected
on manual screening were also detected by PAPNET (Box 3). Fourteen
(15%) of the 91 smears reported as atypical squamous cells of
uncertain significance and 98 (12%) of the 807 reported as low-grade
squamous intraepithelial lesions were reclassified as negative on
PAPNET rescreening (Box 3).
Histological follow-up of PAPNET cytopredictions|
Histology confirmed an abnormality in 28% of smears reported as
atypical squamous cells of uncertain significance and 61% reported
as low-grade squamous intraepithelial lesions (Box 4). Of
the 17 smears reported as high-grade squamous intraepithelial
lesions on PAPNET rescreening, 15 (88%) were confirmed on histology
(Box 4). Three of the 32 cases reported as atypical squamous cells of
uncertain significance and four of the 31 reported as low-grade
squamous intraepithelial lesions (where colposcopy was
recommended) were also shown to have a high-grade lesion on
histological biopsy. In total, there were an extra 22
histologically confirmed high-grade lesions not detected on manual
screening, of which 15 were detected by PAPNET.
Cytologists' work practices|
In our laboratory, cytologists manually screened between 25 and 50
slides (mean, 42; standard deviation [SD], 6) per day. The large SD
reflected the range of experience among the laboratory's
cytologists, who alternate between gynaecological and
non-gynaecological cytology. PAPNET rescreening was included in
After training, cytologists performing PAPNET rescreening were
able to screen between 100 and 160 slides (mean, 126; SD, 15) per day.
This included loading and unloading the tapes, viewing and assessing
the images on the PAPNET screen, documentation and generating
The number of cases submitted for limited rechecking varied between
10%-90%, depending on the cytologist. The number requiring complete
rescreening by a senior cytologist varied between 0 to 10 per
cytologist per day and averaged five per day. The number of cases
submitted to a cytopathologist by each cytologist averaged one per
Pap smears remain the highest-volume pathology test that is not
automated. Attempts have been made to automate this complex process
for nearly 40 years, but it is only recently that advances in the
development of automated systems have occurred which offer
practical advantages in a laboratory.
Although the detection rate of abnormalities on manual screening was
well within normal standards in our study, PAPNET rescreening was
able to detect an increased number of abnormal Pap smears. This
increase was not associated with a loss of specificity, as often
occurs with increased detection by manual screening.3
Many studies have shown the ability of PAPNET rescreening to detect a
significant number of abnormal Pap smears when compared with manual
screening, but these have been predominantly smaller studies using
Our study was undertaken as part of the routine
workflow in a laboratory processing a large number of smears and is the
first Australian study to compare the two screening systems.
The reported increase in detection of abnormalities in other studies
has been as high as 16%,10
and the company which developed PAPNET
(Neuromedical Systems Inc., New York, USA) claims increases of up to
30%. The increase in our study was much smaller (7%), with most
detected abnormalities being low-grade lesions. However, we
believe this increase was worthwhile given the reasonably high rate
of detection on manual screening and the large volume of routine
Boon et al. reported the results of a large series of smears that were
not double-screened, but had been either manually screened or PAPNET
screened. Higher detection rates of abnormalities were found in the
PAPNET group. Our study is the first report of a large series in which
all negative smears were screened both manually and by PAPNET.
Our study showed that the PAPNET system could be feasibly integrated
into a routine working laboratory situation. It was a valuable
educational and quality assurance tool. Cytologists found it easy to
use and were able to review slides at three to four times their
manual-screening rate. From our experience, cytologists did not
become complacent in their manual screening, but rather became more
cautious knowing that all the negative smears would be examined by a
computerised device. The additional work for
cytopathologists and senior cytologists was not great.
PAPNET is only one of a number of automated cytology systems currently
available as an additional test for routine Pap-smear screening. We
did not undertake a comparison of other automated systems or their
cost-effectiveness. Further studies on the cost-effectiveness of
PAPNET testing and its efficacy for use in public health screening
programs are needed. It may be that automated testing technology will
replace current screening methods in the future.
Disclaimer of conflict of interest
This work was conducted independently. Neither Douglass Hanly Moir
Pathology nor any of the individual authors has any financial or other
association with the suppliers of the PAPNET service, Neuromedical
- Mitchell HS, Giles GG. Cancer diagnosis after a report of negative
cervical cytology. Med J Aust 1996; 164: 270-273.
Koss LG. The Papanicolaou test for cervical cancer detection. A
triumph and a tragedy. JAMA 1989; 261: 737-743.
Boscha MC, Rietveld-Scheffers PEM, Boon ME. Characteristics of
false negative smears tested in the normal screening situation.
Acta Cytol 1992; 36: 711-716.
Koss LG, Hin E, Schreiber K, et al. Evaluation of the PAPNET
cytologic screening system for quality control of cervical smears.
Am J Clin Pathol 1994; 101: 220-229.
Boon M, Kok CP. Neural network processing can provide means to catch
errors that slip through human screening of Pap smears. Diagn
Cytopathol 1993; 9: 411-416.
The Bethesda system of reporting cervical/vaginal cytologic
diagnoses: revised after the second National Cancer Institute
Workshop, April 29-30, 1991. Acta Cytol 1993; 37: 115-124.
Royal College of Pathologists of Australasia. Performance
Standards for Australian Laboratories Reporting Cervical
Cytology. Sydney: RCPA Quality Assurance Programs, 1995.
Rosenthal DL, Acosta D, Peters RK. Computer-assisted
re-screening of clinically important false negative cervical
smears using the PAPNET testing system. Acta Cytol 1996; 40:
Sherman ME, Mango LJ, Kelly D, et al. PAPNET analysis of reportedly
negative smears preceding the diagnosis of a high grade squamous
intraepithelial lesion or carcinoma. Mod Pathol 1994; 7:
Elgert PA, Suhrland MJ, Re E, et al. Prospective quality control of
cervical smears using the PAPNET system [abstract No. 5]. In:
Abstracts of the Scientific Session of the 42nd Annual Scientific
Meeting of the American Society of Cytology; 1994 Nov 1-6; Chicago.
Acta Cytol 1994; 38: 793-805.
Boon ME, Kok LP, Beck S. Histologic validation of neural network
assisted cervical screening: comparison with the conventional
procedure. Cell Vision 1995; 2: 23-27.
(Received 10 Jan, accepted 14 Aug 1996)
Douglass Hanly Moir Pathology, 95 Epping Road, North Ryde, NSW.
Annabelle Farnsworth, MB BS, FRCPA, Cytopathologist.
Fay M Chambers, MB BS, FRCPA, Cytopathologist.
Colin S Goldschmidt, MB BCh, FRCPA, Cytopathologist.
Reprints: Dr A Farnsworth, Douglass Hanly Moir Pathology, 95 Epping
Road, North Ryde 2113.
Journalists are welcome to write news stories based on what they read here, but should acknowledge their source as "an article published on the Internet by The Medical Journal of Australia <http://www.mja.com.au>".
|1: The PAPNET system.|
PAPNET consists of a scanning apparatus and a review station.4 In Australia, the cervical slides are sent to Hong Kong for the initial scanning. The scanning apparatus includes an automated microscope with an attached video camera which transmits images to a primary classifier. Selected images are then passed into a neural net computing unit which chooses the 128 highest-ranking images for storage on an optical disk or digital tape, which is sent back to Australia for display at the review station. The review station is located in the cytology laboratory where manual screening takes place. The 128 recorded images are displayed in panels on a high-resolution colour monitor and are reviewed by a cytologist. If an abnormality is suspected, the original slide is manually checked and, if necessary, manually rescreened.
|Back to text|
|Back to text|
3: PAPNET rescreening of PAP smears classified as normal and abnormal on manual screening.|
|Cytoprediction||Normal smears: manual screening||PAPNET rescreening||Abnormal smears: manual screening||PAPNET rescreening|
|Negative*||54 658||54 359||--||112|
|Atypical squamous cells of uncertain significance||--||32||91||77|
|Low-grade squamous intraepithelial lesions||--||217||807||709|
|High-grade squamous intraepithelial lesions||--||17||122||122|
|Squamous cell carcinoma||--||0||2||2|
* Cellular appearance was within normal limits. The 112 negative smears on PAPNET screening had been classified on manual screening as atypical squamous cells of uncertain significance (14 smears) and
low-grade squamous intraepithelial lesions (98 smears).
|Back to text|
4: Histological follow-up of PAPNET cytopredictions.|
|PAPNET-detected abnormalities||Negative/inflammation*||Low-grade squamous intraepithelial lesions||High-grade squamous intraepithelial lesions||Follow-up to come||Lost to follow-up||Total|
|Atypical squamous cells of unknown significance||6||6||3||7||10||32|
|Low-grade squamous intraepithelial lesions||2||15||4||4||6||31|
|High-grade squamous intraepithelial lesions||
*Cellular appearance was within normal limits.
|Back to text|