Effect of Thin Prep^® imaging system on laboratory rate and relative sensitivity of atypical squamous cells, high-grade squamous intraepithelial lesion not excluded and high-grade squamous intraepithelial lesion interpretations

INTRODUCTION

The development of methods for automated screening of cervical cytology specimens began shortly after the widespread introduction of the Pap smear screening for cervical cancer in the 1950's.[1] The first computerized scanning techniques were approved by the United States Food and Drug Administration (FDA) in the 1990's, and since then improvements in technology have been made to incorporate liquid based collected and processed specimens.[2] One of the two computerized imagers in use today, the ThinPrep® Imaging System (TIS) (Hologic Corp. [previously Cytyc Corp.], Marlborough, MA) was first approved in 2003 by the FDA to aid in primary screening of ThinPrep^® Papanicolaou Tests. The other imager in use in the United States, the Becton, Dickinson, and Company (BD) Focal Point™ Imaging System (BD Diagnostics-TriPath, Franklin Lakes, NJ) received FDA approval in 2008.

The TIS consists of an imager, a PC computer system run on Windows NT software (Microsoft Corporation, Redmond, WA), and review microscopes. Once a ThinPrep liquid based cervical cytology specimen is processed, labeled, and stained, the slide can be imaged on the system. The computer system uses an Optical Selection algorithm, which is based on nuclear size and staining characteristics, to display cells of concern. Based on the integrated optical density algorithm, 22 fields of view are chosen to include cells with large or hyperchromatic nuclei and high nuclear to cytoplasmic ratio. Once on the review scope, the imaged slide's coordinates for the chosen fields of view are saved for review by the cytotechnologist. The fields are examined by the cytotechnologist and either signed out as negative if no abnormalities are identified in the 22 fields of view or completely manually screened if potentially significant abnormalities are confirmed. The abnormal slides are then referred to a pathologist for interpretation and sign out.[3]

Automated screening of ThinPrep^® Pap Tests has become increasingly common in clinical practice. Large and busy laboratories in the United States as well as worldwide have adopted the screening methodology as a way to potentially increase productivity while increasing sensitivity and accuracy of cervical cytology screening. Previously published studies have demonstrated an increase in productivity by decreasing the amount of time cytotechnologists spend screening each slide. Using TIS, screening times decreased up to 42% compared with manual screening,[4] and productivity has increased by almost 30%.[5] Sensitivity and accuracy have not suffered from implementation of TIS in laboratories; multiple published studies reveal that TIS has at least equal sensitivity compared to manual screening in all diagnostic categories, and increased rates of detection, sensitivity and accuracy were found in multiple studies.[6–10] The rate of atypical squamous cells of undetermined significance (ASC-US) interpretations and the ASC-US/low grade squamous intraepithelial lesion ratio does not seem to be affected by the use of TIS.[711] In our laboratory, the rate of ASC-US diagnoses and follow-up reflex Human papillomavirus (HPV) testing was not significantly changed after implementation of the TIS.[12]

Depending on the population served, high-grade squamous intraepithelial lesion (HGSIL) diagnoses comprise on average approximately 0.7% of all cervical cytology specimens and high-grade squamous intraepithelial lesion not excluded (ASC-H), 0.4%.[13] When a cervical cytology specimen is interpreted as HGSIL or atypical squamous cells, ASC-H, the American Society for Colposcopy and Cervical Pathology (ASCCP) 2006 guidelines recommend at least a timely colposcopic examination and possible biopsies. In some HGSILs, the ASCCP recommends immediate loop electrosurgical excision, dependent on the woman's pregnancy status and age.[14] Even if colposcopic examination and biopsies obtain negative results, the patients will be followed more frequently with additional cervical cytology specimens. If the HGSIL and ASC-H rates were to increase due to the addition of TIS to cervical cytology screening, the sensitivity would increase, assuming all cases were true high-grade lesions. However, depending on the amount of increase in high-grade interpretations, the cost of management of abnormal cervical cytology could also increase substantially. In addition, if specificity is decreased a woman's reproductive health could be potentially jeopardized by unnecessary biopsy or excisional procedures.

A major goal of the Pap test is to identify high-grade cervical lesions. Women with HGSIL or ASC-H have a significant risk of a high-grade precancerous intraepithelial lesion or invasive carcinoma at the time of colposcopy. These lesions have a high rate of progression and require timely treatment. If the rate of HGSIL and ASC-H were to decrease due to TIS automated screening, the sensitivity for diagnosing clinically important high-grade lesions would likely decrease, and more importantly, potentially serious lesions could be missed and treatment may be delayed.

This study evaluates the detection rates and relative sensitivities and specificities of HGSIL and ASC-H diagnoses in a study population over a single time period randomly receiving manual versus TIS automated screening with histologic follow-up.

MATERIALS AND METHODS

Our laboratory processes and reviews approximately 60,000 Pap tests annually. The overwhelming majority of these specimens are collected and processed with the ThinPrep^® liquid based system; the remaining 5% of the cases consist of conventional Pap preparations and Sure-Path^® (BD, Franklin Lakes, MD) liquid based specimens. A mixed population is served by our laboratory including privately insured women and women from correctional facilities or subsidized health service programs. This study includes only ThinPrep^® Pap tests.

We began screening Pap tests with the TIS in September 2005, 10 months prior to the beginning of this study period. A computerized search of our database was performed for a 1 year period, July 2006 through June 2007. All cervical cytology cases diagnosed by the standardized Bethesda criteria as ASC-H or HGSIL were included in the study.[15] All cases were processed on the ThinPrep^® imager, but not all Pap tests were automatically screened due to a limited number of special automated microscopes. Determining cases for manual versus automated screening on TIS were assigned completely at random by our cytotechnologists. Cases from a group of gynecologic oncology physicians were always manually screened, therefore, these cases were excluded from inclusion in our data. The cases screened on TIS were determined by the inclusion of a standardized comment in the final pathology report.

The follow-up period extended from the initial Pap test until 2010. A computerized search of our database was performed to determine the follow-up biopsy or cytology procedures and diagnoses of the cases.

At the time of the study period, 12 cytotechnologists reviewed all manually and automatically screened cervical cytology and signed out all cases without atypical findings. No changes were made to cytotechnologists screening limits from those established for manual screening based on the incorporation of the TIS during the study period. On average no difference in screening volumes were observed during the study period. All cases with atypical cells or suspicious findings, whether manually or automatically screened, were referred to 1 of 4 pathologists for further interpretation and review. The follow-up cervical biopsies and cone excisions were interpreted in the surgical pathology department by 19 pathologists. Permission to conduct this study was granted by our institutional review board.

Descriptive statistics and Chi-square tests were performed for rates and histologic follow-up of the TIS and manual screened cases. To calculate sensitivity, the histologic follow-up results were used as the standard for comparison. Absolute sensitivity of TIS or manual screening could not be calculated because the number of false negatives and true negatives were unknown in each case as the sample size was large and histologic follow-up was not obtained on the Pap smears interpreted as negative. However, an estimation of the ratio of the two sensitivities was possible. The relative sensitivity is the number of true positives of TIS screened cases adjusted with sample size divided by the number of true positives of manual screened cases adjusted with sample size. In the same way, relative specificity was also calculated.

RESULTS

Over the 1 year study period, 70,522 Pap tests were processed in our laboratory, and slightly less than 1% was interpreted as HGSIL or ASC-H. HSIL was diagnosed in 235 (0.33%) patients, while 215 (0.3%) patients were diagnosed with ASC-H. The age of the women ranged from 19 to 79 (mean 34, median 30) in the HGSIL diagnostic category, and 15-84 (mean 36, median 30) in the ASC-H category.

Excluding those cases meeting exclusion criteria, 23,111, approximately one third (33%), of the cervical cytology specimens during the 12 month study period were screened with the help of TIS. Of the TIS screened specimens, 62 (0.27%) cases were diagnosed as ASC-H [Figure 1]. Similarly, 71 cases (0.31%) were interpreted as HGSIL [Figure 2]. The remaining 47,380 cases were manually screened. Of these manually screened cases, 153 (0.32%) specimens were ASC-H [Figure 3], while 164 (0.35%) were diagnosed as HGSIL [Figure 4, Table 1]. The difference in diagnosis rates for both ASC-H and HGSIL cases screened by TIS versus manual screen is not statistically significant (P = 0.216 and P = 0.399, respectively).

Close

Figure 1

Thin Prep^® imaging system screened cervical cytology specimen interpreted as atypical squamous cells, high-grade squamous intraepithelial lesion not excluded. Follow-up histology was cervical intraepithelial neoplasia II

Export to PPT

Close

Figure 2

Thin Prep^® imaging system screened cervical cytology specimen interpreted as HGSIL. Follow-up histology was cervical intraepithelial neoplasia III

Export to PPT

Close

Figure 3

Manually screened cervical cytology specimen interpreted as atypical squamous cells, high-grade squamous intraepithelial lesion not excluded. Follow-up histology was cervical intraepithelial neoplasia II

Export to PPT

Close

Figure 4

Manually screened cervical cytology specimen interpreted as HGSIL. Follow-up histology was cervical intraepithelial neoplasia III

Export to PPT

Table 1 The distribution of ASC-H and HGSIL cases TIS and manually screened

During the follow-up interval, histologic specimens in the form of colposcopic cervical biopsies or cone biopsies, including one hysterectomy, were available for 74% of ASC-H cases and 70% of HGSIL cases, whether TIS or manually screened. Follow-up cytology was available on the remaining cases without histologic correlation. Histologic follow-up was available for 44 TIS screened ASC-H cases and 111 manually screened ASC-H cases. Of those cases with surgical follow-up, high-grade cervical intraepithelial neoplasia (CIN II/III) was identified on biopsy of ASC-H cytology specimens in 41% of TIS cases and 45% of manually screened cases. CIN of any type, including low grade, were found in 70% of TIS cases and 80% of manually screened cases [Table 2]. The difference in the rate of high-grade confirmation on histology is not statistically significant (P = 0.388). Histologic follow-up was available for 63 HGSIL TIS screened cases and 138 HGSIL manually screened cases. In the HGSIL category, 71% of TIS and 69% of manually screened cases were found to be high-grade (CIN II/III) on histology. When including low-grade and high-grade lesions (CIN I, II, or III), 89% of TIS cases and 87% of manually screened cases were positive by histology [Table 3]. Again, the differences between TIS and manual screening in the HGSIL category on follow-up is not statistically significant (P = 0.422).

Table 2 Available follow-up histology for ASC-H cytology cases TIS and manually screened

Table 3 Available follow-up histology for HGSIL cytology cases TIS and manually screened

Relative sensitivities and specificities were calculated for ASC-H and HGSIL lesions both manually and automatically screened. For the ASC-H category, the relative sensitivity is 0.74, and the relative specificity is 0.43. There was a 26% (1-0.74) relative sensitivity difference between TIS and manually screened ASC-H cases. The relative specificity was in favor of manual screening by 57% (1-0.43) for ASC-H diagnoses. For HGSIL lesions, the relative sensitivity is 0.97, and the relative specificity is 0.85. The relative sensitivity of TIS to manual screening in the HGSIL category was comparable with only a 3% (1-0.97) difference. The specificity difference was 15% (1-0.85) comparing TIS HGSIL cases relative to the manually screened cases.

DISCUSSION

TIS automated screening practices have been compared to conventional Pap cytology and found to have similar or increased detection rates for high-grade lesions.[16–18] When comparing liquid based Pap cytology by screening method, the majority of previously published studies have indicated that automated screening increases productivity while meeting or exceeding the rate of detection of ASC-H and HGSIL lesions compared to manual historic controls. Comparing a newly implemented TIS system to a manually screened historical control, Loanzo et al., described a 38% increase in HGSIL detection rates using the TIS automated system.[7] In a similar study, again using the comparison of a historic control, Miller et al., reported that the TIS system significantly increased the rate of HGSIL diagnoses, but did not show a similar significant increase in the ASC-H diagnoses.[19] Papillo et al., found that TIS screened cytology increased the ASC-H rate of detection by 48% and the HGSIL rate by 24% compared to historical manual controls.[9] Chivukula et al., and Duby et al., found a statistically significant increase in HGSIL rates and an increase in ASC-H rates when comparing TIS screened cases to their pre-implementation cohorts.[1020]

In the previously published studies, the control for TIS screening was cytology cases screened manually previous to implementation of the automated system. While this design has the advantage of detecting changes in Pap screening brought on by the TIS, there are some potential problems with this methodology. Changes in clinical practice, the dynamics of the patients being screened, or the interpretative threshold of the cytotechnologist and cytopathologists, as well as staffing changes from 1 time period to the next may affect the results and confound the effects of TIS alone. We designed our study with the goal of avoiding these possible confounding factors and analyzed manual and TIS screened cases over the same time period and follow-up interval.

In an attempt to ascertain possible difficulties with TIS use, previous studies have reviewed their detection failures. In a study by Halford et al., comparing conventional cervical cytology to liquid based TIS screened cases, more than half of the false negative TIS screened cases on subsequent review of the slides displayed the abnormal cells in the extreme periphery of the 22 fields of view and were missed by the cytotechnologist.[16] In addition, a cytotechnologist survey in a study by Kitchener et al., reported monotony as a factor in using the automated systems for screening, which could lead to decreased concentration and focus by the cytotechnologist when reviewing fields of interest.[21] We did not find a statistically significant difference in the rate of ASC-H and HGSIL diagnoses when comparing manual to the TIS screening system. Although reasons for failure of abnormal cell detection in our cohort were not specifically studied, we can ascertain that the TIS system did not introduce enough screening diagnostic difficulties to affect the rate of high-grade detection.

The majority of the previously published studies were not designed for histologic follow-up after cytology screening, but a few previously published studies did review surgical follow-up after an ASC-H or HGSIL diagnoses with varied results in the HGSIL category. Duby et al., found similar rates of high-grade histology in the follow-up of the ASC-H Pap smears manually and TIS screened, but reported an increase in high-grade histology on TIS screened cases compared to the manual cohort.[20] Papillo et al., did not find a statistically significant difference in the positive predictive value in the ASC-H category, but found a significant increase in the HGSIL category.[9] Similar to our results, Loanzo et al., also reported statistically equivalent histologic follow-up diagnoses for the HGSIL category on manual and TIS screening.[7]

Histologic follow-up data provided the basis for relative sensitivity and specificity calculations as well as valuable information on the accuracy and clinical significance of both screening methodologies in our study. Histologic results were used as the standard for comparison to determine the accuracy of the Pap diagnoses by screening method. An increased rate of ASC-H and HGSIL lesions using manual screening signifies little clinically if the histologic correlations are not accurate or if manual screening leads to an increase in false positive high-grade lesions. When analyzing the surgical diagnoses, the TIS and manual screened groups did not show a statistically significant difference in the correlation percentage of high-grade histology in either the ASC-H or HGSIL categories. Once a high-grade lesion was detected on a Pap smear, the correlation on tissue was similar no matter how the cytology was screened; the accuracy of manual and TIS screening was not significantly different. A high-grade lesion detected by TIS is just as likely as a manual screened case to be a true high-grade lesion.

Varied methodologies were used to calculate sensitivity and specificity of manual and TIS screening in previously published studies. Histologic follow-up was rarely used as a means of comparison. Using retrospective reviews of the same slides for reclassification, or comparing original diagnoses to an adjudicated group of negative cytology slides, the majority of studies based the sensitivity and specificity calculations on cytologic outcomes. Using these cytologic comparisons, all studies found that the sensitivity of TIS screening either met or exceeded by varying degrees the sensitivity of manual screening in the high-grade diagnostic category.[6810] A recently published randomized controlled trial by Kitchener et al., studied the sensitivity and specificity of two automated systems, the TIS and the BD FocalPoint GS Imaging System (BD Diagnostics, Franklin Lakes, NJ, USA), relative to manual screening by comparing cytologic screening to histologic follow-up. Using CIN II histology as a cut-off point for high-grade lesions, Kitchener et al., reported a decreased sensitivity by 8% of TIS relative to manual reading and a modest 0.6% specificity increase.[21] Based on their findings, the authors concluded that automatic screening could not be recommended for primary cervical screening in the UK screening programs.

Our relative sensitivity calculations showed almost equal sensitivity and similar specificity in HGSIL diagnostic category when comparing TIS to manual screening. The differences were wider in the ASC-H category. Due to the small detection rates of HGSIL and ASC-H in our patient population and therefore small numbers of patients with histologic follow-up, the relative differences in sensitivity and specificity between manual and TIS screened cases is effected by the small number of cases for comparison. Though unattainable, increased patients numbers and follow-up information on those negative patients would better indicate the sensitivity and specificity differences between screening methods. However, although absolute calculations of sensitivity and specificity were unfeasible, the relative sensitivity and specificity still allows for a direct comparison between methodologies. While our study was limited by the retrospective design and small histologic data, the histologic follow-up and standard of comparison allows for a more rigorous analysis of the accuracy, sensitivity, and specificity of TIS relative to manual screening. Our findings and calculations indicate that either manual or TIS screening methodologies are acceptable for detecting patients with high-grade or possible high-grade cervical lesions.

Although our data suggests that manual screening is equal to TIS in the detection of high-grade lesions, several relevant issues warrant consideration. First, as the cervical cytology specimens are screened by cytotechnologists even when aided by the TIS system, their role as a factor in our results deserves attention. Manual screening will only be as effective as the comfort, skill, and experience of the cytotechnologist. If cytotechnologists have a high index of suspicion, a low threshold for abnormality, and the added benefit of experience, it is not unreasonable to believe that those individuals may be more effective at detecting high-grade abnormalities on screening, thus making manual screening as sensitive as automated screening for those individuals. Of additional interest is that the amount of cervical cytology slides screened per day by the cytotechnologists at our institution averages less than the limit set forth by regulatory bodies. The additional time available per screened slide may allow for more rigorous attention and focus to allow for a greater detection of abnormalities.

Due to the amount of time necessary to allow for histologic follow-up, our data is from 5 years ago when the TIS was relatively new to the institution. Learning curves have been described with automated systems, and we attempted to adjust for this phenomenon by designing our study period to begin almost a year from when the TIS was implemented. In addition, to further determine the effect of timing and learning curves on our automated laboratory rates of high-grade lesions, we collected the additional detection rate data from 3 years subsequent to our study period. The detection rates of ASC-H and HGSIL lesions were consistently similar by manual screening than TIS over the years. Learning curves did not appear to be a factor in detection rates of the TIS automated system indicated by our study.

Our data was examined for potential sources of bias, and attempts were made to adjust the data to exclude these factors. The distribution of cases assigned to manual or TIS screening was performed at random, except for cases from a group of gynecologic oncology physicians, which were always manually screened. These cases were excluded from inclusion in our data. At the time of the study period, three cytotechnologists preferred the TIS screening method and performed 75% of the TIS screening; therefore, the reading of the cases by the cytotechnologists was not completely randomized.

Of note is that our results were from an older TIS system, and an updated version is currently available. The benefits and changes of the updated system are not yet within our experience.

A final consideration of future study includes the group of women with negative diagnoses on cervical screening by both manual and TIS methods. Due to the large number of women in these subgroups, we are unable to estimate or determine false negative rates and determine the clinical significance of false negatives. Further study directions include determining the follow-up screening interval, subsequent diagnoses and comparison with prior negative results to determine the outcome in those manually and TIS screened with negative cytology. By determining the clinical significance of those false negatives cases, more valuable information could be gained about the relevant differences and clinical impact of manual versus automated screening.

COMPETING INTERESTS STATEMENT BY ALL AUTHORS

There are no competing interests to declare by any of the authors.

AUTHORSHIP STATEMENT BY ALL AUTHORS

All authors of this article declare that we qualify for authorship as defined by ICMJE http://www.icmje.org/#author.

Each author has participated sufficiently in the work and takes public responsibility for appropriate portions of the content of this article.

BK carried out the review of the data and drafted the manuscript. DR carried out the initial data collection and organization and participated in the design of the study. NL carried out the statistical analysis and drafted the statistical methods portion of the draft. TB participated in the design of the study and helped to draft and edit the manuscript. SV conceived of the study, participated in its design and coordination, and helped to draft and edit the manuscript. All authors read and approved the final manuscript.

Each author acknowledges that this final version was read and approved.

ETHICS STATEMENT BY ALL AUTHORS

This study was conducted with approval from Institutional Review Board (IRB) of all the institutions associated with this study as applicable.

Authors take responsibility to maintain relevant documentation in this respect.

EDITORIAL / PEER-REVIEW STATEMENT

To ensure the integrity and highest quality of CytoJournal publications, the review process of this manuscript was conducted under a double blind model (authors are blinded for reviewers and vice versa) through automatic online system.