Is 58% sensitivity for detection of cervical intraepithelial neoplasia 3 and invasive cervical cancer optimal for cervical screening?

The recent recommendation by a Food and Drug Administration (FDA) panel of the Medical Devices Advisory Committee for approval of a Roche Cobas HPV test application as a first line primary cervical screening tool in women 25 and older has been the subject of considerable medical news coverage.[1234] Following approval by the FDA on April 24 of the committee recommendation, clinicians in the US face a new and more complex set of options for FDA-approved routine periodic cervical screening. Available choices will include: (1) cytology alone for women 21 and older; (2) cytology with adjunctive reflex HPV testing after atypical squamous cells of undetermined significance (ASC-US) cytology results; (3) cytology with routine adjunctive HPV co-testing of women 30 and older; and (4) Roche's new algorithm for primary HPV screening of women 25 and older. According to the New York Times, Johns Hopkins University's Dorothy Rosenthal testified to the Committee that “there would be a tremendous gain by moving to the new test”.[1] More cautiously, Dr. Alan G. Waxman, a panel member and professor of Obstetrics and Gynecology at the University of New Mexico commented-“I think women are going to be well served by having more choices, but it's going to be very interesting to watch over the next several years as this rolls out”.[1] What exactly did the data presented to the FDA document regarding this new option for cervical screening?

The FDA panel recommendation was based on review of data from the addressing the need for advanced HPV diagnostics (ATHENA) trial, Roche's US - based cohort prospective trial which enrolled and screened more than 47,000 women for cervical disease and analyzed the impact of proposed screening algorithms based on these data.[5] A few days before the March 12, 2014 committee panel meeting, an FDA summary and analysis of the trial data was posted on the FDA website.[6] Highlights of the summary included: (1) The sensitivity for ≥ cervical intraepithelial neoplasia three (CIN3) was 58.26% for the cobas HPV test algorithm compared to 42.63% for the Pap test algorithm; (2) the risk of ≥ CIN3 (positive predictive value) in women referred to colposcopy by the cobas HPV test algorithm was 12.25% compared to 6.47% for the women referred by the Pap test algorithm; (3) the risk of ≥ CIN3 in women who were not referred to colposcopy by the cobas HPV test algorithm (negative predict value) was 0.42% and 0.59% among the women not referred by the Pap test algorithm; (4) the false-positive rate (specificity) for ≥ CIN3 was 4.09% for the cobas HPV test algorithm compared with 6.04% for the Pap test algorithm.

Perhaps, the most surprising findings of the study were the unexpectedly low verification bias-adjusted CIN3+ sensitivities for both the cobas HPV test (58.26%) and Pap testing (42.63%) algorithms in the trial. Recent modeling studies seeking to estimate the impact of primary HPV testing on incidence and morbidity of cervical cancer in have, for example, assumed that HPV testing can achieve a much higher sensitivity of 95% when cytology is negative and 97% when cytology is positive.[7] Other studies have reported verification bias-adjusted sensitivity for detection of CIN2/3+ with optimized liquid-based cytology (LBC) exceeding 90%.[8910] Why then were the sensitivities of both cobas HPV test and cytology so low for detection of CIN3+ in the ATHENA trial? Furthermore, what are the implications of these findings for application of the new proposed primary HPV testing algorithm in clinical practice?

The ATHENA trial study design acknowledged that crude estimates of cervical disease prevalence would result in verification bias, as all women with positive Pap tests or HPV results in the ATHENA trial were selected to undergo colposcopy, whereas only a much smaller subset of women with both negative Pap and HPV test results were randomly selected to undergo colposcopy.[5] Verification bias adjustment was applied to account for the difference in rates of selection to colposcopy by calculating the likely number of CIN2/3+ cases that would have been found if all women had undergone colposcopy for disease verification. Overall there were 274 cases of CIN3+ and 157 cases of CIN2 identified in the trial, of which nine cases of CIN3 and 20 cases of CIN2 were documented in women referred to colposcopy after “double negative results” (DNR) for both HPV and Pap testing.[6] Extrapolating from the nine cases of CIN3 found in colposcoped and biopsied doubly negative women, there were another estimated 160 (roughly) CIN3 cases in the trial that were not detected; these additional estimated CIN3 cases caused the CIN3+ sensitivities of both the HPV and Pap testing algorithms to decrease from crude unadjusted estimates.[11]

The FDA summary specifically cautions that the results of the trial cannot be readily compared to the results of most randomized clinical trial studies conducted with nonadjustable verification bias. Nonetheless, the verification-bias adjusted sensitivities calculated in the ATHENA trial turned out to be remarkably similar to estimates of “near 50%” for verification bias-adjusted sensitivity with the conventional Pap smear made by the Agency for Healthcare Policy and research around the time new cervical screening technologies began to be introduced in the late 1990s.[12]

Very low verification bias-adjusted CIN3+ cytology sensitivity of 42% was also calculated for laboratories utilizing LBC in the ATHENA trial.[6] The 7% average abnormal rate documented overall for the four US laboratories[5] place them as a group in the lowest 25^th percentile of US laboratories, based on College of American Pathologists (CAP) benchmarking data.[13] Furthermore, data on the ASC/squamous intraepithelial lesion ratios of these laboratories place them in a laboratory performance category linked to suboptimal screening sensitivity.[14] Performance of these four US cytology laboratories[15] appears particularly suboptimal when compared to the performance of United Kingdom (UK) laboratories competing against hybrid capture two HPV testing in the first two rounds of the ARTISTIC trial.[16] The performance of the UK LBC laboratories was so strong that the UK investigators commented: “It is difficult to escape the conclusion that LBC was more sensitive in ARTISTIC than earlier conventional cytology”.[17] In contrast, the low sensitivity of LBC in the ATHENA trial can best be attributed to suboptimal cytology screening performance within the selected four large laboratories. No data on cytotechnologist daily workload, a well-known factor potentially impairing cytology sensitivity, is available in the ATHENA trial publications or FDA summary.[18] Suboptimal cytology screening would impact not only the cytology arm of the ATHENA trial but also the cytology triage component of the HPV arm.

Another factor possibly impacting the ATHENA trial data is misclassification of CIN2/3 biopsy interpretations.[1920] Cervical biopsy interpretations in the ATHENA trial were performed on hematoxylin and eosin, stained histopathology slides by three pathologists working with the manufacturer. p16^INK4A (p16) is a cyclin-dependent kinase inhibitor involved in regulation of cell proliferation,[21] is overexpressed in high-grade cervical neoplasia, and is readily detectible with immunohistochemical staining, potentially improving the reliability of CIN interpretations.[2223] When p16 stains were performed on a subset of eight biopsies of ATHENA trial patients with CIN2/3 diagnoses and negative cobas HPV test results, the pathologists concluded that five of the eight CIN2/3 diagnoses (62.5%) were misclassifications.[19] According to one of the pathologist-investigators answering queries from the FDA panel, p16 staining was only performed on cases that were cobas negative; however, after further panel questioning the same investigator acknowledged for the first time “other studies” with “wider p16 immunostaining on cases with adjudicated diagnoses of CIN2 and CIN3”[24] No p16 data is disclosed in extensive trial-related materials posted on-line by the FDA before the panel meeting. Misclassification in a subset of ATHENA trial CIN2/3 biopsies raises troubling questions about the extent of misclassification of all the CIN2/3 biopsies, both in women with positive and negative cobas HPV test results.[20] P16 evaluation of all CIN2/3 biopsies following discordant Pap and HPV results or following DNR would be of particular interest.

Since verification bias-adjustment is especially sensitive to disease verification and extrapolation from the very limited number of CIN2/3 cases detected after DNR, it is plausible that additional CIN2/3 biopsy misclassifications could substantially alter the verification bias-adjusted sensitivities for each of the arms of the trial. Furthermore, since roughly 12% of ATHENA trial patients tested cobas HPV-positive, while only 7% had abnormal (ASC-US+) cytology results,[5] any trend toward overdiagnoses of CIN2/3 may have disproportionately contributed to increased CIN2/3 diagnoses in the HPV arm of the trial. Only p16 imunohistochemical staining of additional CIN2/3+ biopsies in the ATHENA trial and independent adjudication of the results can clarify these uncertainties.

Although, FDA panel approval of the cobas HPV test application focused mainly on comparing trial results projected from using the cobas primary HPV testing algorithm with results projected from using cytology alone, the manufacturer also compared its proposed primary HPV testing algorithm with other trial results projected with the use of routine ASC-US triage for women ≥25 and co-testing for women 30 and older.[6] According to a brief summary of the microbiology devices panel meeting, the manufacturer asserts that their proposed cobas HPV primary screening algorithm “provides similar protection against CIN3 and invasive cervical cancer as cytology and HPV co-testing”.[25] This is highly questionable, however, as noted by the United States preventive services task force - “the degree of benefit in preventing invasive cervical cancer cannot be determined from test performance studies alone. Such studies suffer from determining sensitivity, specificity, and related values for a surrogate outcome (CIN2/3+) and not invasive cervical cancer”.[26] In available published US studies utilizing conventional Pap smear and hybrid Capture two HPV co-testing at a 3 year screening interval after DNR, projected cervical cancer rates over a 5 year period were 10-15% lower with co-testing than estimated using HPV testing alone.[27] At the FDA hearing, data were also presented to the panel from Quest Diagnostics’ internal clinical database with results from 3,727,894 women ages 30-65 who had co-testing in 2005-2007. Based on Quest data and American Cancer Society estimates of 12,360 women expected to be diagnosed with cervical cancer this year, quest estimated approximately 1,670 women (13.5%) would be undiagnosed with primary HPV screening alone.[28] Co-testing is the current preferred cervical screening strategy for women 30 and older in consensus guidelines from the American Cancer Society, the American Society for Colposcopy and Cervical Pathology, and the American Society for Clinical Pathology.[29]

Food and Drug Administration approval of the cobas HPV test application as a first line primary cervical screening tool in women 25 and older will introduce a fourth FDA-approved screening option and an unprecedented level of complexity in cervical screening choices. Available data indicates that the greatest protection against cervical cancer is achievable with cytology and HPV co-testing at 3 year intervals.[2730] The verification bias-adjusted CIN3+ sensitivity of 58.26% documented by the FDA with the new cobas HPV test screening algorithm is not so different from the “near 50%” sensitivity estimated by the Agency for Health Care Policy and Research for conventional Pap smear screening in 1999.[12]

Due to this low documented CIN3+ sensitivity for Roche's new primary cobas HPV testing algorithm, we believe most clinicians and patients will continue to choose cytology and HPV co-testing as their preferred option. Choice of co-testing is also supported by available studies documenting HPV false-negative rates around 10% for women tested with FDA-approved HPV tests with established cervical cancer diagnoses.[31323334]

Furthermore, reported false-negative HPV rates have increased to 31% for baseline HPV testing of women developing cervical cancer over the next 5 years[27] and to 42% for baseline HPV testing of women developing cervical cancer over the next 8 years.[35] Optimized LBC, as utilized in the UK ARTISTIC trial, appears to increase protection against cervical cancer when compared with conventional smear trial data.[36] In contrast, suboptimal LBC, as documented in the ATHENA trial, clearly led to less favorable results. Choosing quality cytology laboratory services carefully remains as important as ever, even in this new era in which HPV testing is playing an increasing role in cervical screening.

Newer technologies such as HPV testing and p16 are significant advances in cervical screening and diagnosis. Nevertheless, disturbing the time tested the role of periodic cervical cytology, which has dramatically reduced cervical cancer should be taken very seriously. In addition to an immediate risk of confusion at the clinical practice level, any approach relying on newer methodology without data on the long-term impact on cervical cancer in screened populations could lead to results that are less favorable than anticipated.[37]

COMPETING INTERESTS STATEMENT BY ALL AUTHORS

Both authors declare that they have no competing interests.

AUTHORSHIP STATEMENT BY ALL AUTHORS

Each author has participated sufficiently in the work and takes public responsibility for appropriate portions of the content of this article.