Informatics applied to cytology

Test management and reporting systems

This process includes monitoring several variables as a part of a QC/QA program. Laboratory requisition completeness can be assessed upon arrival of the specimen in the laboratory and the problems documented by the accessioner. Most software solutions have fields where QA comments can be entered. Periodic reports can be run to list occurrences and identify trends with any particular physician office sending specimens to the laboratory. Identification of problems could prompt consideration for clinician education and/or redesign of requisition forms. The timeliness of specimen transportation can be assessed by comparing the date of specimen procurement with the date received by the laboratory. Specimen rejection incidents and labeling errors should be documented, and regular reports run to monitor specimen rejection frequency by the clinician's office. Comments entered within available QA fields could be included in the final report sent to a clinician. Electronic monitoring of lost specimens is best accomplished with a simple spreadsheet log, since such specimens would not be accessioned into the LIS database.

Electronic data integrity

Electronic data integrity is a quality indicator that includes common network security and features to ensure that data is entered in a standardized format. Common security measures may include regular back up of data, password protection, data encryption, as well as the use of antiviral software, firewalls, and audit trails. These measures help meet the security and privacy needs related to health data, as proposed in the HIPAA.[20] Dictionaries (maintenance tables) can be built to determine which LIS users have privileges to specific fields. Assigning users different privileges in the LIS allows only certain individuals (e.g. a pathologist) to finalize and sign out abnormal gynecologic diagnoses. Audit trails can be used as a part of a comprehensive QC/QA program to ensure that users only access information relevant to their job duties. Integrity of finalized reports typically entails an electronic securing function, so that content cannot be changed or added to, except in the form of an addendum to an existing report or an amended report, thereby ensuring preservation of the original report.

Data entered using standardized language for each field is more efficiently retrieved and easily analyzed. For this reason, the use of standardized language is encouraged in the practice of cytopathology. Standardized data entry for diagnostic interpretations can be assisted through the use of coded comments, which ensure uniformity of the reporting language. In settings where standardized diagnostic terminology exists and is widely accepted, such as The Bethesda System for the reporting of Pap test results,[21] coded comments allow for quick data entry and unambiguous result reporting to the clinician. Although an internationally accepted unified terminology for the reporting of Pap tests is still required, many developed countries have adopted The Bethesda System or a slight modification thereof.[22–24] In other settings, such as fine needle aspiration biopsies and nongynecologic specimens, greater flexibility in the use of language is frequently needed. Reports that cue from standard codes provide an efficient means to extract data, enabling swift calculations of abnormal rates and breakdown of diagnostic categories for comparison against benchmark data. This can be done for the entire laboratory, as well as for individual cytotechnologists and cytopathologists.

Mandatory entries are useful features present in the LIS. With regard to Pap test reporting, for example, mandatory entries may include the following: specimen adequacy statement, primary diagnosis, billing codes, name of the cytotechnologist (primary screener) and initial diagnosis, and, if the case is forwarded for pathologist review, the pathologist's name and final diagnosis. With such a system in place, reports cannot be issued unless all the mandatory fields are completed. Ideally, the primary cytology diagnostic code should always remain assigned to the case, regardless of what is typed in lieu of, or in addition to, the primary coded remark. This process will ensure that future data analyses based upon retrieval by diagnostic codes are accurate and meaningful. Queries based upon the text entered (i.e. natural language search) are possible with many current laboratory information systems. However, something as simple as typographical errors may interfere with searches based on text alone. This can be avoided by utilizing common built-in software features such as a spell check and automated comments in the LIS.

Workflow indicators

Turnaround time (i.e. the time it takes to provide the patient's caregiver with a cytology report from the time the Pap test specimen was received within the laboratory) provides an excellent gauge of workflow management.[25] As quality in reporting and diagnostic accuracy are very often assumed by both the clinician and the patient, turnaround time unfortunately serves as the major ‘quality’ indicator to both clinical office personnel and the public. As such, the impact of efficiently managing the laboratory to effectively reduce this time cannot be overstated. Reports can be generated to calculate turnaround time, identify outliers, investigate the cause of a delay, and implement corrective actions. Trail sheets can be used to track locations of specimens, should they be sent to different departments or locations outside the laboratory (e.g. to a reference laboratory).

Retrospective reviews may involve randomly selecting a predetermined number of cases from the previous month and checking them for timely retrieval, appropriate storage, and reporting accuracy. Errors should be documented and procedures reevaluated. If necessary, they should be revised. In the United States, CLIA regulations mandate a retrospective review of all negative Pap tests for the last five years for which current cases are diagnosed with a high grade squamous intraepithelial lesion (HSIL) or above (i.e. cancer). A five-year retrospective review has been noted to highlight problem areas for laboratory education and quality improvement efforts, although liability concerns have been reported to be a detrimental consequence of this regulation.[26] Clarification that amended reports be issued only when current patient care is affected, as has been proposed by some authors, should aid laboratories in complying with this regulation.[26]

Clinical Laboratory Improvement Amendments (CLIA) also requires the cytology laboratory to provide and document whether alert information (typically in the form of a medical director letter) was sent to providers, with regard to all patients with a Pap test diagnosis of HSIL (or carcinoma) in which there is no record of further patient follow-up in the pathology or hospital information system. This ensures that those patients at high risk for cervical cancer get appropriate timely intervention and treatment. In order to meet this requirement, laboratory software can be designed to search for cases of HSIL (or carcinoma) during a specific time period (e.g. prior three or four months) and then identify any cytology or surgical specimen(s) that have occurred since that designated period. In the United States, although a Pap test interpretation of HSIL is the trigger for this federal requirement, many laboratories include other ‘lesser’ Pap test interpretations as triggering events, specifically those which also carry a significant risk of the patient harboring a high-grade precancerous cervical or vaginal lesion, or potentially invasive carcinoma. In this way, the quality monitor can be modified to meet the goals and expectations of one's local medical community (and medico-legal environment), based upon the judgment of the laboratory medical director. If no such subsequent specimens can be found in the LIS database, letters that alert the appropriate clinician to this situation and request follow-up information can be generated.

Screening and performance indicators

Quality indicators that are related to screening and performance are both subjective and objective in nature. An important component of any accepted QC/QA program in cytology includes provisions for setting up the maximum workload for individual cytotechnologists. This can be determined by evaluating diagnostic accuracy as it compares to productivity, and is typically described in terms of slides per hour. Federal law in the United States requires that cytotechnologists manually document the number of slides screened within each 24-hour period, and the number of hours spent screening each day. Guidelines prohibit screening more than 100 slides (conventional Pap smears and/or liquid-based Pap tests) over an 8-hour period, and not more than 12.5 slides per hour. It is generally accepted that higher per hour and per day slide counts are permissible when screening Pap tests which have been imaged and screened by computer systems, as they allow the cytotechnologist to review a smaller surface area microscopically per negative slide. However, this matter is complicated (e.g. there is a learning curve inherent with new technology that needs to be taken into consideration) and has been addressed in the guidelines published by the Cytopathology Education and Technology Consortium (CTEC).[27] In a similar fashion, nongynecologic liquid-based preparation slides also count as half-slides in such daily counts. Slides per hour can be calculated by adding the total number of slides and dividing them by the total hours spent screening, which is determined by subtracting time spent doing nonscreening activities from the hours worked. Software solutions can be used to ensure that workload limits are not exceeded. For example, once the daily workload limit has been reached for an individual cytotechnologist, the LIS will not allow him to sign out additional cases.

Common indicators of diagnostic accuracy include rescreening measures. Rescreening is mandated by the CLIA in the United States, but has also been implemented as a measure of QA in screening Pap tests in other countries.[28] This includes a 10% random rescreen of negative Pap tests and a rescreening of a specified percentage of negative cases defined as ‘high risk’. All rescreening should be performed by a supervisory level cytotechnologist or a pathologist. Several studies have shown that a 100% rapid rescreening method is more efficient at detecting false negative results than 10% random rescreening or rescreening on the basis of clinical criteria, and this has been recommended as an internal QC method.[29–30] However, this may not be practical in a large cytology laboratory screening a high volume of cases or one with limited staff. Software can be relied upon to allow the percentage of cases pulled for QC review to be adjusted and this can be set differently for individual screeners. Situations where this may be useful would be performance indicators that mandate closer inspection of work quality, a tool for remedial training, or for new hirees whose work quality is not yet fully ascertained. Quality Control (QC) rescreening is most useful for exposing screening errors, as opposed to diagnostic errors. Accordingly, software often has data entries that allow the degree of error to be defined (e.g. minor disagreement for the missed presence of Candida or major disagreement for a missed squamous intraepithelial lesion/SIL).

A prospective random 10% rescreen (re-examination prior to reporting), typically determined by laboratory software, automatically marks for QC each 10th consecutive negative case entered by a primary screening cytotechnologist on a lab-wide basis. If the laboratory has many cytotechnologists signing out cases simultaneously, the 10% should represent a random selection that is unpredictable from the perspective of each cytotechnologist. Once selected for QC, the laboratory software should ensure that the case cannot be modified by the primary screening cytotechnologist. This case must also have a documented review by a person at an appropriate level, defined in the person dictionary (e.g. supervisory cytotechnologist or pathologist). Reports should be generated to verify that each cytotechnologist has indeed met the minimum requirement of 10% QC rescreen of negative Pap test cases.

Cases are designated as high risk, based upon past and/or current history of abnormal signs, symptoms and/or pathologic findings. The definition of ‘high risk’ varies widely between laboratories, as it is subject to continual adjustment, based upon the specific patient population served, evolving scientific data, development of new clinical condition states (e.g. HIV/AIDS), new testing modalities (e.g. HPV DNA testing), and the judgment of the laboratory medical director. Software solutions can automatically designate cases as high risk, based on database searches for particular entries within certain fields with consideration for a specific time period (e.g. ‘History of LSIL’ in current field of ‘clinical history’, or a previous specimen coded as abnormal in the primary diagnosis interpretation field within the past three years). Computer-assisted designation of high risk cases greatly increases the likelihood that high risk cases are correctly identified to be a part of a secondary review process prior to sign out. Since cases defined as ‘high risk’ are theoretically at higher risk for containing abnormalities, patient care is improved with the (typically small) proportion of negative cases by primary review that are found upon secondary review to contain abnormal findings.

A large component of workload assessment for cytotechnologists involves comparing diagnostic agreement between the cytotechnologist and the reviewing pathologist for each case forwarded for pathologist review. This is most critical for the negative Pap test by cytotechnologist review. As cytotechnologists function not only as Pap test screeners but also as Pap test diagnosticians, solely evaluating and reporting the majority of negative Pap tests and thus the overall majority of all patient Pap test results, the evaluation of their ability to accurately determine a Pap slide as negative is crucial. All Pap tests initially deemed negative for intraepithelial lesion or malignancy by cytotechnologist review, which then undergo second review due to standard QC practices (10% or high risk rescreen) or pathologist review due to the presence of reactive changes, serve as QA data which must be analyzed. Pivot tables, created after downloading data into spreadsheets, are useful tools to cross reference diagnoses by cytotechnologists and pathologists and for analyzing diagnostic correlations and discrepancies of Pap tests, or in addition nongynecological cases. Areas of cytotechnologist over-interpretation and more importantly under-interpretation (potential false negative cases) can be easily assessed on both an individual cytotechnologist or laboratory-wide. Software systems can also allow for individual cases to be easily retrieved for continuing education purposes.

Additional performance monitors that can be tracked include the rates for frequencies of diagnostic categories (e.g. atypical squamous cells of undetermined significance or ASC-US) and the ASC/SIL rates,[31] which can be tabulated on a monthly, quarterly, and/or six month basis for each cytologist and compared against the department as a whole, as well as benchmark data where available. Remedial action may be taken if an individual average exceeds a predetermined variance from the norm (e.g. two standard deviations from the laboratory average). If rates fall outside this range, this will prompt a review and appropriate continuing education of individual and/or lab performance by the medical director. With the increasing availability of Human Papilloma Virus (HPV) DNA testing of liquid-based Pap test material, the ASC-US HPV DNA positivity rate can similarly serve as a useful quality indicator.[32] This rate is particularly informative, with regard to pathologist performance and appropriate utilization of the ASC-US category, providing excellent and timely feedback with well-accepted target performance benchmarks. Performance beyond two standard deviations (SDs) of the mean, for HPV-positive rates in ASC-US Pap tests, has been recommended by some authors to prompt reassessment of diagnostic criteria used in the evaluation of Pap tests and/or investigation of the prevalence of HPV positivity in the population from which the Pap tests are obtained.[33] One can also note the proportion of particular diagnostic categories forwarded for pathologist review by each cytotechnologist, and observe, with discrepancy logs, the percentage of their negative cases signed out by the pathologist as abnormal.[34]

For QC purposes, cytology laboratories need to compare Pap test and cervical biopsy reports (if available) and determine the cause of any discrepancies.[35–36] Such cytologic-histologic correlation should be compiled at regular (e.g. six month) intervals. Continuous monitoring of cytologic-histologic correlation data has been shown to be associated with improvement in cervical cancer screening performance (e.g. higher predictive value of a positive Pap test and superior sensitivity).[37] This entails comparing Pap test data to database searches of subsequent surgical specimens (e.g. colposcopic biopsies or excisional cone procedures). The tissue specimen is often regarded as the ‘gold standard’, although colposcopic examination, even with the most experienced colposcopist, has relatively limited sensitivity. With the previously described QC procedures, specifically 10% and high risk rescreen and cytotechnologist vs. pathologist diagnosis correlation data, coupled with Pap test vs. subsequent biopsy correlation data, potential false negative and false positive Pap tests can be identified and assessed for both individuals and the laboratory as a whole. The subgroup of potential false negative cases should be examined for assessing issues of sampling (cells representative of the lesion are absent or in very low number), as well as examination of the potential false positive Pap slides for confirmation of the abnormal finding as originally diagnosed (colposcopic biopsies missed the lesion). In the latter situation, if the abnormal Pap finding is confirmed, it is imperative to reinforce this to the clinician, so that further biopsy, LEEP excision, or close Pap follow-up can be scheduled. If Pap case selection by the cytotechnologist is random, as well as an equally distributed or random interaction of the cytotechnologist and pathologist pools, then the rates of false negative Pap interpretation can be compared and contrasted between individual cytotechnologists. In large cytology laboratories, interpreting thousands, if not hundreds of thousands of Pap tests, this can only be accomplished utilizing the LIS through QC/QA software tools. One common indicator, amongst others, is the false negative rate (FNR) (also known as the false negative fraction or false negative proportion), which is the proportion of abnormal Pap tests that are falsely negative, i.e. FNR = false negatives/(true positives + false negatives).[13]