JMD 2007, Vol. 9, No. 1
Copyright © 2007 American Society for Investigative Pathology & Association for Molecular Pathology
Development of a Web-Based Query Tool for Quality Assurance of Clinical Molecular Genetic Test Results
Matthew J. McGinniss*,
Rebecca Chen
,
Victoria M. Pratt
,
Arlene Buller*,
Franklin Quan*,
Charles M. Strom*,
Weimin Sun* and
Beryl Crossley
From the Genetic Testing Center
* and Advanced Diagnostics Information Technology,
Quest Diagnostics Nichols Institute, San Juan Capistrano, California; and the Department of Molecular Genetics,
Quest Diagnostics Nichols Institute, Chantilly, Virginia
 |
Abstract
|
|---|
The College of American Pathologists molecular pathology checklist item (MOL.20550) calls for periodic review of molecular genetic statistics, including percentages of normal and abnormal findings and allele frequencies. A web-based query tool application for clinical molecular genetic test results was developed to plot dynamically and display genotype and/or allele frequencies for any time period. This tool is used to produce plots of all high-volume molecular genetic assays (>50 samples per month). A single web page contains pull-down menus, enabling the user to select the type of chart to be generated (genotype or allele frequency), the molecular genetic assays to chart (from one to all), the ending date for data in the chart (month and year), and the duration of the time period to plot (1 to 12 months). The rendered graphical and textual frequency data can then be viewed or printed. This tool can be used by any laboratory and interfaced with a standard laboratory information system. Monthly quality control charts and tables are now generated in minutes compared with the hours it took using manual charting applications. This simplified process enables timely compliance with a College of American Pathologists checklist item.
|
Introduction
|
|---|
Quality assurance monitoring is an important part of clinical laboratory testing and is used to help identify errors or to identify adverse trends that could eventually result in errors. Quality assurance measures are the collective actions involving policies, procedures, and corrective actions taken to ensure that test results are accurate.1
These actions span the gamut of preanalytic (acquire the specimen), analytic (test the specimen), and postanalytic (report and transmit test results in accurate and in a timely manner) phases of clinical testing.
Clinical molecular genetic testing is a relatively new discipline when compared with routine clinical chemistry and clinical pathology. However, a variety of features are common to all laboratory testing, and they include the use of validated tests; the use of independently validated reference materials, such as cell lines or certified DNA samples with known genotypes as positive controls; the provision of adequate staff training; adherence to best practice guidelines; and laboratory accreditation(s).1, 2, 3, 4, 5
Unlike routine clinical chemistry testing, clinical molecular genetic test results are usually qualitative and are not amenable to the standard methods of continuous monitoring and measurements that are routinely applied to monitor results of automated clinical chemistry testing. 5
This tool was developed in part to achieve compliance with the College of American Pathologists (CAP) molecular pathology checklist item (MOL.20550 Molecular Pathology Checklist: 10/6/2005 edition, available at http://www.cap.org/apps/docs/laboratory_accreditation/checklists/molecular_pathology_october2005.doc; accessed January 2006), which calls for periodic review by the laboratory director (or designee) of molecular genetic statistics including percentages of normal and abnormal findings and allele frequencies. Before the development of this query tool, these plots were generated manually by using the chart applications for Microsoft Excel with data from our proprietary molecular genetics database in which the results of all DNA-based molecular genetic testing studies are stored. This complex and labor-intensive process was a hindrance in achieving routine and timely compliance with this checklist item. We desired a simple, automated method to prepare and review charts of molecular genetic assay results on a routine basis. We chose to develop a web-based query tool of our current molecular genetic database as an efficient method to generate charts of molecular genetic assays for quality assurance purposes.
|
Materials and Methods
|
|---|
Initial molecular genetic data in quality control (QC) charts were manually constructed from raw data in our molecular genetics database using Excel-based charting applications. These were constructed as charts of genotype and allele frequencies over a rolling 6-month period. Charts were meant to be constructed monthly, but due to the labor-intensive aspect of this work, it was difficult to achieve routine and timely review of these QC charts. The web query tool was written in PHP, a web scripting language, and developed on a Microsoft Windows 2000 server with Apache Web server 2.0.55. It uses SQL queries and stored procedures to extract allele and genotype frequencies from our SQL2000 proprietary relational database in molecular genetics at both the San Juan Capistrano, CA, and Chantilly, VA, laboratories. This type of query tool is independent of our proprietary database and could be used with minor modifications with any relational database by any laboratory and could be interfaced to a standard hospital or laboratory information system.
|
Results
|
|---|
Our web-based query tool for monthly QC charts was designed to produce charts and accompanying tabular data summaries for all moderate to high-volume (>50 test results per month) molecular genetic assays. Included in our design specifications was the ability to generate plots of genotype and allele frequencies. In addition, we wanted the user to have the option of reviewing tabular summaries of the testing data along with the charts. This is because many of the tests we run give wild-type or normal results for the majority of samples tested, and graphical displays of percent heterozygous carriers over time may not reveal significant departures from past trends because the carrier rates are typically less than 5%. Thus, we felt it was important to review a brief tabular summary containing the data for the number of normals (w/w), heterozygotes (wt/mutant), and homozygous mutant individuals (mutant/mutant) each month. The abbreviation "wt" is used to denote a normal or negative test result.
To comply with the security and privacy issues with the Health Insurance Portability and Accountability Act and CAP, access to the web query tool is restricted to Quest Diagnostics Incorporated personnel and is further password-protected. The web interface for our query tool contains a window (Figure 1)
with pull-down menus enabling the user to select the type of chart to be generated (genotype or allele frequency), the molecular genetic assays to chart (from one to all), the ending date for the data to be included in the chart (month and year), and the duration of the time period to plot (1 to 12 months). Also included is a tab for a chart preferences page to be used to edit the chart and tabular summary outputs, as well as a tab for a page containing frequently asked questions. This web query tool connects authorized users to molecular genetics statistics regardless of their location within the Quest Diagnostics Intranet system.
View larger version (39K):
[in this window]
[in a new window]
|
Figure 1. A screen shot of the web query tool for QC charts for molecular genetic assays. This web page allows the user to select the type of chart to be generated (genotype or allele frequency), the molecular genetic assays to chart (from one to all), the ending date for data in the chart (month and year), and the duration of the time period to plot (6 months, 1 year).
|
|
The graphic output of the chart generated for the genotype frequency of the hereditary hemochromatosis assay results from April 2005 through September 2005 is shown in Figure 2
. This chart shows the frequencies of observed genotypes (as percentage of total samples) for the time period of April through September 2005 at Quest Diagnostics Nichols Institute, San Juan Capistrano. The raw tabular data summary showing all of the monthly totals for each genotype (data not shown) is printed just below this chart. These tabular summaries are very useful, because for many assays such as Tay-Sachs disease where the frequency of heterozygotes is low, the tabular summary allows for a quick visual inspection of the numbers of heterozygotes and mutant homozygotes reported each month. Any significant departures from past trends would be evident in the tabular summary but not particularly evident in the chart showing genotype frequencies over time.
View larger version (23K):
[in this window]
[in a new window]
|
Figure 2. Graphic output of the chart generated for the genotype frequency of the hemochromatosis assay through September 2005. This chart shows the frequencies of observed genotypes from April through September 2005 at Quest Diagnostics Nichols Institute, San Juan Capi-strano, CA.
|
|
A comparison of graphic output of the charts generated for the factor V Leiden (FV) assay performed at two different laboratories (San Juan Capistrano and Chantilly) (Figure 3)
shows that testing is stable at both laboratory locations and that the results of testing are comparable. This is despite the fact that the mutation detection platform currently used at San Juan Capistrano (READIT SNP Genotyping System; Promega Corporation, Madison, WI) differs from that used at the Chantilly laboratory (factor V Leiden Invader assay; Third Wave Technologies, Madison, WI). The carrier frequency of the factor V Leiden mutation ranges from approximately 8% in those from the UK to 15% in those from Greece.6
Results such as these were routinely reviewed on a monthly basis for more than one year, and the charts and summary tables showed no significant differences in raw numbers or trends between or within laboratories. Based on these data, we concluded that bimonthly director review of these chart and tabular results was sufficient for our quality assurance purposes and to fulfill the CAP requirement. However, we also felt the need to have a more immediate short-term method to detect any significant departures from expectations should any unexpected trends occur. To meet this need, we developed an automated query tool that reviews the molecular genetics database on a weekly basis and issues an e-mail alert to the laboratory directors if pre-established control limits on the percent heterozygous carriers or homozygous mutants are exceeded. This continuous monitoring process may prevent reporting of incorrect results by identifying failing reagents before standard methods of detecting assay failures.
View larger version (25K):
[in this window]
[in a new window]
|
Figure 3. Comparison of graphic output of the charts generated for the results of factor V Leiden assay performed at two different laboratories, Chantilly, VA (top) and San Juan Capistrano, CA (bottom).
|
|
We piloted the use of e-mail alerts with some of our high-volume assays such as factor V Leiden, factor II prothrombin thrombophilia, and methylenetetrahydrofolate reductase deficiency. Our review over the last 6 months of these e-mail alerts shows that in the case for FV test results they were always within or just on the border of the "control limits" designed based on historical data. There were 16 occurrences in the FV mutation test results where the e-mail alert was sent to laboratory directors, and in all cases, these alerts were for instances where the calculated genotype frequency was at or just outside one of the control limits (Table 1)
. For example, in October and November of 2005, there were three occurrences. In two cases, the wt/wt frequency (89.03 and 89.04) just exceeded the upper control limit of 88.37%, and the FV/wt heterozygous frequency (10.86 and 10.36) just exceeded the lower control limit of 11.21%. In the third case, the observed FV/FV homozygote frequency was 0.78%, just over the upper control limit of 0.64%. After review of monthly trends, we determined that none of these observed departures was considered significant. However, these e-mail alerts could have been an early warning to a failing assay. These alert criteria may be configured to the needs of the recipient to be more or less stringent as desired.
View this table:
[in this window]
[in a new window]
|
Table 1. Number and Type of Out-of-Range Occurrences for the Factor V Leiden Genotype Frequency during the 6-Month Time Period August 2005 to February 2006
|
|
|
Discussion
|
|---|
Accuracy of results and continuous monitoring of clinical diagnostic results is of particular importance in high-throughput laboratories that rely on automated systems of sample tracking and sample analysis. Periodic review of molecular genetic test results with manual methods is difficult to perform in a timely and routine manner in a high-throughput laboratory setting. However, our automated web-based query tool has allowed us to generate charts of molecular genetic results that can be routinely reviewed on a monthly basis for all high-volume assays. Our experience with monthly review over the last 2 years has shown that our molecular genetic results are stable and that a formal bimonthly review is sufficient. In addition, we have supplemented our routine QC chart review with a system of weekly e-mail alerts for the highest volume assays for more rapid notification of any significant departures in results. Because of the large volumes of testing performed in commercial laboratories, we were able to perform these quality assurance checks at weekly intervals.
In conclusion, we have developed a web-based query tool to enable the routine and timely compliance with the CAP checklist item that calls for periodic review of molecular genetic test statistics. Our monthly QC charts and tabular summaries are now generated in minutes compared with the hours it previously took using manual Excel-based charting applications. We can also use this tool for complex queries, data comparisons, and presentations for clinical research applications and publications. Finally, this tool could be replicated in other settings such as large hospital or academic laboratories.
|
Acknowledgments
|
|---|
We thank Robert Hadaya for writing the initial web query tool script. We also thank Steven J. Potts and Maria L. Thompson for reviewing this manuscript and making helpful suggestions.
|
Footnotes
|
|---|
Address reprint requests to Dr. Matthew J. McGinniss, Molecular Genetics Laboratory, Quest Diagnostics Nichols Institute, 33608 Ortega Highway, San Juan Capistrano, CA 92690-6130. E-mail: matthew.j.mcginniss{at}questdiagnostics.com
All authors are employees of Quest Diagnostics, Inc. and, as such, several co-authors have stock and/or stock options and patents to disclose.
Accepted for publication September 22, 2006.
|
References
|
|---|
- Dequeker E, Ramsden S, Grody WW, Stenzel TT, Barton DE: Quality control in molecular genetic testing. Nat Rev Genet 2001, 2:717-723[Medline]
- McGovern MM, Benach MO, Wallenstein S, Desnick RJ, Keenlyside R: Quality assurance in molecular genetic testing laboratories. JAMA 1999, 281:835-840[Abstract/Free Full Text]
- Braun A, Roth R, McGinniss MJ: Technology challenges in screening single gene disorders. Eur J Pediatrics 2003, 162(Suppl 1):S13-S16[CrossRef]
- Williams LO, Cole EC, Lubin IM, Iglesias NI, Jordan RL, Elliott LE: Quality assurance in human molecular genetics testing. Arch Pathol Lab Med 2003, 127:1353-1358[Medline]
- Strom CM: Mutation detection, interpretation, and applications in the clinical laboratory setting. Mutat Res 2005, 573:160-167[Medline]
- Rees DC, Cox M, Clegg JB: World distribution of factor V Leiden. Lancet 1995, 346:1133-1134[CrossRef][Medline]
Top
Abstract
Introduction
