The independent Evaluation of Genomic Applications in Practice and Prevention (EGAPP™) Working Group reviews the scientific evidence for selected genetic tests and develops recommendation statements about the appropriate use of these tests. The following tables provide summarized information about the methods that the EGAPP™ Working Group uses to conduct this work. These methods have also been published in January 2009 issueExternal of Genetics in Medicine.
Table 1: Test Applications
This table describes categories of genetic test applications and some characteristics of how clinical validity and clinical utility are assessed for each.
Application
Clinical Validity
Clinical Utility
Diagnosis (symptomatic patient)
Association of marker with disorder
Improved clinical outcomes – health outcomes based on diagnosis and subsequent intervention or treatment
Availability of information useful for personal or clinical decision-making
End of diagnostic odyssey
Disease Screening (asymptomatic patient)
Association of marker with disorder
Improved health outcome based on early intervention for screen positive individuals to identify a disorder for which there is intervention or treatment, or provision of information useful for personal or clinical decision making
Risk assessment/susceptibility
Association of marker with future disorder (consider possible effect of penetrance)
Improved health outcomes based on prevention or early detection strategies
Prognostic
Association of marker with natural history benchmarks of the disorder
Improved health outcomes, or outcomes of value to patients, based on changes in patient management
Pharmacogenomic Predicting treatment response or adverse events
Association of marker with a phenotype/metabolic state that relates to drug efficacy or adverse drug reactions
Improved health outcomes or adherence based on drug selection or dosage
Because of the newness of the field of genetic testing, direct evidence to answer an overarching question about the effectiveness and value of testing is rarely available. Therefore, the EGAPP™ Working Group evaluated evidence in three key areas to construct a chain of evidence to address the overarching question.
Analytic Validity
(Technical Performance)
Clinical Validity
(Strength of Clinical Correlation)
Clinical Utility
(Impact on Health Outcomes)
Tests ability to accurately and reliably measure analyte or genotype of interest.
Sensitivity
Specificity
Assay robustness
Quality control
Test’s ability to accurately and reliably identify or predict the disorder of interest.
Sensitivity
Specificity
Positive Predictive Value
Negative Predictive Value
Likelihood that using the test to guide management will significantly improve health-related outcomes.
This table describes how the quality of evidence was graded in terms of its adequacy to address the key questions of each of the evidence components: analytic validity, clinical validity, and clinical utility.
Quality of Evidence
Adequacy of Information to Answer Key Questions
Analytic Validity
Clinical Validity
Clinical Utility
Convincing
Studies that provide confident estimates of analytic Sensitivity and specificity using intendedsample types from representative populations
Two or more Level 1 or 2 studies that are generalizable, have a sufficient number and distribution of challenges, and report consistent results
One Level 1 or 2 study that is generalizable and has an appropriate number and distribution of challenges
Well-designed and conducted studies in representative population(s) that measure the strength of association between a genotype or biomarkerand a specific and well-defined disease or phenotype
Systematic review/meta-analysis of Level 1 studies with homogeneity
Validated Clinical Decision Rule
High quality Level 1 cohort study
Well-designed and conducted studies in representative population(s)that assess specified health outcomes
Systematic review/meta-analysis of randomized controlled trials showing consistency in results
At least one large randomized controlled trial (Level 2)
Adequate
Two or more Level 1 or 2 studies that
Lack the appropriate number and/or distribution of challenges
Are consistent, but not generalizable.
Modeling showing that lower quality (Level 3, 4) studies may be acceptable for a specific well-defined clinical scenario
Systematic review of lower quality studies
Review of Level 1 or 2 studies with heterogeneity
Case-control study with good reference standards
Unvalidated Clinical Decision Rule (Level 2)
Systematic review with heterogeneity
One or more controlled trials with out randomization (Level 3)
Systematic review of Level 3 cohort studies with consistent results
Inadequate
Combinations of higher quality studies that show important unexplained inconsistencies
One or more lower quality studies (Level 3 or 4)
Expert opinion
Single case-control study
Nonconsecutive cases
Lacks consisitently applied reference standards
Single Level 2 or 3 cohort/case-control study
Reference standard defined by the test or not used systematically
Study not blinded
Level 4 data
Systematic review of Level 3 quality studies or studies with heterogeneity
Single Level 3 cohort or case-control study
Level 4 data
Note: The “levels” used in ranking studies are described in Table 4. Top of Page
Table 4: Ranking of Data Sources/Study Designs for Components of Evaluation
This table illustrates how data sources and study designs were ranked in order to assess the quality of evidence for each component of the evidence evaluation.
Levela
Analytic Validity
Clinical Validity
Clinical Utility
1
Collaborative study using a large panel of well characterized samples
Summary data from well-designed external proficiency testing schemes or interlaboratory comparison programs
Well designed longitudinal cohort studies
Validated clinical decision ruleb
Meta-analysis of randomized controlled trials (RCT)
2
Other data from proficiency testing chemes
Well designed peer-reviewed studies (e.g., method comparisons, validation studies)
Expert panel reviewed FDA summaries
Well designed case-control studies
A single randomized controlled trial
3
Less well designed peer-reviewed studies
Lower quality case-control and cross-sectional studies
Unvalidated clinical decision ruleb
Controlled trial without randomization
Cohort or case-control study
4
Unpublished and/or non-peer reviewed research, clinical laboratory, or manufacturer data
Studies on performance of the same basic methodology, but used to test for a different target
Case Series
Unpublished and/or non-peer reviewed research, clinical laboratory or manufacturer data
Consensus guidelines
Expert opinion
Case Series
Unpublished and/or non-peer reviewed studies
Clinical laboratory or manufacturer data
Consensus guidelines
Expert opinion
aHighest level is 1.
bA clinical decision rule is an algorithm leading to result categorization. It can also be defined as a clinical tool that quantifies the contributions made by different variables (e.g., test result, family history) in order to determine classification/interpretation of a test result (e.g., for diagnosis, prognosis, therapeutic response) in situations requiring complex decision-making.
This table demonstrates the possible recommendations derived from the evaluation of the evidence components, the overall level of certainty of net health benefits, and contextual factors.
Level of Certainty
Recommendation
High or Moderate
Recommend for . . . . . . if the magnitude of net benefit is substantial, moderate, or small, unless additional considerations warrant caution.Recommend against . . . . . . if the magnitude of net benefit is zero or there are net harms.
Low
Insufficient evidence . . . . . . if the evidence for clinical utility or clinical validity is insufficient in quantity or quality to support conclusions or make a recommendation.
Disclaimer:
Our office does not offer medical advice to individuals. If you have specific concerns about your health, please discuss them with your doctor.
Links with this icon indicate that you are leaving the CDC website.
The Centers for Disease Control and Prevention (CDC) cannot attest to the accuracy of a non-federal website.
Linking to a non-federal website does not constitute an endorsement by CDC or any of its employees of the sponsors or the information and products presented on the website.
You will be subject to the destination website's privacy policy when you follow the link.
CDC is not responsible for Section 508 compliance (accessibility) on other federal or private website.
For more information on CDC's web notification policies, see Website Disclaimers.