Study Designs for Effectiveness and Translation Research: Identifying Trade-offs
Background
Practitioners and policymakers need credible evidence of effectiveness to justify allocating resources to complex, expensive health programs. Investigators, however, face challenges in designing sound effectiveness and translation research with relevance for “real-world” settings.
Methods
Research experts and federal and foundation funders (n=∼120) prepared for and participated in a symposium, held May 4–5, 2004, to weigh the strengths, limitations, and trade-offs between alternate designs for studying the effectiveness and translation of complex, multilevel health interventions.
Results
Symposium attendees acknowledged that research phases (hypothesis generating, efficacy, effectiveness, translation) are iterative and cyclical, not linear, since research in advanced phases may reveal unanswered questions in earlier phases. Research questions thus always need to drive the choice of study design. When randomization and experimental control are feasible, participants noted that the randomized controlled trial with individual random assignment remains the gold standard for safeguarding internal validity. Attendees highlighted trade-offs of randomized controlled trial variants, quasi-experimental designs, and natural experiments for use when randomization or experimental control or both are impossible or inadequately address external validity. Participants discussed enhancements to all designs to increase confidence in causal inference while accommodating greater external validity. Since no single study can establish causality, participants encouraged replication of studies and triangulation using different study designs. Participants also recommended participatory research approaches for building population relevance, acceptability, and usefulness.
Conclusions
Consideration of the study design choices, trade-offs, and enhancements discussed here can guide the design, funding, completion, and publication of appropriate policy- and practice-oriented effectiveness and translational research for complex, multilevel health interventions.
aGuide to Community Preventive Services, National Center for Health Marketing, Centers for Disease Control and Prevention, Atlanta, Georgia
bChild Health and Quality Improvement, Office of Extramural Research, Education, and Priority Populations, Agency for Healthcare Research and Quality, Rockville, Maryland
cAgency for Healthcare Research and Quality, Rockville, Maryland
dClinical Prevention and Translation, National Heart, Lung, and Blood Institute, National Institutes of Health, Bethesda, Maryland
eDepartment of Epidemiology and Biostatistics, School of Medicine and Comprehensive Cancer Center, University of California at San Francisco, San Francisco, California
Address correspondence and reprint requests to: Shawna L. Mercer, MSc, PhD, Director, Guide to Community Preventive Services, National Center for Health Marketing, Centers for Disease Control and Prevention, 1600 Clifton Road, NE, Mailbox E-69, Atlanta GA 30333.
ABSTRACT