Designing a Study You Can Defend
Not every research limitation is a design flaw. Some limitations are defensible consequences of answering a particular question under ethical, practical, or methodological constraints, while others undermine the very inference the study claims to make.
Read Guide →
Designing a Study You Can Defend
A difference between research groups or a change over time does not necessarily establish an intervention's effect. Learn how to identify threats to internal validity, assess competing explanations, and protect the credibility of your study's main comparison.
Read Guide →
Designing a Study You Can Defend
When research groups differ before an intervention begins, their subsequent outcomes may not provide a fair comparison. Understand how selection differences threaten internal validity, when they create confounding, and which design strategies can make causal conclusions more credible.
Read Guide →
Designing a Study You Can Defend
An improvement after an intervention does not necessarily mean the intervention caused it. Understand how external events create history effects, when they threaten before-and-after comparisons, and which research designs can help distinguish intervention effects from concurrent changes.
Read Guide →
Designing a Study You Can Defend
Participants may improve, deteriorate, or adapt naturally during a study, even without an intervention. Learn when maturation threatens causal conclusions, how it differs from other time-related threats, and which design strategies help researchers distinguish natural change from intervention effects.
Read Guide →
Designing a Study You Can Defend
Participants selected because of unusually high or low scores may appear to improve or deteriorate even without an effective intervention. Learn why regression to the mean occurs, how it can distort causal conclusions, and how researchers can design more credible comparisons.
Read Guide →
Designing a Study You Can Defend
A difference between research groups or measurement occasions may reflect changes in how outcomes were assessed rather than genuine intervention effects. Understand instrumentation threats, recognize when measurements are not comparable, and learn how to protect the validity of your research conclusions.
Read Guide →
Designing a Study You Can Defend
When participants in one research group receive elements of another group's intervention, the intended comparison may change. Learn how contamination occurs, why it can distort treatment effects, and how to prevent, measure, and report intervention spillover.
Read Guide →
Designing a Study You Can Defend
Participants do not always receive or follow interventions as assigned. Understand how nonadherence changes the treatment comparison, why excluding noncompliant participants can introduce bias, and how to design and analyze studies that preserve credible causal conclusions.
Read Guide →
Designing a Study You Can Defend
Missing observations can distort research findings when the data that remain systematically differ from the information that was not collected. Understand how missing-data mechanisms affect internal validity, which analytical approaches may help, and why preventing missingness is often more valuable than correcting it afterward.
Read Guide →
Designing a Study You Can Defend
Research findings do not automatically apply beyond the participants studied. Learn how sampling, setting, eligibility, treatment conditions, and methodology shape generalizability, external validity, transportability, and qualitative transferability.
Read Guide →
Designing a Study You Can Defend
Internal validity concerns whether a study supports a credible inference within the conditions studied, while external validity concerns whether that inference extends to other populations, settings, or circumstances. Strong research considers both, but their importance depends on the claim being made.
Read Guide →
Designing a Study You Can Defend
Some design choices that strengthen control can narrow the populations and conditions represented by a study, but internal validity and generalizability are not inherently opposing goals. The real question is which design choices improve one inference while restricting another.
Read Guide →
Designing a Study You Can Defend
Strict eligibility criteria may improve intervention consistency and participant safety while excluding people who would use the intervention in practice. Learn when restrictions limit generalizability, which exclusions are justified, and how to design studies that remain methodologically credible and relevant to real-world populations.
Read Guide →
Designing a Study You Can Defend
A tightly controlled experiment may identify an intervention's causal effect convincingly while providing limited evidence about its performance in ordinary settings. Understand why internal and external validity can diverge, when experimental control becomes restrictive, and how to design research that supports credible and applicable conclusions.
Read Guide →
Designing a Study You Can Defend
Conducting research at multiple institutions may strengthen evidence about how findings apply across settings, but the number of sites alone does not guarantee generalizability. Learn how site selection, contextual variation, and treatment-effect heterogeneity determine what multisite research can establish.
Read Guide →
Designing a Study You Can Defend
There is no universal minimum number of research sites needed for generalizability. Learn how to determine site requirements based on contextual diversity, research objectives, statistical precision, and the evidence needed to understand variation across settings.
Read Guide →
Designing a Study You Can Defend
Convenient research sites may make a study feasible, but they may exclude settings essential to understanding how findings apply elsewhere. Learn when contextual diversity matters, how to select informative sites, and why purposive diversity and statistical representativeness serve different research purposes.
Read Guide →
Designing a Study You Can Defend
Readers cannot judge whether research findings apply to their setting without sufficient information about participants, context, intervention delivery, and outcomes. Learn how to build applicability into study design and reporting rather than treating it as an afterthought.
Read Guide →
Designing a Study You Can Defend
Your research question may name one construct while your data capture something narrower or different. Learn how measurement decisions can quietly change the empirical question your study actually answers.
Read Guide →
Designing a Study You Can Defend
Reliability asks whether measurement is sufficiently consistent; validity asks whether the evidence supports the interpretation you want to make from it. A measure may be highly reliable without measuring the intended construct well.
Read Guide →
Designing a Study You Can Defend
Construct, content, criterion, and face validity describe different questions researchers may ask about measurement, but they should not be treated as four independent certificates of validity. Modern validity frameworks emphasize the evidence supporting a particular interpretation and use of scores.
Read Guide →
Designing a Study You Can Defend
A measure can produce highly consistent results and still fail to support the interpretation a researcher wants to make from them. Reliability is important, but consistency alone cannot establish validity.
Read Guide →
Designing a Study You Can Defend
Using an instrument with strong validity evidence can strengthen one part of your methodology, but it does not validate the study as a whole. Sampling, design, implementation, analysis, and interpretation still determine whether your conclusions are defensible.
Read Guide →
Designing a Study You Can Defend
A study can implement an intervention correctly and collect reliable measurements while failing to investigate the construct it claims to address. Learn how to examine the relationship between theoretical concepts, intervention components, operational definitions, and evidence supporting construct interpretation.
Read Guide →