Do Eligibility Criteria Make the Sample Too Narrow for the Authors’ Conclusions?

Do the Conclusions Describe People Who Could Never Have Entered the Study?

Eligibility criteria are necessary in most human research. Researchers specify who can participate according to age, diagnosis, disease severity, prior treatment, language, comorbidities, medication use, educational level, institutional membership, or other characteristics relevant to the study.

These criteria can make a study safer, more interpretable, and better aligned with its scientific objective. They can also create a highly selected sample that differs substantially from the population in which readers, clinicians, policymakers, educators, or researchers later want to apply the findings.

The critical issue is not whether the study has restrictive criteria. It is whether the population implied by the authors' conclusions is broader than the population their eligibility rules allowed them to study.

Narrow Eligibility Is a Problem When the Claim Is Broader Than the Evidence

Eligibility Criteria Define the Direct Evidential Boundary of a Study

Eligibility criteria serve legitimate scientific purposes

Inclusion and exclusion criteria determine who qualifies to enter a study. Researchers may use them to identify people with the condition or experience of interest, reduce safety risks, ensure that an intervention can be delivered, improve measurement validity, avoid incompatible treatments, or focus on a scientifically meaningful population.

Therefore, a long list of criteria is not automatically evidence of a weak design. Sometimes a tightly defined population is exactly what the research question requires.

The methodological concern begins when the paper's interpretation becomes broader than those criteria.

Write the eligibility criteria as a population description

A useful appraisal technique is to translate the methods section into an ordinary-language description of who could actually participate.

Imagine a trial that enrolls adults aged 18–45 with a recently diagnosed condition, no major comorbidities, no concurrent medication, fluency in the study language, reliable internet access, and no previous exposure to the intervention. That list defines a much more specific population than simply “adults with the condition.”

Now compare that description with the abstract and conclusion. If the paper discusses the intervention as effective for “patients with the condition” generally, the difference between those two populations deserves attention.

Restriction can improve internal validity while limiting external validity

Researchers sometimes deliberately restrict eligibility to make participants more comparable, reduce potential complications, improve adherence, or isolate a particular scientific question. Such restrictions can simplify interpretation within the study.

But greater homogeneity can come with an external-validity cost. Real-world populations often include older people, multiple diagnoses, concurrent treatments, different socioeconomic circumstances, varied levels of prior experience, and other complexities deliberately minimized in research settings.

This creates a genuine trade-off rather than a simple methodological mistake.

Ask whether excluded characteristics could modify the result

Not every eligibility restriction has equal importance. Suppose researchers exclude people with a characteristic that has no plausible relationship with the outcome, intervention response, exposure, or mechanism being studied. The external-validity consequence may be relatively small.

Now suppose the restriction removes older adults from a treatment study even though age affects pharmacokinetics, comorbidity, baseline risk, or treatment tolerance. Extending the observed treatment effect to those older adults requires substantially more caution.

The same logic applies outside medicine. An educational intervention tested only among high-achieving students may behave differently among students who need greater academic support. A workplace intervention studied only among permanent office-based employees may not transfer unchanged to temporary or remote workers.

The key question is whether the eligibility characteristic could modify the outcome, association, intervention effect, feasibility, or harms relevant to the conclusion.

Safety exclusions deserve special treatment

Some people cannot ethically be exposed to an experimental intervention when credible safety concerns exist. Excluding them may therefore be essential rather than a methodological compromise that could simply be removed.

The resulting limitation still matters. If high-risk patients were excluded from a trial, the study cannot directly establish safety in those patients. Ethical necessity narrows the evidence; it does not magically supply evidence about the excluded group.

Later studies, observational evidence, pharmacovigilance, or other research may eventually address that gap.

Routine exclusions should still have a rationale

Some eligibility criteria persist because they are conventional rather than because they are necessary for the particular study. A criterion copied from earlier protocols can narrow a sample even when its scientific justification is weak.

When an exclusion materially reduces the population represented by the study, ask what scientific, ethical, or measurement reason supports it. Current NIH inclusion policy, for example, expects inclusion across the lifespan in NIH-supported human-subjects research unless age-related exclusions have scientific or ethical justification. NIH also requires inclusion of women and members of racial and ethnic minority groups in covered clinical research unless specified justification supports exclusion. These policies apply to NIH-supported research rather than constituting universal eligibility rules, but they illustrate the expectation that consequential exclusions should be justified rather than treated as automatic.

The number of excluded people can reveal practical relevance

When possible, look for information about how many people in the intended real-world population would fail the study's eligibility criteria. A trial may have excellent internal execution yet directly represent only a relatively selected subset of people who would be candidates for the treatment in practice.

Screening flow diagrams can also be revealing. If thousands of potentially relevant people are assessed but only a small fraction qualify, ask why. A low eligibility proportion does not automatically invalidate the trial, but it makes the definition of the eligible population especially important when interpreting applicability.

Eligibility is only one stage of narrowing

Do not confuse people who are ineligible with eligible people who simply do not participate. First the criteria determine who may enter. Recruitment and consent then determine who does enter. Attrition determines who remains.

Each transition can narrow the evidence further. A study may therefore face both restrictive eligibility and additional selection processes after eligibility.

Broad conclusions require a bridge beyond the eligible sample

Sometimes researchers have legitimate evidence for extending findings beyond the study's eligibility criteria. Previous studies, biological or theoretical knowledge, transportability analyses, pragmatic evidence, or replication in complementary populations may support broader inference.

That bridge should be visible. “We found an effect in this narrowly eligible sample” and “therefore the same effect applies to everyone with the condition” are separate inferential steps.

If no such bridge exists, the more defensible approach may be to keep the conclusion within the population actually supported by the evidence.

Do not judge eligibility by representativeness alone

A study designed to answer a question about a narrowly defined population does not need to represent people outside that population. Conversely, a demographically varied sample can still be poorly suited to the target population if important eligibility restrictions remove people relevant to the intended application.

This is why representativeness matters differently across research questions. Eligibility should be evaluated against the inferential goal, not an abstract ideal of diversity.

From a Highly Selected Trial to a Broad Clinical Claim

The solution is not necessarily to criticize the investigators for excluding higher-risk patients. Those exclusions might have been ethically necessary during this stage of research. The appropriate criticism concerns the scope of the later claim.

Common Mistakes When Evaluating Eligibility Criteria

Translate Eligibility Rules Into the Population They Actually Produce

When reading a study, do not skim the eligibility criteria as procedural detail. Convert them into a description of the people represented by the evidence, then compare that description with the population named in the conclusion.

Also distinguish narrow eligibility from broader patterns of excluded or inaccessible groups. A protocol may have permissive eligibility criteria yet still produce a narrow sample because its recruitment methods reach only a small portion of the intended population.

A useful review comment therefore does more than say “the exclusion criteria limit generalizability.” Specify which criterion creates the limitation, which population is missing, why the result might differ there, and which claim consequently needs qualification.

How to Decide Whether Eligibility Criteria Are Too Restrictive

Questions About Eligibility Criteria and Generalizability

The Conclusion Should Respect the Population Defined by Eligibility

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.