01 · The Question
What If the People We Know Least About Are Also the Hardest to Include?
Research often becomes easier when participants are easy to recruit, easy to reach, able to comply with demanding protocols, available for repeated follow-up, and uncomplicated by conditions that make interpretation difficult.
That convenience can have a scientific cost.
Older adults with multiple conditions may be excluded from trials. People with disabilities may encounter inaccessible recruitment or study procedures. Participants who cannot attend repeated in-person assessments may disappear from longitudinal research. People with limited access to technology may be absent from studies of digital interventions precisely because participation itself requires reliable technology.
If the same populations repeatedly disappear from the evidence base, researchers may become increasingly confident about what happens among convenient participants while remaining uncertain about people who may encounter the intervention, policy, technology, or phenomenon under substantially different conditions.
The gap is therefore more specific than “this population is underrepresented.” The important question is what we still cannot know because the people needed to answer it are systematically difficult to include.
03 · What You Need to Know
Look Beyond Who Is Missing and Ask Why They Keep Disappearing
Systematic exclusion is different from ordinary underrepresentation
A population can be underrepresented for many reasons. It may be uncommon in the target population, difficult to recruit in a particular study, or simply outside the scientific scope of the research question.
Systematic exclusion is more concerning when similar barriers operate repeatedly across studies.
The distinction matters because the solution differs. If one study happens to recruit few participants from a relevant group, another study may improve representation. If the entire research process repeatedly filters the same people out, merely repeating the conventional design may reproduce the same evidence gap.
Underrepresentation
A relevant population appears less often in the available evidence than would be desirable for the intended inference.
Systematic exclusion
Recurring eligibility criteria, recruitment practices, participation requirements, accessibility barriers, or other research processes repeatedly prevent a relevant population from entering or remaining in studies.
Exclusion can happen before recruitment even begins
The most obvious exclusions appear in eligibility criteria.
Researchers may exclude participants because of age, comorbid conditions, disability, concurrent treatments, language requirements, cognitive or communication needs, pregnancy, medication use, unstable housing, lack of technology, or other characteristics.
Some restrictions may be essential. A safety concern may make participation inappropriate. A tightly specified mechanistic study may legitimately require a narrowly defined population. Certain measurements may not yet be validated for some participants.
The methodological problem arises when exclusion becomes habitual rather than justified by the specific scientific question.
Current NIH policy provides one concrete example of how a major research funder addresses this concern. For NIH-funded clinical research, women and members of racial and/or ethnic minority groups and their subgroups are expected to be included unless a clear and compelling rationale justifies exclusion. NIH states that inclusion should be appropriate to the scientific question, and its current policy requires applicants to provide a rationale for the composition of the proposed study population.
Formal eligibility is only one gate
People can technically qualify for a study and still face participation requirements that effectively exclude them.
Consider a study requiring participants to attend a research center every weekday for six weeks. The eligibility criteria might appear broad. Yet the protocol itself favors people who live nearby, have flexible schedules, can travel independently, can afford transportation, and have responsibilities compatible with repeated attendance.
A digital study requiring a recent smartphone and stable broadband creates another implicit filter. So does a recruitment strategy conducted entirely in one language.
These exclusions may never appear in the eligibility section of the paper, but they can shape who ultimately generates the evidence.
The easiest participants may not encounter the same conditions as everyone else
This is where systematic exclusion becomes an inferential problem.
Suppose a digital health intervention is tested primarily among participants with reliable internet access, current devices, high digital literacy, and enough flexibility to complete repeated online assessments. The intervention performs well.
The evidence may accurately describe effectiveness under those conditions. It does not automatically establish how well the intervention performs among people facing unstable connectivity, accessibility needs, limited digital experience, or competing demands.
The unanswered question concerns transportability of the finding, not merely demographic representation.
Excluded populations can differ in ways that affect benefits and harms
The FDA's current guidance on enhancing participation in clinical trials explicitly considers demographic and non-demographic characteristics, including comorbid conditions, disabilities, organ dysfunction, extremes of weight, and other characteristics relevant to the people likely to use an approved intervention. The guidance notes that broader participation can permit assessment of how such characteristics affect safety and effectiveness.
The principle is broader than drug trials. Relevant participant characteristics can alter intervention exposure, adherence, implementation, measurement, mechanisms, baseline risk, benefits, harms, or feasibility.
That is what makes exclusion scientifically consequential.
Exclusion can make effectiveness look easier than it will be in practice
A study may establish efficacy under controlled conditions among participants capable of adhering closely to a protocol. Real-world implementation may involve people with substantially more complicated circumstances.
Suppose a learning platform performs well among students with consistent attendance, reliable devices, strong internet access, and high digital literacy. If students experiencing unstable connectivity or limited device access are effectively unable to participate in the research, the study may underestimate implementation difficulties encountered by the actual target population.
The missing evidence is not simply “What happens to another subgroup?” It may be “Does this intervention remain feasible and effective under the constraints experienced by the people who were least able to enter the original studies?”
Systematic exclusion can leave several kinds of questions unanswered
| Who is repeatedly excluded |
Why exclusion may matter |
What may remain unanswered |
| People with common comorbid conditions |
Effects, interactions, adherence, or risks may differ |
Whether findings apply to people with more complex real-world conditions |
| Older or younger participants |
Development, physiology, behavior, exposure, or implementation may differ by age |
Whether benefits and harms change across relevant age groups |
| People with disabilities |
Access, measurement, implementation, and intervention requirements may differ |
Whether the intervention is accessible and produces comparable outcomes |
| People without reliable technology |
Participation and intervention delivery may depend on infrastructure |
Whether digital interventions work under constrained access conditions |
| People unable to attend intensive study visits |
Study burden may select unusually available participants |
Whether feasibility and adherence persist under ordinary life constraints |
| People excluded because of language requirements |
Language can affect access, measurement, communication, and implementation |
Whether findings transfer when research and intervention materials must operate across languages |
More inclusive research does not mean removing every eligibility criterion
There is an important corrective here. Inclusion should not become an automatic demand that every study enroll every possible participant.
Eligibility criteria protect participants, define the scientific target, control certain sources of variation, and sometimes make a study feasible at all.
The relevant question is whether each consequential exclusion is scientifically or ethically defensible relative to the research question.
Current NIH policy reflects this logic rather than requiring indiscriminate inclusion. Its expectation of inclusion allows exclusions when a clear and compelling rationale establishes that inclusion would be inappropriate for participant health or the purpose of the research.
Removing exclusions does not guarantee useful evidence about subgroups
There is another trap. A study might technically include members of a population yet recruit too few of them to support informative analysis.
Presence is not the same as evidence.
If researchers need to know whether an effect differs across groups, the study must contain sufficient information and an appropriate analysis to investigate that question. Simply reporting that several members of a population were enrolled does not resolve uncertainty about outcomes for that population.
NIH's current policy illustrates this distinction in NIH-defined Phase 3 clinical trials by requiring consideration of whether intervention effects differ by sex and race and/or ethnicity, rather than treating enrollment alone as the endpoint of inclusion.
Systematic exclusion can be mistaken for a generic population gap
There is considerable overlap with research in which study populations are too narrow to support the intended generalization.
The distinctive issue here is the process generating the narrow evidence base.
If a group is absent because it is irrelevant to the research question, there may be no problem. If the same relevant group disappears repeatedly because conventional protocols make participation difficult, the gap is structural rather than accidental.
The strongest question identifies what exclusion prevents us from knowing
A weak formulation says:
“People with disabilities are underrepresented in research on educational technology.”
A stronger formulation asks:
“It remains uncertain whether the technology produces comparable learning outcomes for students using assistive technologies because accessibility requirements and study procedures have repeatedly limited their participation in evaluations.”
The second statement still requires evidence that the exclusion actually occurs. But it identifies the scientific consequence of the exclusion rather than treating representation as a numerical end in itself.
06 · What This Means for You
Audit How Participants Enter, Experience, and Leave the Research
If a relevant population is repeatedly absent, investigate the mechanism of exclusion before proposing another study.
Look beyond demographic tables. Examine eligibility criteria, recruitment channels, consent procedures, language requirements, technology requirements, transportation, compensation, assessment burden, scheduling, accessibility, retention, and reasons for dropout.
A simple decision framework
If exclusion is scientifically or ethically necessary
State the restriction clearly and limit the intended generalization accordingly.
If conventional eligibility criteria exclude relevant real-world participants
Determine whether broader criteria can be used without compromising safety or the scientific question.
If participation procedures create the barrier
Redesign recruitment, access, scheduling, measurement, or follow-up rather than simply attempting to recruit harder.
If a group is included but evidence remains sparse
Determine whether the study can generate enough information to address the relevant uncertainty rather than treating nominal inclusion as sufficient.
If there is no credible reason exclusion could change the answer
Avoid assuming that demographic absence alone establishes a consequential research gap.
The objective is to connect inclusion to inference. A more inclusive study is scientifically valuable when it allows researchers to answer something important that previous evidence could not.
07 · A Quick Checklist
Check Whether the Research Process Systematically Filters People Out
Before claiming an exclusion-based research gap, check:
Which relevant populations are repeatedly absent or sparsely represented in the evidence?
Are they excluded formally by eligibility criteria or indirectly by how studies recruit and operate?
Are the exclusions scientifically, ethically, or practically justified for the specific research question?
Could characteristics of the excluded population plausibly change effectiveness, harms, feasibility, measurement, adherence, or implementation?
Do accessibility, language, technology, transportation, scheduling, cost, or study-burden requirements create hidden participation barriers?
Are relevant participants lost disproportionately during follow-up even when they initially enroll?
Would changing the protocol allow the study to generate genuinely informative evidence about the excluded population?
Can I state what important conclusion remains uncertain because of the exclusion?
