03 · What You Need to Know
When Population Differences Become a Meaningful Review Gap
Begin with the mechanism, not the population label
Suppose existing reviews concern “adults,” while you want to review “university students.” Those categories differ, but that fact alone tells you little about whether the evidence should differ.
Ask what characteristic of university students could plausibly affect the question. Age distribution? Educational environment? Digital access? Baseline skills? Exposure intensity? Institutional structures? A different mechanism of implementation?
The rationale becomes stronger when you can connect the population distinction to a plausible difference in effects, experiences, risks, outcomes, implementation, or applicability.
Different population label
The proposed review uses a participant category that differs from previous reviews.
Meaningful population distinction
The population differs in characteristics that could plausibly affect the research question or interpretation of its evidence.
The second provides a scientific rationale. The first provides only a boundary.
Check whether your population is already inside the existing review
A review can have a broad population while containing substantial evidence about your narrower group.
Before proposing another review, inspect the characteristics of the included studies and any subgroup analyses. Your population may already be represented sufficiently for the existing synthesis to provide useful evidence.
For example, a review of working adults might contain several studies conducted specifically among teachers. If your proposed review concerns teachers, you need to determine whether those studies have already been analyzed separately, whether their data can be distinguished, and whether the broader review's conclusions are reasonably applicable.
This is part of the larger task of determining whether existing reviews already answer the question you intend to ask.
Population differences can affect applicability
Evidence generated in one population does not automatically apply identically to another.
Cochrane's guidance on equity and specific populations emphasizes examining whether findings are likely to apply to populations of interest and why. Population characteristics can influence baseline conditions, access to interventions, feasibility, values and preferences, outcomes, and the effects or absolute benefits associated with an intervention.
This is closely related to indirectness: evidence may address the intervention and outcome of interest but provide only indirect evidence for the target population because the participants studied differ in consequential ways.
The existence of indirectness does not automatically require another systematic review. It tells you to evaluate how important the population difference is and whether population-specific evidence is available to resolve it.
Age can matter, but age categories need a rationale
Age is a common reason for separating populations, and sometimes a strong one.
Cochrane notes that differences between children and adults, and among children and young people of different ages, can affect the nature or course of a condition, delivery of an intervention, effectiveness or safety, and important outcomes.
Similar reasoning may apply outside healthcare. A learning intervention could function differently among primary-school pupils, adolescents, university students, and working adults because the learning environment, developmental stage, autonomy, prior knowledge, or mode of implementation differs.
But “nobody has reviewed ages 18 to 24 separately” is not itself sufficient. Explain why separating that age group could produce a more useful answer.
Professional groups can differ in consequential ways
A review involving healthcare professionals may not necessarily answer a question specifically about nurses, physicians, pharmacists, or allied health professionals.
The distinction becomes relevant when professional roles affect exposure, workflow, training, decision authority, implementation, or outcomes. If the mechanism and circumstances are essentially the same across professions, separating one group may add little.
The same principle applies in education. Faculty members, schoolteachers, administrators, and students may all interact with the same technology, but they do so in fundamentally different roles. Combining them could answer a broad adoption question while obscuring a role-specific question.
Different populations can produce different baseline risks
Even when relative effects are similar, the practical consequences of an intervention may differ because baseline risk or baseline conditions vary.
For example, the same relative reduction in an undesirable outcome can translate into very different absolute effects in populations with high versus low baseline risk. Cochrane's guidance on specific populations highlights differences in baseline conditions as one reason evidence may have different implications across groups.
This means population-specific synthesis can sometimes improve practical interpretation even when there is no strong reason to expect dramatically different relative effects.
Do not assume subgroup differences merely because groups look different
A plausible population distinction does not prove that effects differ.
Subgroup analyses are especially easy to overinterpret. Cochrane recommends that subgroup analyses be justified and preferably prespecified, and cautions that relationships observed across subgroups can be confounded by other study characteristics.
If one subgroup shows a statistically significant effect and another does not, that alone does not establish that the effects differ. Appropriate direct comparison of subgroup effects is needed.
Watch Out
Do not justify a population-specific review by assuming in advance that the population must respond differently. A credible rationale explains why a difference is plausible and then tests it; it does not treat the expected difference as already established.
Check whether enough population-specific evidence exists
A meaningful population question still needs evidence capable of answering it.
If only one tiny study includes the population of interest, conducting a narrowly restricted systematic review may simply produce a very short catalogue of insufficient evidence. That result can occasionally be useful, but you should know in advance whether a separate synthesis is likely to add information beyond what the broader review already reports.
Conduct a scoping search and inspect the relevant primary studies. Determine how many studies contain the target population, whether their data are separable, what outcomes they report, and whether they are sufficiently comparable for the synthesis you propose.
Sometimes re-analysis is more useful than starting again
If existing systematic reviews contain the relevant population but combine it with others, the problem may concern synthesis rather than evidence identification.
Cochrane's guidance for overviews explicitly recognizes that researchers interested in a subpopulation represented within existing reviews may wish to re-extract and re-analyse outcome data. Depending on the research question, this may be more informative than treating the entire field as though it had never been reviewed.
Your design should follow the unresolved question. Sometimes that means a new systematic review of primary studies. Sometimes it means an overview, subgroup analysis, individual participant data approach, or another method better suited to investigating population differences.
Geography is a population distinction that deserves special caution
Country and region often correlate with meaningful contextual differences, but geography should not be used as an automatic proxy for them.
Cochrane cautions, for example, that focusing solely on low- and middle-income countries is not sufficient unless the problem, outcomes, or expected functioning of the intervention differ in ways that make the distinction relevant.
The same logic applies more generally. The absence of a review conducted specifically in your country does not automatically create a country-level review gap. Identify the contextual mechanism that makes geography matter.
Population-specific reviews can also create unnecessary fragmentation
There is a cost to dividing evidence into increasingly narrow groups.
A review of university students becomes a review of undergraduate students, then first-year undergraduates, then first-year undergraduates in private universities, and eventually a population so narrow that the synthesis loses both evidence and usefulness.
Restricting the population can reduce heterogeneity when the distinction is meaningful, but it can also discard informative evidence and reduce precision. The narrowest possible review is not necessarily the most informative one.
If an adequately conducted broader review already addresses the important uncertainty, another narrowly defined synthesis may move toward unnecessary review duplication rather than useful specialization.
