01 · The Question
What if studies exist, but they do not report the outcome you need?
You may encounter an evidence base that looks substantial until you examine the outcomes. Perhaps many studies evaluate the intervention, exposure, or phenomenon of interest, but only two report your primary outcome. Others measure related outcomes, intermediate outcomes, different instruments, or outcomes assessed at different time points.
Should those outcomes be brought into the evidence base?
Sometimes yes. Different measures may genuinely represent the same outcome domain, and related outcomes may provide useful supporting evidence. But broadening outcomes can also quietly change the question. An outcome that is easier to measure, more frequently reported, or statistically convenient is not automatically a substitute for the outcome that matters to your research question.
03 · What You Need to Know
First determine what kind of outcome difference you are dealing with
“Different outcome” can describe several quite different situations. Two studies may measure the same underlying construct with different instruments. They may assess the same outcome at different time points. One may measure an intermediate outcome while another measures the final outcome that matters to participants. Or they may simply measure different things.
These distinctions determine whether broadening is sensible.
Different measures can still represent the same outcome
Suppose your outcome is depressive symptoms. One study uses one validated depression scale while another uses a different validated scale. The numerical scores are not directly interchangeable, but both may represent the same underlying outcome domain.
Similarly, studies of academic achievement might use different validated achievement measures. Patient functioning may be captured with different instruments. Quality of life can be operationalized through different established scales.
This is not necessarily the same as broadening from one outcome to another. Cochrane guidance distinguishes broad outcome domains from the more detailed specification and grouping of outcome measures for synthesis. Decisions about which outcomes matter and how related measures will be grouped should ideally be planned in advance.
The important question is whether the measures are credible operationalizations of the same construct and whether differences in measurement affect interpretation.
A related outcome may still answer a different question
Now consider a more substantial change. Your target outcome is whether students can independently perform a clinical procedure, but most studies measure only their knowledge of the procedure. Knowledge and performance are related, but they are not the same outcome.
Or perhaps the target is reduction in disease, while available studies report a biological marker. The marker may be associated with disease risk, but evidence that an intervention changes the marker does not automatically establish that it changes the clinical outcome.
Alternative measure
A different method or instrument used to assess substantially the same outcome construct.
Alternative outcome
A different endpoint that may be related to the target outcome but does not measure the same thing.
Confusing these situations can make a review appear to have more direct evidence than it actually does.
Surrogate outcomes require a stronger inferential bridge
A surrogate outcome is used as a substitute for an outcome of direct interest. Examples can include biomarkers, physiological measurements, intermediate events, or other endpoints expected to predict an outcome that matters more directly.
Surrogates can be scientifically useful. They may occur earlier, require smaller studies, or make research feasible when the final outcome is uncommon or takes years to observe. But usefulness as a research endpoint does not by itself validate a surrogate as a substitute for the outcome of interest.
GRADE identifies surrogate outcomes as an important source of outcome indirectness when evidence for the surrogate is used to inform conclusions about another outcome. The strength of the relationship between the surrogate and the target outcome therefore matters.
Watch Out
An intervention can improve an intermediate or surrogate outcome without producing the expected improvement in the final outcome. Do not translate an effect on a surrogate directly into an effect on the target outcome unless that inferential relationship is adequately supported.
Timing can make the same named outcome indirect
Outcome directness also depends on when it is measured. A study reporting anxiety one week after an intervention and another reporting anxiety twelve months later may use the same scale and outcome label, yet answer meaningfully different questions.
If your decision concerns sustained benefit, immediate post-intervention improvement is not equivalent to long-term improvement. Conversely, if the question concerns an immediate procedural outcome, a distant follow-up may capture additional influences that were not central to the original question.
Recent GRADE guidance explicitly identifies duration of follow-up as one way in which study outcomes can mismatch the outcome specified in the target question.
Outcome definitions can hide important differences
Even identically named outcomes may be defined differently across studies. “Treatment success,” “recovery,” “engagement,” “adherence,” “academic performance,” or “completion” can represent quite different thresholds and measurement rules.
Before combining them, inspect how each study operationalized the outcome. A shared label is not sufficient evidence that the measurements represent the same construct.
Do not use outcome reporting to decide which studies exist
There is another important issue that is easy to miss. In many intervention systematic reviews, studies are not supposed to be excluded simply because they fail to report a particular outcome of interest.
Cochrane guidance states that outcome reporting should rarely determine study eligibility. Excluding otherwise eligible studies because a desired result is unavailable can create bias, particularly when outcomes are selectively reported according to the results.
This means “only three studies report my primary outcome” is not necessarily the same as “only three studies are eligible for my review.” The review may contain many eligible studies while the evidence available for a particular outcome remains sparse.
Important outcomes should not be replaced merely because inconvenient outcomes are sparse
Outcome selection should reflect the research question and what matters for interpretation, not merely what investigators most commonly measured. Cochrane recommends considering outcomes that are meaningful and including both beneficial and adverse outcomes where relevant.
If the outcome most important to your question is rarely reported, replacing it with a frequently reported but less meaningful outcome can conceal a genuine evidence gap.
This is one reason sparse evidence should first be treated as an informational finding rather than a numerical problem to solve. The broader question of what to do when direct evidence remains scarce should be resolved before expanding the evidence base simply to obtain more analyzable data.
Broadening can occur without pretending all outcomes are equivalent
You may decide that several related outcomes are worth including while keeping their roles distinct. A review could identify a patient-important outcome as primary, examine a validated surrogate as supporting evidence, and report other related outcomes separately.
Similarly, different instruments assessing the same domain may sometimes be synthesized using statistical methods appropriate for differently scaled measurements, while genuinely different outcome domains remain separate.
The critical methodological move is classification. Decide which measures represent the same construct, which are distinct but related, and which require an inferential bridge to inform the target outcome.
If that bridge becomes too long or depends on assumptions that cannot be defended, you have reached the broader problem of when additional evidence becomes too indirect .
04 · A Practical Example
What if your target is clinical performance but most studies measure knowledge?
Hypothetical Example
An educational intervention with many knowledge studies but little performance evidence
Suppose you are reviewing whether a simulation-based educational intervention improves students' clinical performance. Only three eligible studies assess performance in a clinical or realistically simulated task. Another fifteen studies assess knowledge using written tests.
Define the target outcome
Your question concerns whether learners can perform, not merely whether they know factual or procedural information.
Classify the additional outcome
Knowledge is plausibly related to performance, but a knowledge test does not directly measure whether a learner can execute the clinical behavior successfully.
Identify the inferential assumption
Using knowledge outcomes to support a performance conclusion requires assuming that improvements in tested knowledge translate into improved clinical performance.
Preserve the distinction
You may include knowledge as a separate secondary outcome if it falls within the review's scope, but you do not combine knowledge scores with performance measures or describe the fifteen knowledge studies as fifteen studies demonstrating improved clinical performance.
Interpret the sparse direct evidence
Your conclusion about clinical performance remains primarily dependent on the three studies that actually measured performance. The knowledge evidence can add context without erasing the direct evidence gap.
This distinction can feel frustrating because the broader outcome produces a much larger literature. Methodologically, however, fifteen studies answering a neighboring question do not automatically outweigh three studies answering the question you actually asked.
06 · What This Means for You
Classify broader outcomes before deciding what they can tell you
When direct outcome evidence is scarce, determine the relationship between the available outcome and the target outcome before deciding how it belongs in the review.
A simple decision framework
If different instruments measure substantially the same outcome construct
They may be eligible for the same outcome domain, using synthesis methods appropriate to the measurement differences.
If the outcome is related but conceptually distinct
Consider including it as a separate outcome rather than treating it as equivalent to the target outcome.
If the available outcome is a surrogate or intermediate endpoint
Assess the evidence supporting its relationship with the target outcome and explicitly account for indirectness.
If follow-up timing differs materially from the time horizon of interest
Keep the time points distinct or explain why evidence from one time point can inform another.
If the broader outcome requires several unsupported assumptions to answer the target question
Do not use it as a substitute merely because direct evidence is sparse.
Broadening outcomes is only one possible response to scarcity. In another review, the more relevant question may be whether to broaden the population or incorporate evidence from related conditions, interventions, or contexts . These decisions should be considered independently because each changes a different part of the evidentiary relationship.
07 · A Quick Checklist
Before broadening the outcomes
Before adding broader outcomes, check:
Define the outcome that directly answers the research question before examining which outcomes are most commonly reported.
Determine whether the proposed broader outcome is another measure of the same construct or a genuinely different outcome.
Check how each study defines and operationalizes outcomes rather than relying only on outcome labels.
Examine whether differences in follow-up time change the substantive meaning of the outcome.
If using a surrogate or intermediate outcome, evaluate the basis for assuming that it predicts or represents the target outcome.
Keep conceptually distinct outcomes separate during synthesis rather than combining them merely to increase the amount of data.
Do not automatically exclude otherwise eligible studies solely because the outcome of interest is unreported.
Report outcome indirectness explicitly when broader outcomes are used to inform conclusions about the target outcome.
11 · Cite this Guide
How to Cite This Guide
This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.
Recommended (Field Guide)
APA
MLA
Chicago
Copy Citation
