Should You Include a Study With the Right Population but Wrong Outcome?

What if the participants are exactly right but the outcome is not?

You find a study involving precisely the population you need. The intervention, setting, and perhaps even the comparator fit. Then you reach the outcomes and discover that the researchers measured something different from what your review is interested in.

It seems intuitive to exclude the paper. After all, how can a study help answer your question if it does not report the outcome you need?

In some reviews, that conclusion is appropriate. In others, particularly systematic reviews of interventions, automatically excluding studies according to the outcomes they report can create a methodological problem of its own. The key distinction is whether the outcome is legitimately part of study eligibility or merely determines which synthesis the study can contribute to.

The wrong outcome does not always make the study ineligible

Outcome mismatch is more complicated than it first appears

Start by asking what "wrong outcome" actually means

Several quite different situations can look like an outcome mismatch.

The study may never have measured your outcome. It may have measured the underlying construct using a different instrument. It may have measured your outcome but not reported the results. It may report a related surrogate outcome rather than the outcome your review prioritizes. Or it may measure the correct outcome at a time point outside your planned synthesis.

Those situations should not automatically receive the same decision.

Before excluding anything, therefore, establish which of these problems you actually have.

In many intervention reviews, outcomes do not determine study eligibility

This is a point researchers sometimes find counterintuitive. Cochrane states that the population, intervention, comparator, and eligible study designs generally form the basis of prespecified study eligibility. Outcomes define the scope of the question, but it is rare to use them as criteria for including studies.

The reason is methodological rather than semantic. If studies are eligible only when they report a desired outcome, inclusion can become dependent on what study authors chose to publish.

Suppose ten eligible trials evaluate the same intervention in the correct population. Eight report your outcome, while two measured it but omitted the results. If you exclude the latter two because the outcome is absent from their reports, the evidence base may become shaped by selective reporting rather than by the underlying studies that addressed the intervention question.

Reported outcomes and measured outcomes are different

This distinction deserves special attention. A published article may omit an outcome even though investigators measured it. Conversely, an abstract may not mention an outcome that appears later in the full report or supplementary material.

Cochrane specifically advises that studies should not be excluded merely because they fail to report outcome data they may have measured or because they provide "no usable data." This principle is intended to reduce bias arising from selective reporting.

Empirical methodological research provides a reason for that concern. A Cochrane methodology review found discrepancies between outcomes specified in systematic-review protocols and those subsequently reported, and the broader literature on selective reporting shows why outcome availability should not casually determine what evidence is considered.

Sometimes the outcome legitimately is an eligibility criterion

The rule is not "outcomes never matter for inclusion." Cochrane explicitly recognizes exceptions.

Outcome measurement may legitimately determine eligibility when the purpose of the intervention differs according to the outcome being targeted or when a review specifically investigates whether an intervention prevents a particular outcome.

The same principle extends conceptually to other kinds of review questions. If the phenomenon under investigation is intrinsically defined by a particular outcome or endpoint, including studies that never assess it may not answer the intended question.

The methodological requirement is to make that decision when defining the review, not after discovering which outcomes individual studies happen to report.

Do not confuse a different instrument with a different outcome

Two studies can measure the same outcome domain in different ways. One study might measure depressive symptoms using one validated scale and another using a different scale. In educational research, academic achievement might be operationalized using a standardized test, course examination, or another prespecified measure.

Whether those measures belong to the same outcome domain is a conceptual and methodological decision. Cochrane recommends planning at the protocol stage how variants of outcomes will be grouped for synthesis.

Therefore, do not label an outcome "wrong" merely because the authors used a different instrument. Return to your operational definition.

A related outcome is not automatically an equivalent outcome

The reverse mistake also occurs. Researchers sometimes stretch an outcome definition because a study otherwise looks perfect.

Engagement is not automatically learning. Satisfaction is not necessarily acceptability. Knowledge is not behavior. Intention is not actual adoption. A surrogate endpoint is not interchangeable with the endpoint it is intended to predict merely because the two are related.

If your eligibility criteria specify a construct narrowly, preserve that distinction. A study with the right population does not gain the right outcome merely because its measure is adjacent to what you wanted.

The study may be eligible even when it cannot enter a particular synthesis

Cochrane distinguishes the overall review PICO from the PICO for each individual synthesis. An included study can fall within the review's scope while lacking the outcome, time point, comparison, or data needed for a particular synthesis.

This distinction prevents a common conceptual error: treating "cannot contribute an effect estimate to this meta-analysis" as equivalent to "should not be included in the review."

A study may still need to be described among included studies, assessed for bias where appropriate, or considered when evaluating missing outcome information even if it contributes no numerical estimate to a particular meta-analysis.

Your protocol should settle the rule before the results tempt you

Outcome rules should be specified before screening wherever possible. Cochrane requires review objectives, including PICO components, to be defined in advance and recommends planning how outcomes will be grouped for analysis.

This matters because outcome-based decisions can become vulnerable to knowledge of the findings. Choosing which outcomes to include or report after seeing their statistical significance, magnitude, or direction can introduce selective inclusion or reporting bias.

If you discover during screening that your outcome definition is genuinely inadequate, you may need to reconsider it. But that is a protocol-level methodological decision, not permission to admit appealing studies one at a time.

When the right population and intervention come with an unexpected outcome

Common mistakes when the outcome does not seem to fit

Diagnose the outcome mismatch before deciding eligibility

When the population is right but the outcome looks wrong, do not make the decision from the outcome label alone. Determine what was measured, what was reported, and what your protocol actually requires.

This is why a study can sometimes answer only part of the broader review question and remain eligible. The crucial issue is whether the missing component defines study eligibility or merely limits the contribution the study can make.

If your real problem is the reverse, where the outcome is correct but the participants are not, the decision involves a different set of considerations because population characteristics commonly play a more direct role in eligibility. That distinction is especially important when evaluating evidence with the right outcome but a different population.

Before excluding a study for having the wrong outcome, check this

Questions about studies with the right population but wrong outcome

The population can be right even when the outcome needs closer examination

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.