Should You Include a Study That Is Relevant but Answers Only Part of the Question?

Does a study have to answer your entire research question?

Research questions are often broader than individual studies. Your review might ask about an intervention's effectiveness, acceptability, and implementation across several populations, while one paper evaluates only effectiveness. Another might examine the right intervention and population but report evidence relevant to only one planned comparison.

The paper is clearly related to your topic. It may even contain useful evidence. But is answering only part of the question enough to justify inclusion?

The answer depends less on how completely one paper mirrors the wording of your research question and more on whether it satisfies the eligibility criteria established for the review. Individual studies contribute pieces of evidence. The synthesis, rather than each paper separately, is usually what addresses the broader review question.

Partial evidence can still be eligible evidence

Your review question and each individual study do not have to be identical

A synthesis can answer more than any one study does

A literature review brings evidence from multiple studies together. Consequently, an individual study does not ordinarily need to reproduce every component, comparison, subgroup, outcome, or secondary question contained in a broad review objective.

Cochrane distinguishes among the review PICO, the PICO for a particular synthesis, and the PICO actually investigated by an included study. The review PICO defines the overall scope and eligibility of the review, while the PICO for each synthesis specifies the narrower question addressed by a particular synthesis. Individual studies may investigate only some of those planned questions or comparisons.

This distinction is useful well beyond intervention reviews. A broad review can legitimately assemble studies that make different contributions, provided those contributions fall within its prespecified scope.

Ask whether the study meets the criteria, not whether it answers everything

Suppose your review examines several interventions, populations, settings, or outcomes. Requiring every included study to address every combination could eliminate perfectly relevant evidence.

Instead, translate the research question into operational eligibility criteria. Then ask whether the study satisfies the criteria needed to contribute to at least one intended part of the evidence synthesis.

The second category should not be rescued by calling it "partial evidence." A paper can be interesting, informative, and worth citing in the background while still being ineligible for the review's main synthesis.

Broad questions often require different studies to contribute different pieces

Imagine a review asking whether a digital learning intervention improves academic performance and engagement among undergraduate students. One eligible study reports academic performance but not engagement. Another measures engagement but not academic performance.

Whether both belong depends on how the review's eligibility criteria treat outcomes. If either outcome is sufficient under the protocol, both studies may contribute to different outcome syntheses. If the protocol explicitly requires studies to measure both, the decision changes.

There is an important methodological wrinkle here. For Cochrane intervention reviews, outcomes are usually not used as study eligibility criteria. The Handbook cautions that studies generally should not be excluded simply because they fail to report outcome data, partly because doing so can create bias related to selective outcome reporting. There are, however, legitimate situations in which measurement of a particular outcome forms part of eligibility.

Missing one outcome is different from having the wrong population or intervention

This distinction becomes especially important during screening. A study of the correct population and intervention that does not report one review outcome may still be eligible under some systematic-review methodologies. By contrast, a study of an entirely ineligible population or intervention may fall outside the scope regardless of how useful its outcomes appear.

Cochrane guidance explicitly notes that population, intervention, comparator, and study-design characteristics commonly form the basis of eligibility criteria, while outcomes are more rarely used to determine whether a study enters the review.

So "it answers only part of my question" needs to be unpacked. Which part is missing? Whether that missing component matters for eligibility should already be determined by your protocol or review design.

One study may contribute to one synthesis but not another

Inclusion in the review does not mean a study must appear in every analysis.

A study might contribute to the synthesis of academic achievement but not engagement. Another may contribute to short-term outcomes but not long-term outcomes. A third may provide evidence for one intervention comparison but not another.

Cochrane's distinction between the overall review PICO and the PICO for each synthesis helps formalize this. Eligibility for the review and eligibility for a particular synthesis are related decisions, but they are not necessarily identical.

Do not change the criteria because the partial evidence looks useful

Problems arise when a study fails an established criterion but seems too interesting to lose. At that point, researchers may be tempted to reinterpret the question so the study can remain.

If the paper genuinely fails a predefined eligibility criterion, its perceived importance does not automatically make it eligible. The issue then becomes whether there is a defensible reason to amend the criteria, a decision that should be documented and justified rather than made study by study. The same principle applies when an apparently important study falls outside the original criteria.

Do not confuse inclusion with evidential weight

Even when a study is eligible, its contribution may be narrow. That is acceptable. A paper answering one subquestion should be used as evidence for that subquestion, not stretched into evidence for claims it never investigated.

This is where synthesis requires discipline. Describe which aspects of the question each study actually informs, and preserve gaps where evidence is absent. Ten studies that each answer one narrow component do not magically become ten studies answering every component. Academic arithmetic remains stubborn on this point.

How one study can legitimately answer only part of a broader question

Common mistakes with studies that provide partial evidence

Judge partial studies against the structure of your review

When a paper answers only part of your question, resist both extremes. Do not exclude it merely because it is incomplete, and do not include it merely because it seems related. Return to the criteria.

This approach also helps with borderline studies that otherwise invite arbitrary judgment. The more precisely your criteria define what constitutes a legitimate partial contribution, the less you need to improvise during screening.

Before including a study that answers only part of the question, check this

Questions about studies that answer only part of a review question

A study can contribute one piece without answering everything

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.