Have You Distinguished Direct Evidence From Indirect Evidence?

Does the Evidence Actually Answer the Question You Are Asking?

You find a rigorous study. Its methods are careful, its analysis is appropriate, and its findings are relevant to your topic. Can you treat it as direct evidence for your conclusion?

Not necessarily.

The study may examine a different population, a related but different intervention, an imperfect comparison, or a proxy for the outcome you actually care about. In other cases, the relationship you want to establish may have been inferred through several pieces of evidence rather than examined directly.

The evidence may still be useful. But the distance between what was actually studied and what you want to conclude should remain visible in your synthesis.

Directness Depends on How Closely the Evidence Matches Your Claim

Directness Is About the Distance Between Evidence and Claim

Start with the question you actually want to answer

You cannot determine whether evidence is direct in the abstract. Directness is relative to a specific question or claim.

In evidence synthesis for intervention questions, GRADE commonly examines how closely the available studies align with the target population, intervention, comparator, and outcome. A meaningful mismatch can introduce indirectness. The broader principle extends beyond intervention research: compare what the study actually investigated with what your synthesis is trying to conclude.

Population differences can make evidence indirect

Suppose your conclusion concerns the effectiveness of an instructional intervention for primary-school pupils, but the available experiments involve university students. Those studies may tell you something useful about the intervention. They do not directly establish the same effect in younger learners.

Not every population difference is consequential. A mismatch matters when there is a reasonable basis to think it could change the relevant effect, relationship, or interpretation. The size and importance of that mismatch should therefore be judged rather than mechanically assumed.

Population indirectness also connects to the broader question of where your synthesis conclusions may or may not apply.

The intervention or exposure may not be the one in your conclusion

Two interventions can share a label while differing substantially in duration, intensity, implementation, or components. Similarly, evidence about one form or level of an exposure may not directly support conclusions about another.

Imagine studies evaluating a semester-long, instructor-supervised AI tutoring system while your conclusion concerns occasional student use of general-purpose generative AI. Both involve AI-assisted learning, but the intervention actually tested is not identical to the practice described in the conclusion.

The broader the category used in your synthesis, the easier it becomes to hide this kind of indirectness.

The comparator can change what a finding means

Evidence that intervention A performs better than no intervention does not directly establish that A performs better than intervention B. If your research question concerns A versus B but the literature contains only A versus C and B versus C, any comparison between A and B requires additional inference.

This is one recognized form of indirect comparison. In some fields, specialized methods such as network meta-analysis are designed to estimate comparative effects using networks of direct and indirect comparisons. Such analyses have their own assumptions and should not be treated as equivalent to casually comparing results from separate studies.

A proxy outcome is not the same as the outcome you actually care about

Outcome mismatch is another common source of indirectness. Researchers may measure something that is easier or faster to observe because the final outcome is difficult, expensive, or slow to obtain.

For example, a study might demonstrate that an educational intervention increases time on task. If your conclusion is that the intervention improves academic achievement, you need evidence connecting increased time on task with improved achievement. The first finding does not directly establish the second.

Indirectness can occur through a chain of evidence

Sometimes no study directly tests the full relationship of interest. Instead, separate bodies of evidence establish different links.

Suppose one literature shows that an intervention increases a particular behavior, while another shows that the behavior is associated with a desired outcome. Combining those literatures may support a plausible inference that the intervention could affect the outcome. But unless the full pathway has been tested appropriately, the conclusion relies on a chain rather than a direct empirical link.

The inference may be reasonable, but its strength depends on the credibility of each link and on whether the assumed pathway connecting them is justified.

Direct evidence can still be weak

Directness should not be mistaken for overall evidential quality. A poorly designed study may directly examine exactly the population, intervention, comparison, and outcome of interest. It remains direct, but its methodological limitations may substantially reduce confidence in its findings.

Conversely, an exceptionally rigorous study can provide indirect evidence if it answers a meaningfully different question.

This is why you should assess how strongly the evidence supports a conclusion separately from how directly it bears on that conclusion.

Indirect evidence should usually change your language before it disappears from your synthesis

Researchers sometimes react to indirectness in one of two extremes: treating the evidence as though it were perfectly direct or excluding it entirely. Neither response is always appropriate.

If indirect evidence is informative, use it while making the inferential boundary clear. Depending on the circumstances, phrases such as “may extend to,” “provides indirect support for,” “suggests,” or “has not been tested directly” can communicate the relationship more accurately.

The appropriate qualification depends on how consequential the mismatch is. Minor differences need not be dramatized. Material differences should not be hidden.

When Evidence About Engagement Becomes a Claim About Learning

The studies have not suddenly become irrelevant. They answer a neighboring question directly and your focal question indirectly. Recognizing that distinction prevents the synthesis from making the evidence travel farther than the research itself.

Where Indirect Evidence Is Easily Mistaken for Direct Support

Compare Each Major Claim With What the Studies Actually Examined

A practical directness audit begins with the conclusion rather than the citation. Write down exactly what you are claiming, then compare it with the characteristics of the evidence supporting it.

This exercise may reveal that your original conclusion is broader than the evidence permits. In that case, you can narrow the claim, qualify it, identify the indirectness explicitly, or conclude that the available evidence still leaves the question uncertain.

Whatever you decide, preserve the evidence trail. A reader should still be able to trace the conclusion back to the studies and findings from which it arose.

How Directly Does the Evidence Support Your Claim?

Questions About Direct and Indirect Evidence

Do Not Make the Evidence Answer a Question It Did Not Study

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.