How Do You Distinguish Evidence That Something Works From Evidence About Why It Works?

Does showing that an intervention works also show why it works?

Suppose several well-designed studies indicate that an intervention improves an outcome. Researchers may then offer explanations: perhaps the intervention increases motivation, changes behavior, improves knowledge, reduces a barrier, or activates some other mechanism.

Are those explanations established by the same evidence that established the effect?

Usually not. Evidence that an intervention changes an outcome and evidence explaining the processes through which that change occurs address different questions. A credible effect estimate can exist without a well-established mechanism, while a plausible mechanism can exist without convincing evidence that the intervention improves the outcome.

A strong synthesis therefore asks two questions separately before connecting them: does the intervention change the outcome, and what evidence explains how that change may occur?

Separate evidence of effects from evidence explaining the process that produced them

Effects and explanations are different inferential targets

“Does it work?” requires an outcome contrast

Evidence about whether an intervention works asks whether outcomes differ under the intervention compared with an appropriate alternative. Depending on the research question, that alternative might be no intervention, usual practice, placebo, or another active intervention.

A randomized trial can provide particularly strong evidence for such a contrast when randomization and other aspects of the study are implemented appropriately. Non-randomized designs can also contribute causal evidence under suitable assumptions and methods.

The resulting effect estimate tells you something about the consequence of receiving one condition rather than another. It does not necessarily reveal the chain of events responsible for that difference.

“Why does it work?” opens a different set of questions

An explanation of an intervention effect may involve several layers. The intervention must first be delivered and received. It may then trigger particular responses or mechanisms. Those processes occur within a context that can facilitate, inhibit, or alter them.

Medical Research Council guidance on process evaluation distinguishes three closely connected areas: implementation, mechanisms of impact, and context. Process evaluation can therefore investigate what was delivered, how participants responded, how change may have occurred, and how surrounding conditions shaped implementation and outcomes.

An effect estimate does not identify the mechanism

Imagine that a mentoring program improves student retention. The trial may establish an effect of offering the program relative to usual support. It does not automatically establish whether retention improved because students felt a stronger sense of belonging, received better academic advice, developed study skills, gained access to institutional resources, or responded to some combination of these processes.

Researchers need additional evidence to distinguish among such explanations.

That evidence might come from process evaluations, mediation analyses, qualitative research, measurements of intermediate variables, factorial experiments, component studies, or other designs chosen for the particular mechanistic question. Different methods illuminate different parts of the explanation.

A proposed mediator is not automatically a demonstrated mechanism

Mediation analysis is often used to investigate whether an intervention's effect operates through an intermediate variable. But observing that the intervention changes a proposed mediator and that the mediator is associated with the outcome does not, by itself, establish the complete causal pathway.

Causal interpretation of mediation requires additional assumptions. Confounding can occur in relationships involving the mediator and outcome, and some complications arise when variables affected by the intervention also influence both the mediator and outcome.

Mechanism claims should therefore be calibrated to the design and assumptions supporting them. “The findings are consistent with mediation through X” can be appropriate when “X explains the effect” would overstate the evidence.

Implementation failure and theory failure are different explanations

Suppose an intervention does not improve the primary outcome. One explanation is that the intervention's underlying theory is wrong: even when implemented properly, the proposed mechanism does not produce the expected change.

Another possibility is implementation failure. Perhaps the intervention was rarely delivered, participants received little of it, staff adapted key components, or the intended population could not access it.

Process evaluation can help distinguish these possibilities by examining implementation alongside mechanisms and context. MRC guidance specifically emphasizes that understanding how an intervention was implemented can be important for interpreting outcomes.

Context can be part of the explanation

Why an intervention works in one setting but not another may depend on more than the intervention itself. Staffing, institutional structures, resources, participant characteristics, social relationships, implementation capacity, and other contextual factors can shape whether proposed mechanisms are activated.

Updated MRC guidance emphasizes interactions between interventions and their contexts and treats mechanisms of change as causal links connecting intervention components with outcomes.

This means that “why it works” sometimes includes “under what conditions does this mechanism operate?” A mechanism need not function identically across every setting.

Mechanistic plausibility is useful, but plausibility is not outcome evidence

An intervention may have an elegant theoretical rationale. Laboratory research, qualitative accounts, biological knowledge, behavioral theory, or previous studies may make the proposed pathway highly plausible.

That still does not demonstrate that the intervention produces the intended outcome in the population of interest.

This is why mechanistic plausibility should not be treated as proof of an observed effect. Understanding how an intervention could work and establishing whether it actually does work are complementary but distinct tasks.

Conversely, an effect can be credible before the mechanism is fully understood

Researchers sometimes hesitate to accept an intervention effect because the mechanism remains uncertain. Mechanistic uncertainty can matter for interpretation, transportability, refinement, and implementation, but it does not automatically invalidate a well-estimated outcome contrast.

Evidence of effect and evidence of mechanism should therefore be judged on their respective merits. Understanding processes is important, but MRC guidance has long cautioned that understanding processes does not replace evaluation of outcomes.

Do not let “why” become a post hoc story

When an intervention produces an unexpected result, it is easy to inspect secondary variables and construct a plausible explanation afterward. Such hypotheses can be valuable, but post hoc explanations should be distinguished from mechanisms specified and tested in advance.

Mechanistic synthesis should therefore consider whether the proposed pathway was prespecified, how it was measured, whether temporal ordering is appropriate, whether alternative mechanisms were considered, and whether the study design can distinguish among competing explanations.

A compelling narrative is not the same thing as a tested causal pathway. Research has enough plot twists without assigning motives to the variables.

What would separate evidence of effect from evidence of mechanism?

Notice that the effect and the explanation can have different levels of certainty. You may be relatively confident that the intervention changes the outcome while remaining uncertain about precisely how it does so.

Where evidence about effects and mechanisms becomes confused

Build separate conclusions for effects and explanations, then connect them carefully

When reviewing intervention evidence, extract outcome effects separately from evidence concerning mechanisms, implementation, and context. This prevents explanatory evidence from quietly inflating confidence in an effect estimate, or a strong effect estimate from making a speculative mechanism appear proven.

This distinction is also useful when considering whether an effect will travel beyond the conditions in which it was first demonstrated. Understanding mechanisms and context can help explain why results vary, but evidence from controlled conditions and evidence from routine implementation still raise the separate question of efficacy versus real-world effectiveness.

The goal is not to choose between effect evaluation and explanation. A mature evidence base often needs both. The discipline lies in knowing which question each piece of evidence actually answers.

Before concluding what works and why, check:

Common questions about what works and why it works

Knowing that something works is not the same as knowing why

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.