Could Different Implementation of the Same Intervention Explain Conflicting Findings?

Did Participants Actually Receive the Same Intervention?

Two papers may say they evaluated the same program, curriculum, therapy, policy, or implementation strategy. That does not guarantee that participants experienced the same intervention.

One study might deliver every planned session through trained providers with high participation. Another might omit components, shorten sessions, substitute materials, experience staff turnover, or reach only a fraction of the intended participants. Implementation research distinguishes several aspects of this process, including adherence, dose or exposure, quality of delivery, participant responsiveness, and differentiation from other interventions.

If outcomes differ, therefore, the relevant question may not simply be whether “the intervention” works. It may be whether sufficiently comparable versions of the intervention were actually delivered.

Yes, the Same Intervention Can Be Implemented Very Differently

What “The Same Intervention” Can Hide

An intervention on paper is not necessarily the intervention participants receive

Research protocols describe intended interventions. Implementation describes what happens when those interventions meet actual organizations, providers, participants, schedules, resources, and constraints.

This distinction is central to interpreting effectiveness. A systematic review using the RE-AIM framework found substantial variation in implementation reporting across behavioral interventions and noted that incomplete reporting makes it harder to understand for whom, under what conditions, and how interventions succeed.

Fidelity has several dimensions

Implementation fidelity is often discussed as though it were a single percentage. In practice, several dimensions can matter. The implementation literature commonly distinguishes adherence to the protocol, exposure or dose, quality of delivery, participant responsiveness, and whether the intervention remains distinguishable from comparison activities.

A program could therefore achieve high adherence because every planned component was technically delivered while still being implemented poorly if sessions were rushed or participants rarely engaged. Conversely, providers might adapt some procedures while preserving the intervention's important functions.

Adherence asks whether planned components were delivered

Adherence concerns the extent to which implementation follows the intervention protocol. If a ten-session program routinely delivers only six sessions, or an intervention requiring individualized feedback omits that component, the delivered intervention may differ materially from the intended one.

Such differences can weaken the contrast between intervention and control groups and potentially change outcomes. They also make replication difficult because another researcher cannot tell whether an ineffective result reflects the intervention itself or an incomplete version of it.

Dose asks how much intervention participants actually received

Even when all intervention components are available, participants may not receive the same amount. Attendance, exposure duration, frequency, intensity, and completion can differ.

Suppose two studies evaluate the same twelve-week educational program. In one, most students attend nearly every session. In another, attendance is sporadic. Calling both studies evaluations of the same twelve-week program conceals an important difference in actual exposure.

Quality of delivery can matter even when the protocol is followed

Providers can follow the same manual with different levels of clarity, skill, responsiveness, and competence. Fidelity frameworks therefore distinguish adherence from quality of delivery.

This matters particularly for complex interventions that depend on interpersonal delivery, professional judgment, coaching, facilitation, teaching, or counseling. Checking boxes on a protocol may establish that components occurred without establishing that they were delivered effectively.

Participant engagement is part of what actually happens

An intervention can be delivered faithfully yet fail to engage its intended recipients. Participants may attend but not participate, receive materials but not use them, or have access to a digital tool without meaningfully interacting with it.

A systematic review of complex face-to-face health behavior interventions emphasized the importance of measuring both fidelity of delivery and participant engagement for understanding effectiveness. Fewer than half of the included studies measured both dimensions.

This distinction prevents an overly simple interpretation in which implementation is judged entirely from the provider side.

Adaptation is not automatically implementation failure

Real-world implementation often involves adaptation. Providers may modify examples, scheduling, language, format, or procedures to fit local needs. Some adaptations may preserve the intervention's intended function or improve its fit. Others may remove components that are necessary for the mechanism to operate.

The important question is therefore not simply “Was anything changed?” It is what changed, why, and whether the modification plausibly altered the intervention's active ingredients or intended mechanism.

In one systematic review using RE-AIM, 22% of included interventions reported adaptations to program delivery, illustrating that adaptation is not unusual in applied intervention research.

Context and implementation often interact

Implementation does not vary randomly. Staffing, resources, leadership, workload, infrastructure, incentives, and organizational routines can affect what providers are able or willing to deliver.

This means different study contexts can produce different outcomes partly because they produce different implementation. Separating the two analytically can be useful even though they are often causally connected.

Poor reporting can make implementation differences invisible

A major practical problem is that intervention reports frequently omit implementation details. A systematic review of 100 hand-hygiene intervention studies found that only eight reported all five fidelity dimensions examined by the reviewers.

Another review of complex face-to-face behavior-change interventions found substantial gaps in reporting measures of fidelity and engagement.

Implementation differences are a hypothesis, not an automatic explanation

If Study A shows an effect and Study B does not, it is tempting to rescue the intervention by claiming that Study B implemented it poorly. That reasoning is weak unless implementation evidence supports it.

The relationship between fidelity and outcomes can itself be complicated. A review of community-based interventions noted that relatively few studies had directly examined fidelity-outcome relationships and that reported associations were not uniformly positive.

Implementation evidence should therefore be analyzed rather than used as an all-purpose explanation for disappointing findings.

How One Program Can Become Two Different Interventions in Practice

Even here, implementation should not be treated as proven causal explanation merely because Study B delivered less of the program. The universities may differ in other ways. The implementation data tell you that the studies are less directly comparable than their shared intervention label initially suggests.

Common Mistakes When Interpreting Implementation Differences

How to Decide Whether Implementation Explains the Conflict

When comparing intervention studies, reconstruct the intervention as delivered rather than merely recording its name. Examine the protocol, supplementary materials, process evaluation, fidelity measures, attendance or exposure data, provider training, adaptations, and participant engagement where available.

Be particularly careful when implementation is confounded with context. If the intervention was implemented poorly only in one low-resource setting, you cannot easily determine whether implementation, broader contextual conditions, or both produced the different result.

Likewise, implementation evidence should be interpreted alongside differences in research design and other methodological features. A stronger design does not compensate for an intervention that was never properly delivered, while excellent implementation does not rescue a study from serious bias.

When writing the synthesis, replace vague claims such as “implementation may have differed” with the actual evidence. If one study delivered substantially fewer sessions, say so. If provider competence was not measured, say that it remains unknown. This produces a more defensible account of why the literature does not have one clear answer.

What to Compare Before Calling the Intervention Identical

Questions About Implementation and Conflicting Findings

Compare What Participants Received, Not Just the Intervention Name

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.