01 · The Question
What If Most of the Evidence Comes From the Same Researchers?
You search the literature and initially find what looks like a substantial evidence base: fifteen papers, several datasets, multiple analyses, and findings that appear remarkably consistent.
Then you notice the author lists.
The same names appear repeatedly. Sometimes the order changes. New collaborators come and go. Several papers come from the same laboratory or institution. Some appear to use related samples. Others extend earlier studies, analyze different outcomes, or report additional follow-up periods.
The literature may still contain valuable evidence. But fifteen publications produced by one interconnected research program do not provide the same kind of corroboration as fifteen genuinely independent investigations.
The synthesis problem is therefore not whether to exclude prolific researchers. It is to determine how much independent evidence actually exists beneath the publication count.
03 · What You Need to Know
A Large Literature Can Have a Surprisingly Small Number of Independent Sources
Start with studies, not publications
The first problem is deceptively simple: a paper is not necessarily a study.
One study can produce a primary outcomes paper, secondary analyses, subgroup analyses, methodological papers, follow-up reports, conference abstracts, and later publications examining additional outcomes. Treating those reports as independent studies can substantially exaggerate the apparent size of the evidence base.
Cochrane explicitly treats the study, rather than the report, as the principal unit of interest in systematic reviews. Multiple reports from the same study should be identified and linked rather than counted as separate studies.
Publication count
The number of articles, reports, abstracts, or other outputs you retrieved.
Independent evidence count
The number of distinct studies, samples, datasets, or other evidential units that provide genuinely separate information for the inference being made.
The two numbers can differ substantially.
Author overlap is a clue, not proof of dependence
Seeing the same senior author on several papers does not mean those papers report the same study. Productive research groups can conduct genuinely independent projects.
Likewise, different author lists do not guarantee independence. Collaborating teams can publish multiple papers from the same trial or dataset, and secondary publications may omit some authors who appeared on the primary report.
Cochrane recommends using multiple characteristics to determine whether reports belong to the same study, including registration identifiers, author names, sponsors, settings, intervention details, participant numbers and baseline characteristics, and dates of recruitment or follow-up.
In practice, you may need a small amount of scholarly detective work. Literature reviewing occasionally rewards the instincts of someone who has spent too much time comparing sample sizes in table 1.
Map the provenance of the evidence
When one group dominates a field, create an evidence-provenance map before writing the synthesis.
For each publication, record enough information to determine where its evidence came from. Depending on the literature, this may include:
- author team and institutional affiliations;
- study or trial registration number;
- dataset or cohort name;
- recruitment sites;
- sample size and distinctive participant characteristics;
- recruitment dates;
- intervention and comparison details;
- funding source;
- follow-up period;
- whether the analysis is primary, secondary, exploratory, or a follow-up;
- whether participants overlap with another publication.
Your goal is not to police productivity. It is to understand the dependency structure of the evidence.
Repeated analyses of the same participants are not independent replications
Suppose a research team publishes five papers from one cohort. One examines achievement, another motivation, another subgroup differences, another two-year follow-up, and another mediation.
Those papers may answer five legitimate questions. But if all five are used to support the same overarching claim, they do not constitute five independent replications.
Cochrane notes that effects calculated from the same participants are statistically dependent. When multiple eligible outcomes or measures arise from one study, review methods should avoid treating dependent information as though it came from independent samples.
The principle matters beyond formal meta-analysis. A narrative review can also accidentally double-count evidence by writing each publication as another independent confirmation.
Research-group dependence extends beyond shared participants
Even genuinely separate studies conducted by the same team may share features that make them less independent in a broader scientific sense.
The investigators may use the same recruitment networks, intervention protocol, instruments, analytic conventions, theoretical assumptions, implementation expertise, or institutional environment. These common features are not necessarily flaws. In fact, methodological consistency can be valuable.
But repeated success within one research program answers a somewhat different question from successful replication by researchers working independently.
Repeated evidence within a research program
Shows whether a finding can recur across studies conducted under related investigators, methods, procedures, or contexts.
Independent replication
Provides evidence about whether a finding can recur when some of those researcher-specific dependencies are removed.
Both matter. They should not be described as though they provide identical evidence.
Consistency within one group can still be informative
A concentrated evidence base should not be dismissed merely because the same investigators produced much of it.
A research program may progressively test a phenomenon across samples, refine measurements, examine mechanisms, conduct follow-up studies, or replicate findings in different settings. Such work can generate substantial knowledge.
The appropriate interpretation is narrower: the finding may be well replicated within that research program while remaining less independently replicated across research teams.
This distinction prevents two opposite errors: treating concentration as proof of bias and treating repeated publications as broad independent confirmation.
Check whether apparently separate studies share the same dataset
A research group may publish numerous analyses from a large cohort, longitudinal project, administrative database, or institutional dataset. Those analyses can look like separate studies because they ask different questions or use different subsets.
If they repeatedly contribute to the same synthesis, however, their dependence matters. This issue becomes sufficiently important when one dataset dominates the literature that it deserves separate treatment rather than being reduced to an author-overlap problem.
When relevant, trace whether multiple publications ultimately rely on the same underlying dataset.
Do not double-count participants in quantitative synthesis
Dependency becomes especially consequential in meta-analysis. Including the same participants more than once as though they were independent can produce a unit-of-analysis error and spuriously increase precision.
Cochrane specifically warns against double-counting participants when multiple comparisons share groups and recommends analytical approaches that account for the dependency.
The exact statistical solution depends on the dependency structure. The general principle is simpler: more effect estimates do not automatically mean more independent information.
Research-group concentration can interact with conflicts of interest
When one group produces much of a literature, it can be useful to examine funding and investigator interests alongside the ordinary assessment of study quality. Cochrane recommends considering study funding sources and author conflicts of interest because these may inform interpretation of heterogeneity, risk of bias, and missing results.
This should not become guilt by association. A declared conflict does not automatically invalidate a study, and absence of a declared conflict does not guarantee impartiality.
The relevant question is whether characteristics of the research program provide plausible explanations for patterns in the evidence and whether the studies themselves show methodological vulnerabilities.
Look for what happens outside the dominant group
If independent studies exist, compare them with the dominant group's findings.
Several patterns are possible:
| Pattern |
What it may indicate |
Main caution |
| Independent studies broadly reproduce the dominant group's findings |
The pattern extends beyond the originating research program |
Check whether methods, settings, or datasets are genuinely independent |
| Independent studies find smaller effects |
Research-group or methodological differences may matter |
Do not assume investigator identity itself caused the difference |
| Independent studies produce inconsistent findings |
The apparent consensus may be less general than publication counts suggest |
Examine differences in design, population, measurement, and setting |
| No independent studies exist |
The finding lacks independent replication |
This does not mean the existing studies are false |
| Many papers trace back to a few original studies |
The evidence base is smaller than the publication count suggests |
Count studies and independent samples rather than papers |
Do not turn research-group identity into an explanation without evidence
Suppose one research team consistently reports large effects while independent teams report smaller ones. That difference deserves investigation. It does not justify writing that the original team exaggerated the effect.
The groups may differ in implementation expertise, populations, settings, measurements, study designs, intervention fidelity, or other features.
Researcher identity can identify a cluster in the evidence. Explaining why that cluster differs requires further evidence.
A concentrated literature should narrow your claim, not necessarily reverse it
If ten rigorous studies from one group consistently support a finding, the correct conclusion is not that there is “no evidence.” There may be substantial evidence.
The qualification concerns independence and breadth.
You might conclude that the relationship has been repeatedly observed by one research program across several studies, but independent replication remains limited. That statement accurately represents both the strength and the boundary of the evidence.
This is part of avoiding greater certainty than the literature warrants.
