01 · The Question
What If the Studies You Can Read Are Not All the Studies That Were Done?
A literature search usually begins with published research. That is sensible, but it creates a potential blind spot: not every completed study becomes a journal article, and not every result from a study is necessarily reported in the published paper.
More importantly, what becomes publicly visible may not be random. Research with statistically significant, favorable, striking, or otherwise publishable findings can have a different probability of dissemination from research with null or unfavorable findings. Cochrane's current guidance summarizes empirical evidence showing that publication of study reports can depend on the nature and direction of results.
If that mechanism is relevant to your question, restricting the evidence base to easily located journal articles can make the available literature look more favorable, more decisive, or simply different from the underlying body of research.
02 · The Short Answer
Search Beyond Published Articles When Missing Evidence Could Distort the Answer
In Brief
You should search for unpublished or otherwise difficult-to-locate evidence when the absence of studies or results from conventional publications could materially bias your understanding of the evidence.
The appropriate effort depends on the research question and review type. Trial registries, conference proceedings, theses, regulatory sources, repositories, organizational websites, protocols, and contact with investigators may be useful in some contexts, but unpublished evidence is not automatically complete, unbiased, or high quality simply because it is unpublished.
03 · What You Need to Know
Why Published Literature Can Be an Incomplete Evidence Base
Publication is an outcome of the research process
Researchers sometimes treat publication as though it were merely the final administrative step after a study is completed. Methodologically, publication can also act as a selection mechanism. Investigators may decide whether and where to submit findings, journals decide what to publish, and some studies remain available only through registries, reports, theses, conference materials, regulatory records, or other channels.
This becomes a source of bias when dissemination is related to the study's results. Cochrane cites evidence from cohorts of registered or otherwise traceable studies showing that studies with statistically significant results have been more likely to be published than studies with non-significant results. Evidence from biomedical conference abstracts similarly indicates that not all conference research subsequently reaches full publication.
Missing study
A study was conducted but is absent from the set of reports or sources available to your review.
Missing result
A study is known, but an eligible outcome, analysis, time point, or other result is unavailable or incompletely reported.
The distinction matters. Finding an unpublished study does not guarantee that the result you need is available, while locating a published article does not guarantee that every eligible result from that study was reported. Cochrane therefore treats bias due to missing evidence as broader than the simple published-versus-unpublished distinction.
Unpublished evidence and grey literature are not exactly the same thing
"Unpublished evidence" can refer to studies or results that have not appeared as conventional journal publications. "Grey literature" is broader and can include material produced outside traditional commercial or academic publishing channels, such as reports, theses, working papers, government documents, and some conference materials.
Some grey literature is publicly available and formally disseminated. Some unpublished information is not readily public at all. The terms therefore overlap, but treating them as exact synonyms can obscure what source you are actually trying to locate.
Where might evidence exist outside conventional journal articles?
The answer depends heavily on discipline and study design. Clinical trials may be registered before completion, and some registries contain summary results. Regulatory submissions can contain information unavailable in journal reports. Theses and dissertations may contain completed research that was never converted into journal articles. Conference abstracts can provide traces of studies whose full reports are unavailable. Government agencies, research organizations, funders, and institutional repositories may also disseminate research outside journals.
Possible source
What it may reveal
Important limitation
Study or trial registries
Ongoing, completed, or otherwise known studies and sometimes results
Registration and result reporting vary by field, period, jurisdiction, and study type
Conference abstracts or proceedings
Studies presented before or without full journal publication
Methods and results may be too abbreviated for confident appraisal or synthesis
Theses and dissertations
Completed academic research not represented in journals
Discovery, accessibility, and indexing can vary substantially
Regulatory sources
Studies or results submitted during regulatory processes
Availability depends on the intervention, regulator, period, and submission context
Institutional or subject repositories
Reports, manuscripts, preprints, theses, and other research outputs
Coverage and metadata quality vary
Researchers, sponsors, or organizations
Additional reports, clarification, or missing data
Responses may be incomplete or unavailable
Trial registries can reveal studies that journal databases cannot
For questions involving clinical trials, registries can be particularly important because they can identify initiated, ongoing, or completed studies regardless of whether a conventional journal report is available. Cochrane notes that registry information can also help reviewers assess whether evidence may be missing.
In one investigation summarized in the Cochrane Handbook, researchers searched trial registries for 95 systematic reviews of pharmaceutical interventions that had not already done so. Eligible additional trials were identified for 41 of the reviews. Results were available for only a subset, and incorporating those results did not alter every synthesis, but some meta-analytic estimates changed appreciably. This is evidence that registry searching can matter in some settings, not evidence that it will always change a review's conclusion.
Searching for unpublished evidence does not guarantee that you will find all of it
This limitation deserves emphasis. Unpublished evidence can be difficult to discover precisely because it lacks conventional publication and indexing pathways. Even a serious search may retrieve only a subset of what exists.
Cochrane consequently cautions that the unpublished studies researchers manage to locate may themselves be an unrepresentative sample of all unpublished studies. Searching beyond journal publications can reduce one source of bias without magically producing a complete hidden evidence base.
Unpublished does not mean unbiased
A report's publication status tells you where and how it was disseminated. It does not establish methodological quality. An unpublished study may have serious design problems. A conference abstract may contain too little methodological detail for confident appraisal. Registry information may be incomplete or outdated. Conversely, journal publication and peer review do not guarantee that a study is methodologically strong.
Every eligible source still requires appropriate critical appraisal. Publication status should not become a shortcut for deciding how much evidential weight a study deserves. In other words, finding more evidence and deciding how strongly different evidence should influence your conclusion are separate tasks.
Searching and including are different decisions
You may identify a registry record or conference abstract that clearly describes an eligible study but provides insufficient data for your planned synthesis. That study still matters. Its existence may affect your assessment of missing evidence even if you cannot calculate an effect estimate from it.
Cochrane specifically cautions against ignoring a known eligible registered study simply because its results are unavailable. A completed study with no accessible results can itself be relevant when judging the possibility of bias due to missing evidence.
Publication status should not usually become an arbitrary eligibility rule
For Cochrane intervention reviews, studies are expected to be included irrespective of publication status unless exclusion is explicitly justified. The rationale is that excluding unpublished, partially published, or grey-literature studies can amplify publication bias. This is a methodological requirement within that review framework, not a universal rule for every form of literature review.
Your own decision should follow the purpose and methodology of the review. A systematic review seeking all eligible studies generally has a stronger reason to investigate missing evidence than a narrative overview that makes no claim of exhaustive retrieval.
Searching beyond journals should be documented just as carefully
PRISMA 2020 asks systematic-review authors to specify databases, registers, websites, organizations, reference lists, and other sources searched or consulted, together with relevant search dates. It also asks authors to present the full search strategies for databases, registers, and websites.
PRISMA-S extends this reporting logic to citation searching, contacts, and other search methods. The basic principle is straightforward: searching beyond journals should not become an undocumented excursion through the internet. Readers need to know where and how you looked.
04 · A Practical Example
How Missing Studies Could Change the Story
Hypothetical Example
A review of a digital intervention for reducing examination anxiety
Suppose a systematic review identifies six published controlled trials. Five report results favoring the intervention, while one reports little difference. Based only on those papers, the literature appears consistently positive.
Search conventional databases
The team identifies the six journal articles and completes the initial screening.
Search appropriate registries and repositories
The researchers find records for two additional completed eligible studies that do not have corresponding journal articles.
Investigate the missing evidence
One registry provides usable summary results showing little difference between groups. The other confirms that an eligible study was completed, but no usable results can be located.
Update the evidence assessment
The additional result is incorporated where methodologically appropriate. The study with unavailable results is documented and considered when assessing the possibility of missing-evidence bias.
Interpret more cautiously
The published literature still contains positive findings, but the complete known evidence landscape is less uniformly favorable than the journal articles alone suggested.
This example does not imply that unpublished studies will always weaken an effect or reverse a conclusion. Cochrane's synthesis of methodological evidence indicates that including results from sources beyond journal articles can affect estimates in different directions and by different magnitudes. The point is that you should not assume the visible published literature is an unbiased sample of everything that was studied.
06 · What This Means for You
Decide Whether Missing Evidence Could Plausibly Change Your Interpretation
The effort devoted to unpublished evidence should be proportionate to the review question and the consequences of selective availability. Ask what kinds of research you are including, whether those studies are commonly registered or disseminated outside journals, and whether favorable or statistically significant findings could have a different path to publication than null or unfavorable findings.
A simple decision framework
If your review includes prospectively registered studies such as many clinical trials
Search appropriate study registries and investigate eligible completed studies without corresponding publications.
If theses, reports, conference materials, or regulatory documents are plausible dissemination routes
Identify the sources most relevant to your discipline rather than assuming journal databases cover them adequately.
If a study is known but its results are missing
Document the study, seek additional information where feasible, and consider its implications for bias due to missing evidence.
If an unpublished source provides usable results
Apply the same eligibility rules and appropriate critical appraisal used for other evidence rather than favoring or dismissing it because of publication status.
If your review does not search beyond conventional publications
Avoid claims of comprehensiveness that the search cannot support and consider the omission when discussing limitations.
Searching more places is useful only when those places plausibly contain relevant evidence. The broader question of whether your information sources collectively provide adequate coverage should therefore be considered alongside database selection .
07 · A Quick Checklist
Before You Decide Published Articles Are Enough
Before closing the search, check:
I considered whether publication or result availability could plausibly depend on the direction, magnitude, or statistical significance of findings.
I identified the non-journal dissemination routes that are genuinely relevant to my study designs and discipline.
Where relevant, I searched appropriate study or trial registries for ongoing, completed, or otherwise unpublished studies.
I considered relevant theses, conference materials, repositories, regulatory sources, organizational reports, or researcher contact where these could materially improve coverage.
I tracked known eligible studies even when their usable results could not be obtained.
I did not assume that unpublished evidence is automatically stronger, weaker, or less biased than published evidence.
I checked whether multiple reports refer to the same underlying study before treating them as separate evidence.
I documented the additional sources and search methods clearly enough for readers to understand what I searched and when.
11 · Cite this Guide
How to Cite This Guide
This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.
Recommended (Field Guide)
APA
MLA
Chicago
Copy Citation
