01 · The Question
How Comprehensive Does a Systematic Review Search Need to Be?
For a systematic review, literature searching is not merely preparation for writing. It is part of the research method. The studies identified through the search become the pool from which evidence is selected, analyzed, and synthesized. If the search systematically misses relevant evidence, sophisticated analysis later in the review cannot simply repair that omission.
This creates a demanding standard. Does a systematic review need to find literally every eligible study? In practice, absolute completeness cannot usually be proven. The methodological objective is instead to search comprehensively, systematically, transparently, and reproducibly enough to minimize bias and make important eligible evidence unlikely to be missed.
03 · What You Need to Know
The Search Determines What Evidence Can Enter the Review
Comprehensiveness Is a Methodological Requirement, Not a Decorative Ideal
A systematic review attempts to identify evidence according to explicit eligibility criteria rather than according to which studies happen to be familiar, easily found, or supportive of an expected conclusion. Search methods therefore influence the evidence available for screening and synthesis.
Cochrane describes systematic reviews as requiring a thorough, objective, and reproducible search across a range of sources and recommends that searches be as extensive as possible within resource constraints. For intervention reviews, it specifically warns that searching MEDLINE alone is inadequate because relevant studies may be absent from the database or difficult to retrieve even when indexed there.
The precise requirements differ across review types and disciplines. Cochrane guidance is primarily designed for health intervention reviews and should not simply be transplanted into every field. The general principle nevertheless travels well: information sources should be selected according to where eligible evidence is likely to exist, not according to which database the review team happens to use most often.
There Is No Universal Minimum Number of Databases
"How many databases do I need?" sounds like a straightforward question, but database count is a poor substitute for coverage. Two highly overlapping databases may contribute less than two sources covering genuinely different segments of the eligible literature.
Database selection should follow the review question, disciplines, study designs, publication types, geography, language, and other characteristics of potentially eligible evidence. A health intervention review, educational evidence synthesis, engineering review, and interdisciplinary social-science review may require substantially different information sources.
Where an established methodological handbook or review organization specifies mandatory sources for your type of review, follow those requirements. For example, current Cochrane guidance specifies core databases and trial-register searching for Cochrane intervention reviews.
Search Strategies Should Favor Sensitivity Without Becoming Pointlessly Broad
A systematic review search generally prioritizes sensitivity, sometimes called recall: the ability to retrieve relevant records. This often means accepting many irrelevant records during screening because an overly restrictive search can silently remove eligible studies before reviewers ever see them.
That does not mean adding every conceivable concept to a query. Cochrane cautions against using too many search concepts and recommends using a broad range of terms within the concepts that are included. A complicated search can actually become less sensitive if every additional concept creates another condition a record must satisfy.
Sensitivity
The proportion of relevant records in a source that the search successfully retrieves.
Precision
The proportion of retrieved records that are actually relevant.
Systematic-review searching often accepts lower precision in exchange for higher sensitivity, but the optimal balance depends on the review and the feasibility of screening the resulting records.
Use Both Database Vocabulary and Natural Language Where Appropriate
Relevant studies may be indexed under controlled subject headings while also using varied terminology in titles and abstracts. Robust database strategies commonly combine controlled vocabulary, where available, with free-text terms.
Searchers should consider synonyms, acronyms, spelling variants, historical terminology, technical terminology, and differences in language across disciplines. A search designed in one database also needs appropriate translation for other databases because indexing systems and search syntax differ.
Do Not Assume Bibliographic Databases Contain All Eligible Evidence
Depending on the review question, relevant evidence may appear in trial registers, dissertations, conference materials, regulatory documents, reports, repositories, or other sources outside conventional journal databases. Searching these sources can reduce the risk that the review disproportionately represents readily published evidence.
Cochrane, for example, requires trial-register searching for its intervention reviews and recommends consideration of relevant grey literature and previous reviews. The appropriate supplementary sources for another discipline may be different.
Citation Searching Can Find Evidence Keyword Searches Miss
Reference lists of included or highly relevant studies can reveal earlier work, while forward citation searching can identify later studies that cite them. Citation searching can be particularly useful when terminology is inconsistent or indexing is weak.
It should normally complement rather than casually replace appropriate database searching when comprehensive retrieval is the objective. The search process should also document how citation searching was performed when it contributes to study identification.
Search the Evidence, Not Merely the Journal Literature
Publication bias and related reporting processes matter because published journal articles may not represent all research that has been conducted. Depending on the review, searching only conventional published literature can therefore produce an incomplete or systematically distorted evidence base.
This is one reason systematic-review methodology pays attention to registers and other non-journal sources. The relevant sources depend on the review, but the search should be designed around eligible evidence rather than around a single publication format.
Search Methods Should Be Reproducible
Readers should be able to understand what sources were searched, when they were searched, and how. PRISMA-S was developed specifically to improve reporting of literature searches for systematic reviews and related evidence syntheses. Its 16 reporting items are intended to support complete and reproducible reporting of information sources and search methods.
This is more demanding than simply writing "PubMed and Google Scholar were searched using relevant keywords." For a method-driven evidence synthesis, the need for a reproducible search follows directly from the role retrieval plays in determining the review's evidence base.
Watch Out
PRISMA-S is principally a reporting guideline. Reporting a weak search perfectly does not make the search methodologically adequate. Search conduct and search reporting are related but distinct quality problems.
Consider Peer Review of the Search Strategy
Complex search strategies can contain errors in Boolean logic, subject headings, text words, syntax, limits, or translation of the research question into searchable concepts. The PRESS guideline provides a structured framework for peer review of electronic search strategies and was developed for systematic reviews, health technology assessments, and related evidence syntheses.
Current Cochrane guidance also recommends involving an experienced librarian or information specialist from the protocol stage. The value of specialist involvement becomes especially apparent when the topic is difficult to search, terminology is unstable, multiple databases require translation, or missing eligible evidence could materially affect the synthesis.
Searches Usually Need Updating
Systematic reviews take time. A search conducted during protocol development can become outdated before publication. The review team should therefore plan how and when searches will be updated according to applicable methodological guidance and the pace of the field.
An updated search is not simply a courtesy to readers. New eligible studies can change estimates, interpretations, certainty assessments, or conclusions.
When Is the Search Thorough Enough?
Unlike an ordinary background search, a systematic-review search should not stop merely because additional searching feels repetitive. The stopping point is determined by the planned search methods and applicable review methodology.
Once all prespecified and otherwise justified information sources and supplementary search methods have been completed, documented, quality-checked, and updated as necessary, the review can proceed without claiming impossible certainty that no eligible study exists anywhere. The defensible claim is that a methodologically appropriate search was conducted to identify the eligible evidence.
06 · What This Means for You
Design the Search Around Eligible Evidence, Not Around Convenience
For a systematic review, search planning belongs in the methodology. Decide where eligible evidence is likely to be found, how it might be described, what supplementary methods are necessary, and how the process will be documented before screening begins.
A simple decision framework
If your search relies on one general database
Determine which relevant literature that database does not cover adequately and add appropriate sources.
If the search retrieves very few records despite a substantial known literature
Reconsider the search concepts, terminology, indexing terms, fields, filters, and Boolean logic before assuming the literature is genuinely small.
If your search retrieves known eligible studies only inconsistently
Investigate why and revise the strategy where appropriate.
If eligible evidence may exist outside conventional journals
Identify appropriate registers, grey-literature sources, repositories, regulatory sources, or other resources relevant to the review.
If the strategy is complex or the review is consequential
Consider involving an information specialist and using structured peer review of the search strategy.
If you cannot reproduce exactly what was searched
Improve documentation before treating the search as complete.
The practical goal is not a magical point called "100% complete." It is a search process sufficiently comprehensive, transparent, and reproducible that reasonable methodological choices, rather than convenience or selective retrieval, determine which evidence has the opportunity to enter the review.
07 · A Quick Checklist
Before Declaring a Systematic Review Search Complete
Before screening or finalizing the review, check:
The search is explicitly aligned with the review question and eligibility criteria.
Database selection reflects where eligible evidence is likely to be indexed rather than which databases are easiest to access.
Search concepts use appropriate synonyms, free-text terminology, and controlled vocabulary where available.
The strategy has been translated appropriately for the syntax and indexing of each database.
Known relevant studies have been used, where appropriate, to help test whether the search behaves as expected.
Relevant registers, grey literature, citation searching, previous reviews, or other supplementary sources have been considered according to the review question and methodology.
Search limits and filters are justified and are not silently excluding eligible evidence.
Search strategies, sources, platforms, dates, limits, and supplementary methods are documented sufficiently for transparent reporting and reproducibility.
The search has been updated when required by the review timeline and applicable methodology.
I have checked the methodological and reporting standards appropriate to this specific type of systematic review rather than assuming one framework applies universally.
