Manuel B. Garcia

Manuel B. Garcia serves as the Senior Director for Educational Technology and Digital Learning at FEU Institute of Technology, Manila, Philippines. Read More

Contact Info

1607, FEU Tech Building,
P. Paredes St, Sampaloc,
Manila, Philippines
mbgarcia@feutech.edu.ph

Follow Me

Did You Search Enough Sources to Reduce Database Bias?

Searching more databases does not automatically make a literature review comprehensive. What matters is whether your chosen sources collectively cover the disciplines, publication types, regions, and terminology relevant to your question.

843
Reducing Database Bias Guide 843 of 899
01 · The Question

Can Your Choice of Databases Shape the Evidence You Find?

You can construct a careful search strategy and still retrieve a distorted slice of the literature if you search it in the wrong places. Bibliographic databases do not contain identical collections of studies. They differ in journal coverage, disciplinary emphasis, geographic representation, indexing practices, document types, and search functionality.

This creates a practical problem. If your evidence exists across several research communities but you search only the database most familiar or convenient to you, some relevant studies may have little chance of entering your review at all.

The question is therefore not simply, "How many databases did I search?" A better question is: Did the sources I searched collectively give the relevant evidence a reasonable opportunity to be found?

02 · The Short Answer

There Is No Universal Number of Databases That Is Always Enough

In Brief

You have searched enough sources when your choice of databases and complementary search methods provides defensible coverage of the evidence relevant to your question, rather than merely reaching a predetermined number of databases.

For rigorous evidence synthesis, relying on a single general database may miss relevant records. The appropriate combination depends on the topic, disciplines involved, study designs sought, geographic scope, and review methodology. Adding databases with largely redundant coverage, however, does not automatically make a search better.

03 · What You Need to Know

Why Searching the Wrong Sources Can Bias Your Evidence Base

Databases are overlapping windows, not copies of the same library

Researchers sometimes speak of "the literature" as though it were one searchable collection. In practice, each bibliographic database exposes a particular portion of it. MEDLINE, Embase, PsycINFO, CINAHL, ERIC, Scopus, Web of Science, EconLit, and other databases were developed for different purposes and do not index precisely the same sources.

Even substantial overlap does not make databases interchangeable. In an exploratory study of 58 systematic reviews, Bramer and colleagues found that 16% of relevant references retrieved through database searching appeared in only one of the databases examined. The study also found that specialized databases could contribute unique references when the review topic matched their disciplinary focus. Those findings came from a particular collection of systematic reviews and should not be converted into a universal database prescription, but they illustrate why coverage matters.

Database bias begins before you type a search term

A search can retrieve only material that is accessible through the sources being searched. If an important journal, discipline, region, or type of evidence is poorly represented in those sources, an excellent Boolean strategy cannot recover what the database does not contain.

This is why database selection and search-term selection solve different problems. Expanding your range of search terms may improve retrieval within a database, but it does not correct gaps in that database's underlying coverage.

Vocabulary coverage Can your search recognize the different words researchers use for the concept?
Source coverage Are you searching places where the relevant research is actually indexed or discoverable?

Think about complementary coverage, not database count

Searching five databases is not necessarily more defensible than searching three. If all five emphasize similar journals and disciplines, the additional searches may mostly produce duplicates. Conversely, one specialized database can materially broaden a search if it reaches a body of evidence poorly represented elsewhere.

A useful way to select sources is to map your research question against the kinds of evidence it might generate. Ask which disciplines study the problem, where those disciplines publish, whether regional literature matters, and whether relevant studies are likely to appear outside conventional journal databases.

Feature of the question What to consider Why it matters
Cross-disciplinary topic Databases representing each major contributing discipline Terminology and journal coverage may differ substantially across fields
Specialized professional field A field-specific database where one is available Specialist journals may be incompletely represented in broad databases
International or regional question Relevant regional or national databases Major international databases may not provide equivalent coverage of all regions
Question involving multiple study types Sources that index the relevant designs and publication types A database strong for one evidence type may be less useful for another
Evidence likely to exist outside journals Registries, repositories, organizational sources, or other appropriate channels Bibliographic databases alone may not expose all relevant evidence

Broad and specialized databases can serve different purposes

Broad multidisciplinary databases can help researchers search across disciplinary boundaries. Specialized databases may provide deeper coverage of a particular field and often use discipline-specific indexing. Regional databases can matter when the research question requires evidence beyond the publication ecosystems emphasized by large international indexes.

PRISMA-S explicitly recognizes broad disciplinary, specialized, and regional databases as possible components of a systematic-review search. The appropriate combination is question-dependent. A psychology review and an economics review should not be expected to use identical database portfolios simply because both are systematic reviews.

The type of review changes what "enough" means

A systematic review intended to identify all eligible studies generally requires a more sensitive and reproducible search than a narrative review designed to provide a selective scholarly overview. Scoping reviews, evidence maps, rapid reviews, and other forms of synthesis may also make different trade-offs between comprehensiveness and feasibility.

For Cochrane intervention reviews, for example, methodological standards specify particular sources including CENTRAL, MEDLINE, and, when available to the review team, Embase. That is a requirement within that methodological context, not a universal rule that every literature review in every discipline must follow.

Watch Out

Do not turn a database combination validated for one field or review type into a universal formula. Evidence about retrieval performance is shaped by topic, eligibility criteria, database coverage, and the studies used to evaluate the search.

Database searching is only one part of information retrieval

Depending on the review, adequate source coverage may extend beyond bibliographic databases. PRISMA 2020 asks systematic-review authors to report databases, registers, websites, organizations, reference lists, and other sources used to identify studies. PRISMA-S similarly treats literature searching as broader than database searching alone.

This distinction becomes particularly important when unpublished or difficult-to-locate evidence could affect the synthesis. Trial registries, institutional or government sources, preprint servers, conference materials, citation searching, and direct contact may sometimes reveal evidence that conventional bibliographic databases do not.

How can you tell whether another database is adding anything?

One practical clue is the incremental contribution of each source. After deduplication and screening, examine where eligible studies were found. If a specialized database repeatedly contributes eligible records not found elsewhere, its value is apparent. If an additional source contributes almost nothing unique, that information can help you understand the marginal value of the source, although absence of unique included studies does not by itself prove that the database was unnecessary.

You can also test your search against several known relevant studies identified independently. If important papers that should satisfy your eligibility criteria cannot be retrieved, investigate why. The problem may lie in database coverage, indexing, terminology, or the construction of the search itself.

Report the sources precisely enough for someone else to understand the search

Transparency matters because readers cannot judge search coverage from a statement such as "several electronic databases were searched." PRISMA-S recommends naming each database and the platform through which it was searched. PRISMA 2020 likewise asks authors to specify all information sources and when each was last searched or consulted.

The platform is not trivial metadata. The same database can be available through different interfaces, and search syntax and functionality can differ. Recording both database and platform makes the search easier to evaluate and reproduce.

04 · A Practical Example

What Adequate Database Coverage Might Look Like

Hypothetical Example

A review of digital mental-health interventions for university students

Suppose a research team is conducting a systematic review of digital interventions intended to improve university students' mental health. The team begins with a major biomedical database because the outcome is health-related.

1. Map the evidence landscape The question intersects health, psychology, education, and digital technology. Relevant studies could therefore be published outside predominantly biomedical journals.
2. Identify complementary sources The researchers select databases that provide meaningful coverage of the principal disciplines rather than adding databases simply to increase the count.
3. Test retrieval They check whether known eligible or highly relevant studies can be retrieved and examine whether specialist sources produce potentially eligible records not captured by the initial search.
4. Investigate gaps A known psychology study is missing. The team determines whether the cause is database coverage or the search strategy itself. They revise the appropriate part rather than assuming that another generic database will solve the problem.
5. Document the rationale The final methods report names every database and platform, the search dates, and any complementary sources used so readers can evaluate the breadth of the search.

The lesson is not that every review of this topic must use a particular fixed set of databases. The defensible choice follows from the evidence landscape. If the research question crosses disciplinary boundaries, the search should not quietly impose the boundaries of a single database on the evidence.

05 · What Researchers Often Get Wrong

Common Mistakes When Deciding Where to Search

Misconception

Is One Large Database Comprehensive Enough?

Size is not the same as complete coverage. A large multidisciplinary database may index an enormous number of records while still missing specialist, regional, or otherwise relevant sources. Whether one database is sufficient depends on the purpose of the review, but a claim of comprehensive evidence identification requires stronger justification than database size alone.

Misconception

Does Searching More Databases Automatically Reduce Bias?

Not necessarily. Additional databases help when they broaden meaningful coverage. Searching several highly overlapping sources can greatly increase duplicate screening while adding little new evidence. The objective is complementary coverage, not an impressive database count in the methods section.

Misconception

Can Google Scholar Replace Bibliographic Databases?

Google Scholar can be useful as a supplementary discovery and citation-searching resource, but its search functionality, result presentation, and reproducibility differ from conventional bibliographic databases. Its usefulness in a particular review should therefore be considered in relation to the search objective rather than assuming that its broad reach makes other sources unnecessary.

Misconception

If Two Databases Return Similar Results, Is One Redundant?

Large overlap does not mean complete overlap. The records unique to a second database may include studies that matter to the conclusion. Judge redundancy by the needs of the review and the source's potential contribution, not simply by the number of duplicate records.

Misconception

Does Finding Many Papers Mean the Search Was Comprehensive?

No. Retrieval volume says little about coverage. Ten thousand records can still omit an important body of research if the wrong sources were searched. Comprehensiveness concerns the probability of finding relevant evidence, not the raw size of the search results.

Misconception

Can Citation Counts Tell You Which Sources Deserve Attention?

Citation visibility is not a substitute for systematic retrieval. Letting highly visible papers dominate discovery can overrepresent highly cited studies while making less visible but relevant evidence easier to miss.

06 · What This Means for You

Choose Sources From the Evidence Landscape, Not From Habit

Before running your final search, be able to explain why each major source is there. "I always use this database" is a weak rationale. "This database covers a discipline central to the question that is incompletely represented by our other sources" is much more informative.

You should also be able to explain important omissions. Access constraints, time, language capacity, and the methodological purpose of the review can all affect what is feasible. A transparent limitation is preferable to silently presenting a constrained search as exhaustive.

A simple decision framework

If your question sits mainly within one discipline
Identify the principal database or databases for that field, then determine whether another source adds relevant complementary coverage.
If your question crosses disciplines
Represent the major contributing fields rather than allowing one discipline's database to define the evidence base.
If geography or language matters
Consider whether regional databases or other sources are needed to avoid an evidence base dominated by the coverage of major international indexes.
If important evidence may not be conventionally published
Consider appropriate non-bibliographic sources rather than expecting journal databases to retrieve it.
If another database mostly duplicates existing results
Consider its likely unique contribution, methodological requirements, and the consequences of missing evidence before deciding whether it remains necessary.

After retrieval, continue looking at the composition of the evidence base. Even a broad search can produce a literature set dominated by one country or research ecosystem. If that happens, distinguish between a genuine feature of the available evidence and overrepresentation created by how evidence was located or selected.

07 · A Quick Checklist

Before You Decide Your Literature Search Is Broad Enough

Before finalizing your sources, check:
I identified the disciplines and research communities likely to produce evidence relevant to my question.
I selected databases for their relevant coverage, not simply because they were familiar or easily accessible.
I considered whether specialist or regional databases could contain evidence poorly represented in my main sources.
I checked whether known relevant studies can be retrieved and investigated important failures.
I considered whether evidence outside bibliographic databases could materially affect the review.
I can justify what each major information source contributes to the search.
I did not assume that a high number of retrieved records means that coverage is adequate.
I documented each database, its search platform, the search date, and other information sources clearly enough for the search to be evaluated.
08 · Frequently Asked Questions

Questions About Database Coverage and Literature Searching

How many databases should I search for a systematic review?

There is no single number that is appropriate for every systematic review. Database selection should reflect the question, disciplines, eligible study types, and applicable methodological standards. Some review frameworks impose specific source requirements, so check the guidance governing your review.

Is searching two databases always better than searching one?

Not automatically. A second database is valuable when it increases relevant coverage or serves another methodological purpose. Two strongly overlapping databases may provide less additional coverage than adding one well-chosen specialist source.

Should I search Scopus and Web of Science?

Either or both may be useful depending on your question and the other sources being searched. Their broad, multidisciplinary coverage can complement specialist databases, but neither should be treated as universally required. Base the choice on coverage relevant to your review and any applicable methodological guidance.

Can PubMed be my only database?

For some limited search purposes, one source may be defensible. For a review claiming comprehensive identification of evidence across fields, relying only on PubMed requires careful justification because relevant material may be indexed elsewhere. The required breadth depends on the review question and methodology.

Do I need to search a database if most of its results are duplicates?

Duplicate volume alone cannot answer that question. What matters is whether the database can contribute relevant records that your other sources miss and whether your review methodology requires it. A source can have substantial overlap while still contributing important unique evidence.

What if I cannot access an important subscription database?

Explore institutional library access, interlibrary collaboration, or appropriate alternative sources, but do not imply that an alternative necessarily has equivalent coverage. If access constraints materially limit the search, report that limitation transparently and consider its possible effect on the evidence base.

Should I report databases that produced no included studies?

Yes, when they formed part of the search. Reporting information sources describes how the evidence was sought, not only where the final included studies happened to be found. Omitting an unproductive database would make the search appear different from the one actually conducted.

09 · The Bottom Line

A Good Search Is Broad for a Reason

The Bottom Line

You have not necessarily searched enough because you used many databases; you have searched enough when your sources collectively provide a defensible opportunity to identify the evidence relevant to your question.

Choose databases according to disciplinary, methodological, geographic, and publication coverage, supplement them when important evidence may lie elsewhere, and document the choices transparently. The goal is not to collect database names. It is to reduce the chance that the architecture of your search quietly determines the conclusion of your review.

10 · Sources and Further Reading

Sources and Further Reading

Is this conversation helpful so far?
11 · Cite this Guide

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.

Has the Field Guide helped your research?

If a guide helped clarify a question, inform a research decision, or move your work forward, I would love to hear about your experience. Your story may also help other researchers discover the Field Guide.

Share Your Experience
Takes only a few minutes