When Should You Leave an Outcome Out of the Search Strategy?

If the Outcome Matters to the Question, Why Would You Leave It Out of the Search?

Suppose your review asks whether an educational intervention improves critical thinking. Critical thinking is clearly central to the question. It defines an outcome you care about and may determine whether a study is ultimately eligible.

It therefore seems natural to add a critical-thinking block to the database search. Yet doing so creates a new requirement: potentially relevant records must now contain terminology that your search recognizes as representing that outcome.

That can become a problem when authors describe the same outcome in different ways, use an instrument name rather than the construct itself, or mention the outcome only in parts of the article that your database search does not cover. In such cases, an outcome can be conceptually essential but strategically unhelpful as a retrieval condition.

Leave the Outcome Out When Requiring It Would Unnecessarily Restrict Retrieval

Why Important Outcomes Are Sometimes Poor Search Concepts

An outcome can be essential for eligibility without being essential for retrieval

The distinction between the research question and the search strategy matters here. Your question defines the evidence you ultimately want. The database query defines which records get an opportunity to be screened.

Those functions overlap, but they are not identical.

If your review includes only studies measuring academic achievement, academic achievement remains an eligibility criterion whether or not the words "academic achievement" appear in the database query. You can retrieve studies using population and intervention concepts and then determine during screening whether each study actually measures the required outcome.

This follows the broader principle that not every concept in a research question must become a search block.

Outcome terminology can vary considerably

Many outcomes do not have one stable label. A researcher interested in academic achievement might encounter expressions such as academic performance, educational achievement, learning performance, examination performance, course grades, test scores, or names of particular assessments.

Even these expressions may not capture every eligible study. One paper might report "final examination scores" without using a broader achievement term. Another may identify the assessment instrument but not the construct your review assigns to it.

The difficulty becomes greater for multidimensional constructs such as engagement, well-being, digital competence, critical thinking, quality of life, implementation success, or attitudes. Different disciplines may operationalize ostensibly similar outcomes using quite different language.

A long list of outcome synonyms can help, but length does not guarantee conceptual coverage. As discussed in relation to very long search strategies, a large vocabulary block can still miss the terminology used by relevant records.

The outcome may not appear in the searchable record

Even perfect terminology cannot retrieve text that is absent from the fields being searched.

An article may clearly measure your outcome but emphasize the intervention, population, or primary objective in its title and abstract. Secondary outcomes may receive little or no abstract space. Detailed outcome measures may appear only in the methods, tables, supplementary material, or full text.

Whether this creates a problem depends partly on which database fields your strategy searches. Bibliographic searches commonly rely heavily on titles, abstracts, controlled vocabulary, and other indexed metadata. If the outcome is not represented there, adding more synonyms cannot solve the underlying problem.

Adding an outcome with AND creates another point of failure

Suppose your search contains:

(university students) AND (generative AI)

A relevant record needs to satisfy those two concept blocks.

Now add:

AND (critical thinking)

The record must also satisfy your representation of the outcome. Any eligible study that fails that third textual or indexing condition may disappear from retrieval.

This is a direct consequence of Boolean intersection. Adding another search concept can therefore make a search worse even when the concept itself is scientifically important.

Systematic-review searches often prioritize sensitivity

For systematic reviews, missing an eligible study may be more consequential than retrieving additional irrelevant records. Cochrane's current guidance recommends maximizing sensitivity while striving for reasonable precision and advises avoiding too many different search concepts. Within selected concepts, a wide range of terms may then be combined with OR.

This does not establish a universal rule that outcomes should never be searched. Rather, it explains why a questionable outcome block deserves scrutiny when its main effect is to make retrieval more restrictive.

When is an outcome more reasonable to include?

Outcome searching can be useful when the outcome is distinctive, consistently reported, and necessary to separate the target literature from a much larger body of unrelated research.

Imagine a question about a highly specific adverse event that has stable terminology and reliable indexing. If searching population and intervention alone retrieves an enormous literature and the outcome terminology performs well during testing, including the outcome may produce a useful improvement in precision.

The important question is empirical: what happens to retrieval when the outcome is added?

Leaving an outcome out transfers work to screening

There is no free lunch in information retrieval, although systematic reviewers have probably suspected this for some time.

If you remove an outcome block, the search usually becomes broader. More records may need title-and-abstract screening, and some records may require full-text inspection before you can determine whether the relevant outcome was measured.

The trade-off is deliberate. You accept more screening in exchange for reducing the chance that the database query eliminates an eligible study before a human reviewer ever sees it.

Do not decide from theory alone

Test both versions where practical. Run the strategy with and without the outcome block. Examine known relevant studies and inspect records that disappear when the outcome is added.

If important records vanish because they use unexpected terminology or do not expose the outcome in searchable metadata, you have direct evidence that the block is risky. Conversely, if the outcome substantially reduces irrelevant retrieval while retaining relevant records, its inclusion may be justified.

The resulting decision should reflect the purpose of the search and the relative importance of sensitivity and precision.

When Searching the Outcome Hides the Study You Need

The broader strategy creates more work, but it avoids requiring authors to describe the outcome using terminology predicted in advance by the searcher.

Common Mistakes When Searching for Outcomes

Make the Outcome Earn Its Place in the Search

Start by examining how relevant studies actually describe the outcome. Do not rely solely on the terminology in your protocol or conceptual framework. Look at titles, abstracts, author keywords, controlled vocabulary, instrument names, and terminology used across disciplines.

Then compare search versions. The purpose is not to prove that an outcome can technically be searched. Almost anything can be turned into a query. The question is whether making it mandatory improves the retrieval task.

For a systematic review, document the decision. An intentionally omitted outcome block is not an omission in the methodological sense when the rationale is that the outcome is better assessed during study selection.

Before Adding an Outcome to Your Search Strategy

Questions About Outcome Terms in Database Searches

An Important Outcome Does Not Automatically Make a Good Search Concept

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.