03 · What You Need to Know
Why Study Design Is Sometimes Searchable and Sometimes Better Left for Screening
Study design can be an eligibility criterion without being a safe search restriction
Suppose your review includes only prospective cohort studies. That design criterion tells you which studies ultimately qualify. It does not automatically establish that every eligible cohort study can be reliably identified through database terminology.
The same distinction applies to outcomes, populations, and other criteria. Eligibility criteria and search concepts do not need to be identical.
If design is omitted from the search, you retrieve records using other concepts and classify the study design during screening. This usually increases screening workload but may protect against losses caused by imperfect terminology or indexing.
Authors do not always label their study design consistently
Study-design terminology can be surprisingly messy.
A study that a reviewer classifies as a cohort study might describe itself as longitudinal, prospective, follow-up, observational, registry-based, or simply explain its procedures without prominently naming the design. Terminology can also be used inconsistently across disciplines.
The same difficulty can arise for quasi-experimental designs, process evaluations, diagnostic studies, qualitative designs, and other methodological categories. Cochrane's qualitative-evidence guidance, for example, notes that some approaches using strings of terms associated with study type or purpose remain experimental and require further development and testing.
A short list of obvious design labels may therefore have good face validity while performing poorly as a retrieval device.
Database indexing can help, but it is not uniformly sufficient
Some databases assign publication types or controlled-vocabulary terms related to methodology. These can be valuable because retrieval does not depend entirely on words chosen by the authors.
However, indexing systems differ between databases, and recently added records may not yet have complete subject indexing. Design categories can also be broader or narrower than your eligibility definition.
This is one reason the same search construction cannot simply be assumed to work across databases. A methodological filter designed for MEDLINE, for example, should not be treated as database-independent syntax.
A search filter is more than a few study-design keywords
A methodological search filter is a search strategy developed to retrieve records with a particular characteristic, often a study design. Filters may combine publication types, controlled vocabulary, text words, field restrictions, Boolean operators, and sometimes exclusion logic.
The important distinction is between a tested filter and an improvised design block.
Methodological search filter
A search strategy designed to retrieve a particular type of record and ideally evaluated for retrieval performance in the relevant database.
Ad hoc design terms
Terms selected because they appear to describe the desired methodology, without necessarily having evidence about how reliably they identify eligible studies.
Cochrane recommends considering published, highly sensitive, validated filters for identifying randomized trials in databases such as MEDLINE, Embase, and CINAHL. Its current guidance also cautions that filters should be assessed for their development, reported performance, current accuracy, relevance, and effectiveness because database interfaces and indexing change over time.
Randomized trials are an important special case
Randomized controlled trials have received considerable attention in search-filter development. Cochrane provides highly sensitive strategies for identifying randomized trials in MEDLINE and controlled trials in other major databases.
These filters do not merely search for the exact phrase "randomized controlled trial." The sensitivity-maximizing MEDLINE strategy, for example, combines publication types and several text or indexing signals associated with trials.
For a review restricted to randomized trials, an established high-sensitivity filter may therefore be more defensible than leaving study design entirely unrestricted, depending on the database and review method.
Even here, context matters. Cochrane explicitly advises against adding randomized-trial or human filters to CENTRAL because CENTRAL is already a specialized source containing records selected for potentially relevant study designs. Applying another filter can be unnecessary or harmful.
Watch Out
Do not assume that a filter is beneficial merely because it is available in a database interface. Check what the filter actually does, whether it is appropriate for your evidence type and database, and whether its performance is acceptable for your purpose.
Other designs can be considerably harder to filter reliably
The evidence for methodological filters varies by design. A Cochrane review of strategies for identifying observational studies in MEDLINE and Embase found substantial variation in performance among evaluated filters. Across the filters examined, sensitivity ranged from 48% to 100%, while precision also varied substantially. The review emphasized that the available evidence was limited and heterogeneous.
This illustrates why the label "study-design filter" is not itself a quality guarantee. A filter can improve precision while sacrificing sensitivity, and performance observed in one development set, topic, database, or period may not transfer perfectly to another context.
Qualitative and complex methodological designs pose their own problems
Some evidence types are difficult to identify because methodology may be described through data-collection techniques, analytical approaches, epistemological traditions, or study purposes rather than one consistent design label.
A qualitative study might use terms such as interviews, focus groups, thematic analysis, grounded theory, ethnography, phenomenology, or qualitative research. Yet none of these alone defines the entire eligible universe, and individual terms may also occur in studies that do not meet the review's methodological definition.
Similar difficulties can arise with mixed-methods research, quasi-experiments, natural experiments, implementation studies, and process evaluations.
In such situations, a study-design restriction may require careful development and testing rather than a few intuitive keywords.
The main trade-off is sensitivity versus screening burden
Leaving design out usually retrieves more records. Many will have designs you eventually exclude. That can make screening slower.
Adding a design filter can improve precision by removing records that are unlikely to qualify. But if the filter has imperfect sensitivity, some eligible studies may disappear as well.
| Approach |
Potential benefit |
Potential cost |
| No study-design restriction |
Reduces dependence on design terminology and indexing |
More irrelevant designs may require screening |
| Validated high-sensitivity filter |
Can reduce screening while retaining high retrieval sensitivity |
No filter is automatically appropriate for every database or purpose |
| Ad hoc design keyword block |
Simple to construct and may reduce results |
Unknown performance may cause relevant studies to be missed |
| Built-in database study-type filter |
Convenient |
Its definitions and indexing behavior may not match the eligibility criterion |
The appropriate balance depends on the purpose of the search. A systematic review intended to identify all eligible evidence may tolerate substantial screening to protect sensitivity. A rapid or exploratory search may make different trade-offs, provided those limitations are understood and reported.
Test the restriction against relevant records
If you are considering a study-design block or filter, test it. Identify relevant studies representing the types of records the search should retrieve and examine whether the design restriction retains them.
If several eligible records disappear, determine why. Perhaps their design is described differently, the database indexing is incomplete, or the filter's operational definition does not match yours.
Retrieving known studies does not prove that a filter captures every eligible record, but losing known relevant studies is an immediate warning that deserves investigation. This is closely related to determining whether a search filter is too restrictive.
06 · What This Means for You
Choose Study-Design Restrictions According to Evidence, Not Convenience
Begin with the design required by your review or research question. Then investigate how reliably that design can be identified in the databases you intend to search.
Look for established filters from authoritative methodological sources. Check which database and interface they were designed for, what type of records they target, whether they have been tested, and whether their sensitivity and precision are compatible with your purpose.
A simple decision framework
If a well-tested, high-sensitivity filter exists for the required design and database
Consider using it, while checking that its purpose and performance fit your review.
If design terminology is inconsistent and no suitable filter is available
Consider leaving study design out and determining eligibility during screening.
If you created your own design keyword block
Treat its retrieval performance as uncertain until you test it rather than assuming obvious terminology is comprehensive.
If a design restriction removes known eligible records
Investigate the reason and reconsider the filter, terminology, or decision to restrict by design.
If the database is already specialized or pre-filtered for the relevant study type
Verify whether an additional methodological filter is necessary before applying one.
Whatever you decide, document it. Cochrane's reporting guidance calls for exact database strategies, including limits and filters, to be reported so readers can evaluate and reproduce the search. A methodological filter is part of the search method, not an invisible convenience setting.
