How Do You Handle Borderline Studies Without Making Arbitrary Decisions?

What should you do when a study almost meets your criteria?

Some screening decisions are easy. The wrong population, an obviously ineligible study design, or a completely unrelated intervention may justify a straightforward exclusion. Borderline studies are different.

Perhaps the participants overlap your age criterion but extend slightly beyond it. An intervention has most, but not all, of the features in your operational definition. A mixed sample contains eligible participants but results are only partly separable. The setting is described ambiguously. Two reviewers can read the same methods section and reasonably reach different conclusions.

The danger is not simply making the "wrong" decision. It is making different decisions for essentially equivalent studies because each case was judged intuitively. Borderline cases therefore need a process that converts ambiguity into explicit, reproducible rules.

Resolve the ambiguity systematically rather than intuitively

A borderline study usually exposes an ambiguity somewhere

First identify what makes the study borderline

"Maybe" is a useful warning signal, but it is not a methodological diagnosis. Ask which specific criterion cannot be resolved.

Is the population partly eligible? Is the intervention poorly described? Does the study design sit between two categories? Is the outcome definition ambiguous? Is an eligible subgroup mixed with an ineligible sample? Is the necessary information simply missing?

Once the problem is named, it becomes easier to determine whether you need more information, a clarification of the criterion, or reviewer adjudication.

Uncertainty about the evidence is different from uncertainty about the rule

This distinction is particularly useful.

If information is missing, seek better evidence. If the rule is ambiguous, repeatedly rereading the paper will not solve the problem. You need to clarify the decision rule itself.

Do not exclude because the abstract leaves you uncertain

Title and abstract screening is an intentionally compressed stage. Many details required for eligibility are absent from abstracts.

If the available information cannot establish that a study is ineligible, moving it forward for fuller assessment is generally safer than guessing. The purpose of full-text screening is to resolve precisely these uncertainties. The same principle applies when an abstract does not provide enough information to decide.

This does not mean every remotely related record should survive initial screening. When an exclusion criterion is clearly met, exclude it. The principle applies when genuine uncertainty remains about whether the record could satisfy the criteria.

Operational definitions prevent many borderline decisions

Eligibility criteria become difficult to apply when they rely on labels without defining what those labels mean.

Suppose your review includes "interactive digital learning." Does watching a recorded lecture count? Does a multiple-choice quiz count? What about an adaptive platform that responds to student performance?

If the protocol never operationalized "interactive," every new technology can become a philosophical seminar in miniature. The solution is not to debate each paper independently. Define the characteristics an intervention must have to qualify, then apply that definition consistently.

Cochrane recommends specifying eligibility criteria in sufficient detail to determine which studies will be included and advises review authors to define important characteristics prospectively. JBI similarly places explicit inclusion criteria at the center of study selection.

Clarification is not necessarily the same as changing the criteria

A borderline case may reveal that a criterion needs clarification rather than substantive alteration.

Suppose your protocol includes "undergraduate students" but does not state how to handle mixed samples containing undergraduates and postgraduates. Establishing a consistent rule for mixed samples may clarify how the original population criterion will be operationalized.

By contrast, deciding halfway through screening that postgraduate students should now be eligible in their own right changes the scope of the population.

The distinction is not always perfectly sharp, so document what changed, why it changed, and which records might be affected. If the modification alters eligibility materially, treat it as a protocol amendment and consider whether earlier records need rescreening. This is why changing inclusion criteria after screening begins requires more care than quietly editing a screening note.

Apply precedent across comparable cases

Once a borderline case produces a defensible interpretation, preserve that decision as a screening rule or decision note.

Suppose reviewers agree that studies combining undergraduate and postgraduate participants are eligible only when undergraduate results can be extracted separately. When the next mixed-population paper appears, the team should not reopen the entire debate unless the new case exposes a materially different issue.

This creates consistency across screening decisions and reduces drift as reviewers encounter hundreds or thousands of records.

A decision log can be particularly useful. It might record the criterion involved, the ambiguous case, the interpretation adopted, the rationale, the date, and whether previously screened records require reconsideration.

Independent reviewers help expose ambiguity

Disagreement is not always evidence that one reviewer performed poorly. Sometimes it reveals that an eligibility criterion can reasonably be interpreted in more than one way.

JBI systematic-review guidance recommends that study selection be conducted independently by two or more reviewers, with disagreements resolved through discussion or a further reviewer. PRISMA 2020 requires systematic-review authors to report how many reviewers screened records and reports, whether they worked independently, and any relevant automation used in the process.

The value of multiple reviewers is therefore not merely arithmetic agreement. Disagreements can expose assumptions that one reviewer did not realize they were making.

Use adjudication to resolve the rule, not merely the paper

When two reviewers disagree, a third reviewer should not simply cast a mysterious deciding vote. First identify why the judgments differ.

One reviewer may interpret "community-dwelling older adults" differently from another. One may count an intervention as blended learning while another considers it ordinary online instruction. Resolving the underlying interpretation produces a rule that can guide subsequent cases.

The decision becomes more useful when the team can say, "For this review, we will treat X as meeting criterion Y when conditions A and B are present," rather than merely recording that Paper 37 was included after discussion.

Do not let perceived importance turn an ineligible study into a borderline one

A famous, highly cited, recent, or methodologically impressive study can create psychological pressure during screening. Reviewers may begin searching for reasons to retain it even when the criteria are clear.

If a paper unambiguously fails a required criterion, it is not borderline merely because excluding it feels consequential. The appropriate question is whether the criterion itself is defensible for the review. If it is, apply it. If it is not, reconsider the criterion transparently for all studies rather than creating an exception.

The same issue arises when an important study does not satisfy the original eligibility criteria.

Record uncertainty rather than erasing it

Not every ambiguity can be resolved perfectly. Authors may not respond to requests for clarification. Reports may omit crucial methodological details. Several publications may describe the same study inconsistently.

When residual uncertainty matters, document it. Depending on your methodology and protocol, the appropriate response may involve adjudication, author contact, a conservative application of the eligibility rule, or another predefined procedure.

What should be avoided is invisible improvisation. A reader should not have to reverse-engineer why two nearly identical studies received opposite decisions.

Turning one borderline case into a reusable decision rule

Common mistakes when studies sit near the eligibility boundary

Build a small adjudication system before borderline cases accumulate

You do not need a bureaucracy worthy of a multinational trial for every screening project. You do need a way to keep one ambiguous paper from being treated differently from the next.

During a large screening exercise, it can also be useful to temporarily flag unresolved records so they are not forced into an immediate binary decision. Whether and how to maintain such a category is considered in recording "maybe" studies separately during screening.

For systematic reviews, document final full-text exclusions using explicit reasons. PRISMA 2020 requires reporting excluded reports that might appear to meet the inclusion criteria together with the reasons for exclusion. A consistent decision log can make that reporting considerably easier and can support the broader practice of tracking why studies were excluded.

Before deciding a borderline study, check this

Questions about borderline studies during screening

Borderline cases need consistent rules, not case-by-case intuition

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.