Should You Conduct a Review of Reviews Instead of Another Systematic Review?

Has the Literature Reached the Point Where You Should Review the Reviews?

Suppose you search your topic and discover not just dozens or hundreds of primary studies, but a substantial collection of systematic reviews examining different interventions, populations, outcomes, or related questions. Starting another systematic review of primary studies may seem redundant. Why not synthesize the reviews that already exist?

That is the logic behind a review of reviews, also commonly called an overview of reviews, overview of systematic reviews, or, in some contexts, an umbrella review. Instead of treating individual primary studies as the main unit of inclusion and analysis, you systematically identify and synthesize existing systematic reviews.

The important question is not whether systematic reviews already exist. It is whether the existing review-level evidence can actually answer the question you want to ask. A topic can contain many reviews and still require a new systematic review of primary studies.

Choose the Level of Evidence That Matches Your Question

The Real Difference Is the Unit of Evidence You Need to Examine

A review of reviews systematically synthesizes existing systematic reviews

The terminology is not completely standardized. Publications may use labels such as review of reviews, overview of reviews, overview of systematic reviews, systematic review of reviews, meta-review, or umbrella review, sometimes with methodological distinctions depending on the field.

The central idea, however, is reasonably clear. In an overview of reviews, the systematic review rather than the individual primary study becomes the principal unit of searching, inclusion, and analysis. Cochrane describes overviews as using explicit and systematic methods to identify multiple systematic reviews addressing related questions and to extract and analyze their findings across important outcomes.

This distinction is more consequential than it may first appear. Moving one level upward changes what information you can access, which methodological decisions you inherit, and which questions you can answer.

Reviewing reviews can make sense when the evidence base has already matured

A review of reviews becomes particularly useful when numerous systematic reviews have accumulated across a broad topic and the research objective is to understand that review-level evidence collectively. Instead of asking, for example, whether one narrowly defined intervention improves one outcome, you might want to compare what systematic reviews collectively report across several interventions, outcomes, populations, or related clinical or policy questions.

This is one reason reviews of reviews can be useful for decision makers. They can provide a higher-level synthesis of an evidence base that would otherwise require readers to locate and reconcile numerous systematic reviews independently.

Before choosing that design, however, establish whether the existing reviews are genuinely sufficient. The mere discovery of several review papers does not establish that the field has a usable review-level evidence base. Some may not actually meet a defensible definition of a systematic review. Others may address questions only tangentially related to yours.

Another systematic review is justified when you need the primary studies

Working from existing reviews places a layer between you and the underlying evidence. You depend partly on decisions made by previous review teams: which databases they searched, when they searched them, what they included, which outcomes they extracted, how they assessed risk of bias, and how they synthesized the results.

If your research question requires information that those reviews did not collect or report, the review-of-reviews design may become a poor fit.

A systematic review of primary studies is generally more appropriate when you need to identify newer studies absent from the existing reviews, apply materially different eligibility criteria, extract variables unavailable at review level, conduct analyses requiring study-level information, or independently reassess the primary evidence using methods that existing reviews did not use.

This is closely connected to whether the question has already been reviewed recently enough that another review would add little. Recent coverage may reduce the justification for repeating substantially the same systematic review, but it does not automatically make a review of reviews the correct alternative.

Review quality becomes part of your evidence problem

When you synthesize systematic reviews, weaknesses in those reviews do not disappear simply because they have already undergone synthesis. An overview must therefore consider the methodological quality or risk of bias of the included reviews rather than treating every publication carrying the label “systematic review” as equally trustworthy.

This creates an important dependency. If the relevant reviews are poorly conducted, inadequately reported, badly outdated, or inconsistent in ways that cannot be resolved, synthesizing them may produce a polished overview of an unreliable evidence base.

Overlapping reviews can repeatedly contain the same primary studies

One of the distinctive methodological problems in reviews of reviews is overlap. Two systematic reviews addressing similar questions may contain many of the same primary studies. A third review may contain another partly overlapping set.

If you simply treat all of those reviews as independent bodies of evidence, the same primary study can effectively appear multiple times in your synthesis. Cochrane specifically warns that this can give some primary-study data excessive influence and, when quantitative re-analysis is attempted, inappropriate statistical weight.

Researchers may therefore need to map which primary studies occur in which reviews and decide in advance how overlapping reviews will be handled. Possible strategies depend on the purpose and analysis of the overview. In some circumstances, researchers may include overlapping reviews while preventing duplicate use of outcome data. In others, predefined criteria may be used to prioritize particular reviews.

This issue also explains why a review of reviews should not be regarded merely as a shortcut. You may search fewer primary reports directly, yet inherit a different set of methodological complications.

Outdated reviews can create an evidence gap between the reviews and the present literature

Publication date alone does not tell you whether a systematic review is current. The more informative date is usually when its literature search was last conducted. A recently published review can still rely on a search that ended considerably earlier.

If important primary research has accumulated after the included reviews' searches, a review of reviews may reproduce an outdated picture of the field. In some overview methodologies and contexts, supplemental primary studies may be considered when reviews are incomplete or outdated, but this should be planned explicitly rather than added casually after the review has begun.

If keeping the evidence current requires extensive searching, screening, extraction, and analysis of primary studies, you should reconsider whether you are effectively conducting a new or updated systematic review instead.

A review of reviews can answer a broader question than the individual reviews

An overview does not have to reproduce the precise question of one existing systematic review. Cochrane guidance recognizes that an overview may describe the existing body of systematic review evidence or address a broader question that was not itself the focus of any single included review.

For example, separate systematic reviews might evaluate different approaches to improving student retention in higher education. A review of reviews could ask how the evidence compares across families of interventions rather than repeating each review's narrower question.

This broader perspective is one reason an umbrella review or other review-level synthesis may be attractive. But breadth is useful only when the included reviews are sufficiently compatible for the intended synthesis.

A review of reviews is not the same thing as simply summarizing review conclusions

A rigorous overview should do more than collect the final sentences from several systematic reviews and report whether their authors sounded positive or negative. Conclusions can differ because review teams used different eligibility criteria, outcomes, analytic methods, judgments about certainty, or interpretations of the same evidence.

The PRIOR reporting guideline defines an overview as using explicit and systematic methods to identify and analyze multiple systematic reviews. Cochrane likewise emphasizes extracting and analyzing actual review evidence rather than merely reproducing review authors' conclusions.

In other words, the word review appears twice in “review of reviews,” but the second occurrence does not cancel the methodological work required by the first. Academic nomenclature occasionally has a sense of humor.

The choice is not really about which design is easier

A review of reviews may sometimes save time and resources because much of the primary-study searching and synthesis has already been performed. Cochrane identifies potential efficiency as one reason an overview may be considered for broad questions.

That advantage should not determine the design by itself. If the review-level evidence cannot answer your question, choosing it because it appears more manageable simply moves the problem into the limitations section.

The stronger question is: At what level must I examine the evidence to answer my research question defensibly?

Choosing Between Existing Reviews and the Primary Studies

Notice that the number of existing reviews does not make the decision. Their fitness for the intended question does. Twenty reviews that omit the outcome you need may be less useful than one excellent recent review plus a primary-study search designed around your actual question.

Common Mistakes When Deciding to Review the Reviews

Decide What Evidence You Actually Need Before Choosing the Review Design

Start with the research question rather than the apparent convenience of the design. Then perform enough preliminary searching to understand both layers of the evidence base: the existing systematic reviews and, at least broadly, the primary research that may have appeared since them.

If you are still deciding among several forms of evidence synthesis, first clarify what type of review your question actually requires. A review of reviews solves a particular problem: synthesizing an already developed body of systematic-review evidence. It is not a universal destination for mature topics.

If you choose an overview, plan it as an independent research synthesis rather than an informal summary. That includes specifying eligibility criteria, searching systematically, appraising included reviews, planning how conflicting and overlapping evidence will be handled, and using reporting guidance appropriate to the type of overview. Whether you should develop a protocol before conducting the review should therefore be considered early rather than after screening has begun.

For systematic or scoping evidence syntheses, you should also consider whether preregistration is appropriate for the project and its disciplinary context.

Before Choosing a Review of Reviews, Check the Evidence Base

Questions About Choosing Between Review Levels

Review the Reviews Only When the Reviews Are the Right Evidence

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.