Can Regulatory Documents Contain Evidence Missing From Journal Articles?

What if the journal article is only one version of the evidence?

A clinical trial may generate far more documentation than the article eventually published about it. For regulated interventions such as medicines and biologics, sponsors may submit extensive information to regulatory authorities, including clinical study reports, protocols, statistical material, analyses, and other documents used during regulatory review.

Some of that information may never appear in a journal article. The differences can concern methodological details, outcomes, numerical results, adverse events, or even entire studies.

That creates an important problem for evidence synthesis: if you rely exclusively on journal publications, are you necessarily evaluating all of the evidence that was available about the intervention?

Yes, regulatory sources can contain evidence absent from publications

Why regulatory evidence can differ from the journal literature

Regulators and journals receive documents for different purposes

A journal article is a scholarly communication designed to report a study within the journal's editorial format. Regulatory submissions serve another purpose: they provide authorities with evidence used to evaluate matters such as the benefits and risks of a regulated product.

The resulting documents can therefore be much more extensive. Clinical study reports, or CSRs, are particularly important. Cochrane describes them as comprehensive documents submitted by pharmaceutical companies as part of regulatory approval applications. They can contain detailed information about study methods and results beyond what appears in conventional publications.

What kinds of missing information might regulatory documents reveal?

The possibilities extend beyond a few extra methodological details. Regulatory sources may help identify studies that are not reported elsewhere, provide fuller descriptions of study methods, contain results for outcomes that are incompletely reported in publications, and offer more extensive information about harms.

Cochrane's current guidance on missing evidence notes that, for some trials, regulatory data may be the only available source of information. It also summarizes research comparing regulatory documents with journal articles in which unfavourable benefit results and adverse events were under-reported in journal publications.

This is one reason the published literature can provide an incomplete picture of the evidence. The problem is not limited to entire missing studies. Evidence can also be missing within a study that was published.

What is inside a clinical study report?

The exact contents vary, but CSRs can be extensive. European Medicines Agency information on clinical-data publication identifies materials such as clinical summaries, reports of individual clinical studies, study protocols, sample case report forms, and documentation of statistical methods among the clinical information associated with regulatory submissions.

This can make a CSR particularly valuable when the journal article leaves uncertainty about protocol details, outcome definitions, analysis populations, adverse events, or other aspects of study conduct and reporting.

There is a practical cost. These documents can be far longer and more complex than journal articles. Finding the relevant result in hundreds or thousands of pages is not quite the same afternoon as reading a six-page paper.

Regulatory reviews are another useful source

You may not always obtain the sponsor's complete CSR. Regulatory agencies can also publish their own scientific or medical reviews, assessment documents, approval histories, and related records.

In the United States, Drugs@FDA includes regulatory history and, for many products, FDA staff reviews evaluating safety and effectiveness. Drug approval packages can contain materials such as summary reviews, medical reviews, statistical reviews, clinical pharmacology reviews, approval letters, and administrative documents.

In Europe, the EMA provides mechanisms for publication of clinical data and access to regulatory documents. The availability of particular documents depends on the product, procedure, date, applicable transparency policies, and possible redactions.

Regulatory documents can reveal entire studies missing from journal searches

Suppose a regulatory application includes several pivotal trials, but your bibliographic search identifies journal publications for only some of them. Regulatory documentation may establish that additional studies were conducted and can sometimes provide their methods and results.

This is closely related to the broader problem of finding negative or null studies that never became journal articles, although you should never assume that an unpublished regulatory study necessarily produced an unfavourable result.

The same trial may tell a different story across sources

Differences do not necessarily mean that one source is fraudulent or that every discrepancy is consequential. Documents may have been produced at different times, analyses may have changed for legitimate reasons, and regulatory reports and journal articles may serve different reporting purposes.

Still, discrepancies should not simply be ignored. Cochrane recommends that reviewers using multiple reports of the same study decide which sources provide the most useful information and have a plan for resolving inconsistencies.

Regulatory evidence is most relevant to regulated interventions

The value of these sources depends heavily on what you are studying. Pharmaceuticals and biologics commonly generate substantial regulatory documentation because evidence is submitted during authorization processes. Regulatory information may also exist for medical devices and other regulated products, but systems and transparency arrangements differ.

For many educational, organizational, social, behavioural, or purely observational research questions, there may be no equivalent regulatory dossier. Searching regulatory agencies simply because regulatory documents can be rich sources would then be a poor use of review resources.

Access is improving, but it is not universal

Researchers should not assume that every regulator publishes the same documents or that complete dossiers are publicly accessible for every product. Availability varies by agency, product, application type, historical period, and transparency regime. Documents may also contain redactions.

For example, FDA states that Drugs@FDA contains most approved drug products dating back to 1939, while the majority of patient information, labels, approval letters, reviews, and other information are available for products approved since 1998. EMA also provides clinical-data publication and access-to-documents mechanisms, but the availability of particular materials depends on the relevant policy and regulatory context.

How regulatory evidence can change what a review knows about a trial

Common mistakes when working with regulatory evidence

Search regulatory sources when they can materially improve the evidence base

The case is strongest when you are conducting a systematic review of medicines, biologics, or another regulated intervention and missing studies, outcomes, or harms could affect the conclusions. Regulatory searching may be particularly valuable when published reports appear incomplete or when you know that a product's development programme included studies that you cannot match to publications.

The extra work can be substantial, so proportionality still matters. A review intended to support consequential clinical or policy decisions may justify deeper regulatory searching than a preliminary narrative overview. The question is not whether regulatory documents are always necessary, but whether excluding them could leave important evidence systematically out of view.

Before relying only on journal reports of regulated interventions

Questions about regulatory documents as research evidence

Do not assume the journal article contains the whole trial

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.