How Do You Separate Empirical Disagreement From Value Disagreement?

What Exactly Are Researchers Disagreeing About?

Two researchers can examine the same evidence and reach different conclusions. Sometimes one of them interprets the empirical evidence differently. Sometimes they agree almost completely about what the evidence shows but disagree about whether the outcome is desirable, acceptable, fair, harmful, or worth the cost.

Those are different disagreements.

The distinction matters because gathering more data can potentially resolve some empirical disputes. It cannot, by itself, determine how much society should value one outcome over another. If a literature review fails to identify which kind of disagreement is occurring, a debate about values can easily be presented as conflicting science, while a genuine empirical dispute can be dismissed as merely ideological.

Ask Whether the Disputed Claim Describes What Is or Judges What Ought to Be

Disaggregate the Argument Before Evaluating the Disagreement

Empirical claims are answerable through observation and evidence

An empirical claim makes a proposition about the observable world. It might concern prevalence, association, causation, prediction, differences between groups, changes over time, experiences reported by participants, or the effects of an intervention.

Examples include claims such as an intervention reduced an outcome, one population reported a particular experience more frequently than another, or exposure to a condition was associated with a later outcome. Whether those claims are supported can be investigated using evidence and methods appropriate to the research question.

Researchers may still disagree about the evidence. They can dispute measurement validity, causal identification, sampling, statistical models, interpretation of qualitative data, generalizability, or the strength of an inference. Those are empirical or methodological disagreements because additional evidence, reanalysis, replication, or stronger research designs could potentially change what is warranted.

Value judgments concern evaluation rather than description alone

A value judgment evaluates a state of affairs according to some standard, priority, ethical commitment, or conception of what matters. Terms such as desirable, unacceptable, equitable, harmful, excessive, worthwhile, or fair may contain evaluative judgments depending on how they are being used.

Suppose a policy reliably increases average academic achievement but also increases an existing disparity between student groups. Researchers could agree completely about both empirical findings and still disagree about whether the policy should be adopted. One may prioritize the average improvement; another may place greater weight on the distribution of benefits.

Another experiment measuring the same two outcomes would not necessarily resolve that disagreement. The dispute concerns how competing outcomes should be valued.

Policy conclusions usually contain both

Statements about what policymakers, institutions, practitioners, or individuals should do commonly combine empirical premises with value judgments.

Consider the argument: “This intervention produces a modest improvement in learning but requires substantially more instructional time, so schools should not adopt it.” The first part contains empirical claims about effects and costs. The recommendation additionally requires a judgment about whether the improvement is worth the additional time and what competing uses of that time matter.

Research can inform those trade-offs by estimating consequences more accurately. It cannot make the evaluative premise disappear.

This decomposition is particularly useful in controversial literatures because researchers can otherwise debate the recommendation while appearing to dispute the underlying evidence.

Look for agreement hidden underneath opposing conclusions

When authors advocate different policies, do not immediately infer that their empirical findings conflict. Compare the factual premises separately from the recommendations.

Two papers might agree about the magnitude of an intervention's benefit and the likelihood of adverse effects but reach different recommendations because they weight those outcomes differently. Calling this “mixed evidence” would be misleading. The empirical evidence may actually be remarkably consistent.

The reverse problem also occurs. Researchers may share similar political values yet disagree about whether a proposed intervention actually produces the outcomes they both want. Shared values do not eliminate an empirical disagreement.

When reviewing politically or socially controversial literature, identify agreement and disagreement at the level of individual claims rather than classifying entire papers into opposing camps.

Values can enter research before the recommendation stage

The distinction between empirical and value disagreement should not be turned into the simplistic claim that science produces entirely value-free facts and values appear only afterward. Philosophers of science have long debated the roles that epistemic and contextual values may play in research, including question selection, evidence assessment, classification, measurement, and decisions about acceptable uncertainty.

A researcher might choose to study one outcome rather than another because it is considered socially important. A construct may contain evaluative assumptions. Decisions about how much evidence is sufficient before acting can depend partly on the consequences of false-positive and false-negative conclusions.

The practical response is transparency rather than pretending those choices do not exist. Ask where a value judgment enters the chain of reasoning and what empirical conclusions remain conditional on it.

Definitions can contain both descriptive and evaluative elements

Some controversies become especially difficult because researchers use concepts that combine description and evaluation. A term may categorize an observable phenomenon while simultaneously implying that it is beneficial, harmful, normal, problematic, equitable, or discriminatory.

In those situations, apparently contradictory findings may partly reflect different conceptual boundaries rather than incompatible observations. Before comparing results, examine whether researchers are using substantively different definitions and what those definitions cause them to measure.

This does not make every definition merely subjective. Definitions can still be evaluated for conceptual clarity, coherence, usefulness, construct validity, and alignment with the research question. It does mean that disagreements over terminology sometimes need to be unpacked before empirical results can be compared.

Methodological criticism should remain methodological

Value disagreement can also leak into judgments about research quality. A researcher may dislike the social implications of a finding and consequently scrutinize its methods more harshly.

That reaction should be separated from legitimate appraisal. If a study has inadequate measurement, serious confounding, inappropriate analysis, or conclusions that exceed its design, explain those problems directly. Ideological disagreement should not substitute for methodological evaluation.

Do not force a clean separation where none exists

The familiar distinction between facts and values is useful analytically, but scholarship in philosophy of science has challenged the idea that the two can always be completely disentangled. Concepts can carry descriptive and evaluative content simultaneously, and value judgments may influence choices about evidence and inference.

For literature reviewing, the important lesson is modest. Do not assume that every statement can be placed effortlessly into a “fact” or “value” box. Instead, decompose the argument as far as reasonably possible and state where empirical evidence ends, where evaluative premises enter, and where the two interact.

Two Researchers Can Agree on the Evidence and Still Disagree

Now change the scenario. Suppose Researcher B instead argues that the apparent improvement disappears after accounting adequately for prior academic performance. That introduces an empirical and methodological disagreement as well. The same debate can therefore contain both kinds at once.

Not Every Disagreement Can Be Settled With More Data

Break Controversial Claims Into Their Components

When a disagreement becomes politically or socially charged, resist summarizing it immediately as “one side says X and the other says Y.” First identify the propositions each side actually accepts and rejects.

This approach can make a polarized literature less dramatic but considerably more informative. Sometimes the apparent scientific battle shrinks once you discover that researchers largely agree about the data and disagree about what society should do with them. Other times, the reverse happens: similar values conceal a genuine dispute about what the evidence supports.

Before Describing a Literature as Divided

Questions About Empirical and Value Disagreement

Do Not Ask Evidence to Resolve a Disagreement It Cannot Resolve

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.