Manuel B. Garcia

Manuel B. Garcia serves as the Senior Director for Educational Technology and Digital Learning at FEU Institute of Technology, Manila, Philippines. Read More

Contact Info

1607, FEU Tech Building,
P. Paredes St, Sampaloc,
Manila, Philippines
mbgarcia@feutech.edu.ph

Follow Me

The Complete Library

All Research Guides

Browse every published guide in the Researcher's Field Guide, including the complete research journey and supplemental collections, listed alphabetically for quick reference.

Complete Library

Every published guide

Showing 3,901–3,950 of 5,065 guides

5,065 guides
PART 02 · SUBPART 23 · GUIDE 405

Can Too Many Quotations Hide Weak Interpretation?

Can Too Many Quotations Hide Weak Interpretation?

Participant quotations should support qualitative analysis, not replace it. A results section can contain abundant participant voice yet remain analytically weak if researchers do little with the evidence.

PART 02 · SUBPART 23 · GUIDE 404

Can Too Few Quotations Weaken a Qualitative Paper?

Can Too Few Participant Quotations Weaken a Qualitative Paper?

Too few participant quotations can make qualitative findings difficult to evaluate, but there is no universal minimum. The real question is whether readers can inspect the evidence behind the analysis.

PART 02 · SUBPART 23 · GUIDE 403

Are the Themes Grounded in the Data?

How Do You Know Whether Themes Are Grounded in the Data?

A convincing qualitative theme should have a defensible relationship to the underlying data while adding interpretation rather than merely repeating quotations. Learn what evidence to look for.

PART 02 · SUBPART 23 · GUIDE 402

Was the Qualitative Analysis Systematic Enough?

How Do You Judge Whether Qualitative Analysis Was Systematic Enough?

Systematic qualitative analysis does not require one universal coding procedure. It requires a coherent, transparent process that shows how researchers moved from raw data to defensible interpretations.

PART 02 · SUBPART 23 · GUIDE 401

Was Qualitative Data Collection Deep Enough?

How Do You Judge Whether Qualitative Data Collection Was Deep Enough?

Qualitative data collection is deep enough when it generates sufficiently detailed, specific, and contextually meaningful evidence for the research question. Duration alone cannot tell you whether that happened.

PART 02 · SUBPART 23 · GUIDE 400

Should Qualitative Research Be Representative?

Should You Expect Qualitative Research to Be Representative?

Qualitative studies usually do not aim to produce statistically representative samples. Their sampling should instead be judged against the research question, the cases selected, the depth of inquiry, and the claims the researchers make.

PART 02 · SUBPART 23 · GUIDE 399

Was the Qualitative Sample Adequate?

How Do You Judge Whether the Qualitative Sample Was Adequate?

There is no universal number of participants that makes a qualitative sample adequate. Sample adequacy depends on how much relevant information the sample provides for the research question, methodology, and intended analysis.

PART 02 · SUBPART 23 · GUIDE 397

What Makes Qualitative Research Convincing?

What Makes a Qualitative Study Convincing?

A convincing qualitative study does more than present interesting themes and quotations. Its claims should be supported by an appropriate design, relevant participants, sufficiently rich data, systematic analysis, reflexivity, and a transparent connection between evidence and interpretation.

PART 02 · SUBPART 22 · GUIDE 396

When to Seek Statistical Expertise

When Should You Seek Statistical Expertise Before Relying Heavily on a Study?

Not every paper requires consultation with a statistician. Seek appropriate statistical expertise when unresolved analytical questions are central to the study's conclusions, difficult to evaluate confidently, or consequential for how you will use the evidence.

PART 02 · SUBPART 22 · GUIDE 394

Was Statistical Adjustment Appropriate?

How Do You Judge Whether Statistical Adjustment Was Appropriate?

An adjusted analysis is not automatically better than an unadjusted one. Judge whether the variables controlled for were appropriate for the study design and target effect, how they were selected, when they were measured, and whether adjustment could itself introduce bias.

PART 02 · SUBPART 22 · GUIDE 393

Recognizing Overfitting in Prediction Models

How Do You Recognize Overfitting in a Prediction Model Paper?

An overfitted prediction model learns features of its development data that do not generalize well to new individuals. Warning signs include excessive model complexity, inadequate information, weak validation, and performance reported only on the data used to build the model.

PART 02 · SUBPART 22 · GUIDE 392

Should You Trust a Post Hoc Subgroup Finding?

Should You Trust a Subgroup Finding That Was Not Prespecified?

A subgroup finding that was not prespecified is not automatically false, but it usually deserves more caution. Judge the interaction itself, the number of subgroup analyses performed, biological or theoretical plausibility, effect magnitude, and independent corroboration.

PART 02 · SUBPART 22 · GUIDE 391

Recognizing Selective Reporting

How Do You Recognize Selective Reporting of Significant Analyses?

Selective reporting occurs when which results are reported, emphasized, or fully presented depends on what the analyses found. Look beyond the published P-values by comparing the paper with protocols, registrations, analysis plans, and other study records.

PART 02 · SUBPART 22 · GUIDE 390

When a Paper Reports Many Statistical Tests

What Should You Look for When a Paper Reports Many Statistical Tests?

When a paper reports many statistical tests, individual P-values cannot always be interpreted in isolation. Examine why so many tests were performed, which analyses were primary, whether they were prespecified, and how multiplicity affected the conclusions.

PART 02 · SUBPART 22 · GUIDE 389

Can Large Samples Make Trivial Effects Look Impressive?

Can Very Large Samples Make Trivial Effects Look Impressive?

Very large samples can produce tiny P-values for effects that are too small to matter. The statistical result may be correct, but its importance must be judged from effect magnitude, uncertainty, and substantive consequences.

PART 02 · SUBPART 22 · GUIDE 388

How to Recognize an Underpowered Study

How Do You Recognize an Underpowered Study From the Published Paper?

You cannot reliably identify an underpowered study from sample size or a nonsignificant P-value alone. Examine what effect the study was designed to detect, whether it achieved its planned information size, and how precisely the observed effect was estimated.

PART 02 · SUBPART 22 · GUIDE 387

Can a Nonsignificant Result Be Informative?

Can a Nonsignificant Result Still Be Informative?

A nonsignificant result does not automatically mean there is no effect. It can still be informative when you examine the estimated effect, uncertainty, precision, and which substantively meaningful effects the data can reasonably exclude.

PART 02 · SUBPART 22 · GUIDE 386

Can Statistical Significance Be Unimportant?

Can a Statistically Significant Result Still Be Unimportant?

A statistically significant result can be too small to matter in practice. Statistical significance concerns evidence relative to a statistical model, while importance depends on effect magnitude, uncertainty, context, and consequences.

PART 02 · SUBPART 22 · GUIDE 385

P-Values, Effect Sizes, or Confidence Intervals?

Should You Focus on P-Values, Effect Sizes, Confidence Intervals, or All Three?

P-values, effect sizes, and confidence intervals answer different questions. Good interpretation usually emphasizes the estimated effect and its uncertainty while using the P-value as supplementary evidence rather than the entire conclusion.

PART 02 · SUBPART 22 · GUIDE 384

Judging a Result You Cannot Reproduce

How Do You Judge a Statistical Result You Cannot Reproduce?

Being unable to reproduce a statistical result does not automatically make it wrong. Judge why reproduction failed, what evidence remains independently assessable, and how much your conclusion depends on the unverified analysis.

PART 02 · SUBPART 22 · GUIDE 383

Do You Need to Recalculate Statistics?

Do You Need to Recalculate Statistics to Critically Appraise a Paper?

Critical appraisal usually does not require reproducing every statistical calculation in a paper. The more important task is judging whether the methods, assumptions, reported estimates, uncertainty, and conclusions make sense together.

PART 02 · SUBPART 21 · GUIDE 382

How Much Should Measurement Quality Matter?

How Much Should Measurement Quality Affect the Weight You Give a Study?

Measurement problems should not automatically disqualify a study, nor should they be treated as minor footnotes. Their importance depends on which variables are affected, how severe the problem is, and how much the study’s conclusions depend on those measurements.

PART 02 · SUBPART 21 · GUIDE 381

Can Operational Definitions Explain Disagreement?

Can Different Operational Definitions Explain Why Studies Disagree?

Two studies can use the same construct label while measuring meaningfully different things. Before treating their findings as contradictory, compare how each study translated the construct into observable variables.

PART 02 · SUBPART 21 · GUIDE 380

Can an Objective Measure Still Be Poor?

Can an Objective Measure Still Be a Poor Measure of the Construct?

Objective measurement can reduce some forms of judgment and reporting error, but it does not guarantee that the resulting variable represents the intended construct. Precision and construct validity answer different questions.

PART 02 · SUBPART 21 · GUIDE 379

When Does Social Desirability Bias Matter?

When Does Social Desirability Bias Threaten a Study’s Findings?

Social desirability bias matters when respondents systematically alter answers to appear more acceptable or avoid negative consequences. Its importance depends on the topic, setting, perceived privacy, and conclusions drawn from the reports.

PART 02 · SUBPART 21 · GUIDE 378

When Does Recall Bias Threaten a Study?

When Does Recall Bias Threaten a Study’s Findings?

Imperfect memory does not automatically create recall bias. The greater concern is whether errors in remembering or reporting past information differ systematically in a way that can distort the study’s comparisons or associations.

PART 02 · SUBPART 21 · GUIDE 377

Can Self-Reported Measures Be Appropriate?

Can Self-Reported Measures Be Appropriate?

Self-report is sometimes the most appropriate way to measure a construct, particularly when the construct concerns experiences only the participant can report. Its quality depends on what is being measured and how the questions are designed and administered.

PART 02 · SUBPART 21 · GUIDE 376

Was the Validated Instrument Modified?

Was the Instrument Modified From the Validated Version?

Changing a validated instrument does not automatically make it invalid, but previous validation evidence may no longer apply unchanged. The consequences depend on what was modified and how the scores are used.

PART 02 · SUBPART 21 · GUIDE 375

Was the Instrument Used in the Same Language?

Was the Instrument Used in the Same Language?

A translated instrument is not necessarily equivalent to the version originally validated. Evaluate how it was translated, adapted, tested, and whether its scores retain the intended meaning.

PART 02 · SUBPART 21 · GUIDE 374

Was the Instrument Validated in the Same Population?

Was the Instrument Validated in the Same Population?

Evidence supporting a measurement instrument in one population may not transfer unchanged to another. The key is whether population differences could alter what the instrument measures or how its scores should be interpreted.

PART 02 · SUBPART 21 · GUIDE 372

How to Evaluate an Unfamiliar Measure

How Do You Evaluate a Measure You Have Never Seen Before?

An unfamiliar measure should not be accepted or rejected merely because you do not recognize its name. Evaluate what it measures, how it was developed, what evidence supports it, and whether that evidence fits its current use.

PART 02 · SUBPART 21 · GUIDE 371

Did the Study Measure What It Claims?

Did the Study Measure What It Claims to Have Measured?

A study can be methodologically sophisticated yet still produce weak evidence if its measures do not adequately represent the constructs it claims to study. Learn how to examine that link critically.

PART 02 · SUBPART 20 · GUIDE 370

When Should You Avoid Generalizing?

When Should You Refuse to Generalize Beyond the Population Actually Studied?

Some findings can reasonably extend beyond the people directly studied, but others should not. Learn when differences in population, setting, selection, measurement, or effect modifiers make broader generalization scientifically difficult to defend.

PART 02 · SUBPART 20 · GUIDE 369

Can Findings From One Institution Apply Elsewhere?

When Can Findings From One Institution Reasonably Apply Elsewhere?

A single-institution study can provide strong evidence, but the institution may also shape who participates, how an intervention is delivered, and what outcomes occur. Generalization requires separating the finding from site-specific conditions.

PART 02 · SUBPART 20 · GUIDE 368

Can Findings From One Country Apply Elsewhere?

When Can Findings From One Country Reasonably Apply Elsewhere?

A study conducted in one country is not automatically local evidence only, nor does it automatically apply worldwide. Generalization depends on whether differences between contexts matter to the finding being transported.

PART 02 · SUBPART 20 · GUIDE 366

Can Low Participation Rates Limit Generalizability?

Can Low Participation Rates Undermine Generalizability?

A low participation rate can threaten generalizability when participants differ from nonparticipants in ways that matter to the study findings. The rate itself, however, cannot tell you how biased a result is.

PART 02 · SUBPART 20 · GUIDE 364

Were Important Groups Excluded?

Were Important Groups Excluded From the Study?

Not every study must include every possible group, but exclusions become consequential when missing populations matter to the research question or conclusions. Learn how to identify and evaluate those gaps.

PART 02 · SUBPART 20 · GUIDE 362

When Does Selection Bias Threaten Findings?

When Does Selection Bias Seriously Threaten a Study’s Findings?

Selection becomes a serious threat when entering or remaining in a study depends on factors that can distort the result being estimated. Learn how to distinguish ordinary sample differences from consequential selection bias.

PART 02 · SUBPART 20 · GUIDE 361

Can a Convenience Sample Be Useful?

Can a Convenience Sample Still Provide Useful Evidence?

Convenience sampling is methodologically limited, but that does not make every study using it worthless. Its value depends on the research question, selection process, and claims researchers make from the resulting evidence.

PART 02 · SUBPART 20 · GUIDE 360

Does Every Study Need a Representative Sample?

Does a Representative Sample Matter for Every Research Question?

Not every research question requires a sample that mirrors a wider population. Whether representativeness matters depends on what researchers are trying to estimate, explain, test, or generalize.

PART 02 · SUBPART 19 · GUIDE 358

Useful Evidence Despite Serious Limitations

Can a Study Have Serious Limitations but Still Provide Useful Evidence?

Serious methodological limitations can substantially reduce confidence without making every finding useless. The key is to determine exactly which claims the limitations threaten and what narrower contribution may remain defensible.

PART 02 · SUBPART 19 · GUIDE 357

No Fatal Flaw but Weak Evidence

Can a Study Have No Fatal Flaw but Still Provide Very Weak Evidence?

A study can avoid any single fatal methodological flaw and still provide very weak evidence. Evidential strength depends on the combined consequences of design limitations, bias, precision, measurement, directness, and the claims being made.