Manuel B. Garcia

Manuel B. Garcia serves as the Senior Director for Educational Technology and Digital Learning at FEU Institute of Technology, Manila, Philippines. Read More

Contact Info

1607, FEU Tech Building,
P. Paredes St, Sampaloc,
Manila, Philippines
mbgarcia@feutech.edu.ph

Follow Me

The Complete Library

All Research Guides

Browse every published guide in the Researcher's Field Guide, including the complete research journey and supplemental collections, listed alphabetically for quick reference.

Complete Library

Every published guide

Showing 251–300 of 5,065 guides

5,065 guides
PART 0D · SUBPART 69 · GUIDE 946

Human Approval Checkpoints for AI Research Agents

Which Research Tasks Require a Human Approval Checkpoint Before an AI Agent Acts?

AI research agents can automate routine work, but some actions require human authorization before execution. Learn which research tasks warrant approval checkpoints, how to distinguish meaningful oversight from symbolic confirmation, and how to design proportionate controls for data modification, participant communication, external submissions, and scientific decisions.

PART 0D · SUBPART 69 · GUIDE 945

Accountability for Autonomous AI Research Errors

Who Is Responsible for an Error Made by an Autonomous AI Research Agent?

When an autonomous AI research agent makes a mistake, responsibility cannot simply be assigned to the software. Learn how researchers, institutions, developers, and service providers may share different responsibilities, why technical causation differs from accountability, and how research teams can establish defensible oversight and incident-response arrangements.

PART 0D · SUBPART 69 · GUIDE 944

Responding to Irreversible AI Agent Actions

What Should Researchers Do if an AI Agent Performs an Irreversible Action?

An AI research agent may accidentally disclose confidential information, delete important records, submit an unfinished manuscript, or take another action that cannot be fully reversed. Learn how researchers should contain the incident, preserve evidence, assess consequences, meet applicable reporting obligations, and prevent recurrence without concealing what happened.

PART 0D · SUBPART 69 · GUIDE 943

Auditing AI Research Agent Actions

How Do Researchers Audit Actions Performed by an AI Agent?

AI agents can execute multiple research operations without continuous human supervision, making it essential to know what they actually did. Learn how researchers can audit tool calls, file modifications, external transmissions, and approval decisions while distinguishing observable action records from claims about an AI system's internal reasoning.

PART 0D · SUBPART 69 · GUIDE 942

Permissions for AI Research Agents

What Permissions Should an AI Research Agent Be Given?

AI research agents need permissions to retrieve information, execute tools, and complete assigned tasks, but excessive access can compromise research data and institutional systems. Learn how to apply least privilege, distinguish read and write permissions, establish approval checkpoints, and design access controls that support useful automation without surrendering research accountability.

PART 0D · SUBPART 69 · GUIDE 941

AI Research Agents and External Submissions

Should an AI Research Agent Be Allowed to Submit Information to External Systems?

AI research agents can upload files, submit forms, call APIs, and transmit information to external services, but these actions may disclose confidential data or create irreversible commitments. Learn how researchers should classify external submissions, restrict permissions, verify destinations, and establish human approval before consequential actions.

PART 0D · SUBPART 69 · GUIDE 940

AI Agents Contacting Research Participants

Should an AI Agent Be Allowed to Contact Research Participants or Collaborators?

AI agents can draft, personalize, and send research communications, but autonomous contact with participants or collaborators introduces ethical and operational risks. Learn when automated communication may be appropriate, why approved recruitment procedures matter, and how researchers can protect consent, confidentiality, and accountability.

PART 0D · SUBPART 69 · GUIDE 939

Autonomous Statistical Code Execution by AI Agents

Should an AI Agent Be Allowed to Execute Statistical Code Automatically?

AI agents can generate and execute statistical code, but a script that runs successfully is not necessarily scientifically correct. Learn when autonomous execution is appropriate, how to validate code and statistical assumptions, and why sandboxing, audit trails, and human approval remain important for research integrity.

PART 0D · SUBPART 69 · GUIDE 938

AI Agents and Research Data Modification

Should an AI Agent Be Allowed to Modify Research Data?

AI agents can clean, standardize, and transform research datasets, but unrestricted modification can compromise the scientific record. Learn which changes may be appropriate, why original data should be preserved, and how permissions, audit trails, validation, and human approval can distinguish legitimate processing from unacceptable manipulation.

PART 0D · SUBPART 69 · GUIDE 937

End-to-End Autonomous AI Research Workflows

Can an AI Agent Search the Literature, Analyze Data, and Write Results Without Human Intervention?

AI agents can connect literature retrieval, data analysis, and scientific writing into multi-step workflows, but completing every stage does not guarantee valid research. Learn how errors propagate across autonomous research pipelines, why independent verification matters, and when human intervention remains essential.

PART 0D · SUBPART 69 · GUIDE 936

Autonomous AI Agents in Research

Can AI Agents Perform Research Tasks Autonomously?

AI agents can autonomously execute selected research tasks, from retrieving literature to running analytical scripts. However, successful automation does not establish scientific validity or remove researcher accountability. Learn which tasks are suitable for delegation, how to evaluate agent reliability, and when human supervision remains necessary.

PART 0D · SUBPART 69 · GUIDE 935

AI Agents vs. AI Chatbots

What Is an AI Agent, and How Is It Different From an AI Chatbot?

AI agents and chatbots may use the same underlying language models, but they differ in how they pursue goals, use tools, maintain state, and perform actions. Learn what makes an AI system agentic, why autonomy exists on a continuum, and how researchers should evaluate agentic systems without confusing conversational ability with independent action.

PART 0D · SUBPART 68 · GUIDE 934

Prompt Optimization and Analytical Overfitting

When Does Prompt Optimization Become a Form of Analytical Overfitting?

Repeatedly refining AI prompts against the same evaluation examples can produce impressive benchmark scores without improving generalizable performance. Learn when prompt optimization becomes analytical overfitting, how adaptive testing introduces selection bias, and how researchers can protect the independence of their final evaluations.

PART 0D · SUBPART 68 · GUIDE 933

Prompt Optimization and Desired AI Results

Can Researchers Optimize a Prompt Until AI Produces the Result They Want?

Researchers can legitimately improve AI prompts, but repeatedly modifying instructions until a model produces a preferred finding can undermine scientific validity. Learn how to distinguish methodological optimization from outcome-driven manipulation, separate development from evaluation, and report prompt selection transparently.

PART 0D · SUBPART 68 · GUIDE 932

Fresh Conversations in AI Research Testing

Should Each AI Research Test Start in a Fresh Conversation?

Starting each AI research test in a fresh conversation can reduce unwanted context carryover, but it is not always the appropriate experimental design. Learn when conversation isolation is necessary, how persistent memory and system instructions complicate testing, and how researchers can distinguish independent trials from realistic multi-turn interactions.

PART 0D · SUBPART 68 · GUIDE 931

Conversation History and AI Responses

Does Conversation History Affect Generative AI Responses?

Generative AI responses may depend on earlier conversation turns, instructions, examples, and retained context. Learn how conversation history can influence research outcomes, when fresh conversations are appropriate, and how researchers can document multi-turn AI interactions for valid and reproducible evaluation.

PART 0D · SUBPART 68 · GUIDE 930

Prompt Order and Generative AI Responses

Does Prompt Order Affect Generative AI Responses?

Changing the order of instructions, examples, or documents can affect generative AI responses even when the underlying information remains the same. Learn how researchers can identify prompt-order effects, distinguish them from response variability, and design evaluations that account for sequence-dependent performance.

PART 0D · SUBPART 68 · GUIDE 929

Reporting Zero-Shot and Few-Shot Prompting

Zero-Shot vs. Few-Shot Prompting: Which Approach Should Researchers Report?

Zero-shot and few-shot prompting provide different instructional conditions for generative AI. Learn when researchers should report each approach, how examples and their ordering influence performance, and why transparent documentation is essential for fair model comparisons and reproducible AI research.

PART 0D · SUBPART 68 · GUIDE 928

Archiving AI Research Prompt Variants

Should Researchers Archive All Prompt Variants Used in an AI Study?

AI prompt variants can influence research findings even when only one prompt appears in the final methodology. Learn which variants should be archived, how to preserve prompt histories and execution records, and how researchers can balance reproducibility, transparency, confidentiality, and practical documentation requirements.

PART 0D · SUBPART 68 · GUIDE 927

Reporting Failed AI Research Prompts

Should Researchers Report AI Prompts That Failed?

Failed AI prompts can reveal methodological limitations, unstable model behavior, and selection decisions that successful outputs alone cannot explain. Learn which unsuccessful prompts researchers should report, how to distinguish development failures from evaluation failures, and how to document negative results without overwhelming the research paper.

PART 0D · SUBPART 68 · GUIDE 926

Prompt Cherry-Picking in AI Research

What Is Prompt Cherry-Picking in AI Research?

Prompt cherry-picking occurs when researchers selectively report prompts or AI outputs that support a preferred conclusion while concealing relevant alternatives. Learn how this practice differs from legitimate prompt optimization, why it threatens evaluation validity, and how to document prompt selection without compromising scientific transparency.

PART 0D · SUBPART 68 · GUIDE 925

Prompt Engineering Bias in AI Evaluation

Can Prompt Engineering Bias an AI Evaluation Study?

Prompt engineering can improve AI performance, but it can also distort scientific evaluations when prompts are optimized selectively or using final test results. Learn how unequal development effort, leading instructions, benchmark contamination, and selective reporting affect validity, and how researchers can distinguish legitimate optimization from biased evaluation.

PART 0D · SUBPART 68 · GUIDE 924

Pilot Testing AI Research Prompts

Should AI Prompts Be Piloted Before Formal Data Collection or Testing?

Pilot testing can reveal weaknesses in AI research prompts before they affect formal study results. Learn how to design a prompt pilot, assess instruction clarity, identify inconsistent outputs, separate development from evaluation data, and document revisions without introducing prompt overfitting.

PART 0D · SUBPART 68 · GUIDE 923

Prespecifying AI Prompts in Research

Should AI Prompts Be Prespecified in a Research Study?

Prespecifying AI prompts can reduce selective optimization and strengthen confirmatory research, but it does not prohibit pilot testing or exploratory prompt development. Learn when prompts should be finalized, how to document amendments, and why independent evaluation data matter more than simply declaring a prompt fixed.

PART 0D · SUBPART 68 · GUIDE 922

Research Prompts in AI-Assisted Studies

What Is a Research Prompt in an AI-Assisted Study?

A research prompt is more than a question submitted to an AI system. Learn how prompts function within research protocols, influence data collection and analysis, and require documentation of instructions, examples, context, and execution conditions to support scientific transparency.

PART 0D · SUBPART 68 · GUIDE 921

Prompting as a Research Method

Is Prompting Part of the Research Method When Generative AI Is Used?

When generative AI contributes to data collection, analysis, classification, or other research procedures, prompt design can influence the resulting evidence. Learn why prompting may constitute part of the research method, how prompts function as experimental conditions, and what researchers should document to support validity, transparency, and reproducibility.

PART 0D · SUBPART 67 · GUIDE 920

Misleading Human-AI Performance Comparisons

When Is a Human-vs.-AI Performance Comparison Scientifically Misleading?

Human-AI comparisons can produce impressive results while supporting much narrower conclusions than their headlines suggest. Learn how inappropriate human baselines, unequal information, selective reporting, flawed scoring, and unrealistic tasks can distort AI performance claims, and how researchers can design and interpret comparisons more responsibly.

PART 0D · SUBPART 67 · GUIDE 919

AI and Human Expert Interchangeability

Does Similar Average Accuracy Mean AI and Human Experts Are Interchangeable?

AI and human experts may achieve similar average accuracy while differing in error patterns, reliability, judgment, and accountability. Learn how researchers can distinguish numerical similarity from statistical equivalence, evaluate interchangeability, and determine what evidence is needed before recommending AI as a substitute for human expertise.

PART 0D · SUBPART 67 · GUIDE 917

Evaluator Expectations and AI Comparison Bias

Can Evaluator Expectations Bias Human-vs.-AI Performance Comparisons?

Evaluators may judge identical responses differently depending on whether they believe the author is human or AI. Learn how expectations, source labels, response presentation, and perceived expertise can influence human-AI comparisons, and how researchers can measure and reduce these biases.

PART 0D · SUBPART 67 · GUIDE 916

Blinding Evaluators in Human-AI Comparisons

How Do Researchers Blind Evaluators to Whether an Answer Came From AI or a Human?

Knowing whether an answer came from AI or a human can influence how evaluators judge its quality. Learn how researchers can implement blinded scoring, standardize response presentation, conceal source-identifying information, test whether blinding worked, and recognize when complete blinding is impractical.

PART 0D · SUBPART 67 · GUIDE 915

Time Limits in Human-AI Performance Comparisons

Should Humans and AI Have the Same Time Limits During Performance Studies?

Equal time limits may appear fair in human-AI comparisons, but they can distort performance when humans and AI complete tasks differently. Learn when fixed deadlines are justified, how to measure task completion time, and why speed, accuracy, and human verification should be evaluated together.

PART 0D · SUBPART 67 · GUIDE 914

Equal Information in Human-AI Comparisons

Should Humans and AI Receive Exactly the Same Information During a Performance Comparison?

Giving humans and AI identical information may strengthen some performance comparisons, but identical inputs do not always create equivalent conditions. Learn how researchers can establish information parity, distinguish controlled capability testing from realistic workflows, and prevent unequal access to evidence from distorting human-AI performance claims.

PART 0D · SUBPART 67 · GUIDE 913

AI Tool Access in Human Comparisons

Should AI Be Allowed Retrieval, Tools, or External Sources During Human-vs.-AI Comparisons?

Giving AI access to retrieval systems, databases, and computational tools can substantially change its performance in human comparisons. Learn when tool access is justified, how to distinguish model capability from system-level performance, and how researchers can prevent unequal information access from producing misleading superiority claims.

PART 0D · SUBPART 67 · GUIDE 912

Human Tool Access in AI Comparisons

Should Human Participants in AI Comparison Studies Be Allowed the Tools They Normally Use?

Restricting human participants from using their normal professional tools can make AI appear more capable than it would in realistic practice. Learn when tool restrictions are justified, how to distinguish unaided capability from workflow performance, and how to design fair human-AI comparisons without introducing avoidable methodological bias.

PART 0D · SUBPART 67 · GUIDE 911

Experts vs. Novices in AI Comparisons

Experts vs. Novices: Which Is the Fairer Human Comparison for AI?

Comparing AI with experts and novices can produce very different conclusions. Learn when each group provides a fair human baseline, how expertise should be defined, and why studies may benefit from evaluating performance across multiple levels of human competence.

PART 0D · SUBPART 67 · GUIDE 910

Selecting Human Comparators for AI Studies

Who Should Count as the Human Comparator in an AI Performance Study?

The choice of human comparator can fundamentally change the conclusion of an AI performance study. Learn how researchers should select relevant participants, define expertise, determine sample sizes, and establish human performance baselines that support meaningful comparisons rather than exaggerated superiority claims.

PART 0D · SUBPART 67 · GUIDE 909

Fair Human-AI Performance Comparisons

How Do Researchers Fairly Compare AI With Human Researchers or Professionals?

A fair comparison between AI and human researchers requires more than asking both to complete the same questions. Learn how to select appropriate participants, control information access, define comparable tasks, reduce evaluator bias, and analyze performance without favoring either condition.

PART 0D · SUBPART 66 · GUIDE 908

Defining Better-Than-Human AI Performance

What Must “Better Than Human” Actually Mean in a Scientific Claim About AI Performance?

Claims that AI performs better than humans require more than a higher benchmark score. Learn how researchers should define the human comparator, establish meaningful superiority, account for uncertainty, and distinguish task-specific performance from broader claims about professional competence.

PART 0D · SUBPART 66 · GUIDE 907

Measuring Human Oversight in AI Evaluation

Should AI Performance Evaluation Include the Amount of Human Oversight Required?

An AI system may achieve high accuracy while requiring substantial human verification and correction. Learn how researchers can measure oversight requirements, distinguish necessary review from avoidable correction, and evaluate whether AI genuinely improves the quality and efficiency of research workflows.

PART 0D · SUBPART 66 · GUIDE 906

Evaluating AI Cost and Speed

Should AI Performance Evaluation Include Cost and Speed?

An AI system may achieve high accuracy but require substantial time or expense to produce usable results. Learn how researchers can measure latency, throughput, token consumption, total workflow cost, and cost-effectiveness without confusing technical speed with practical efficiency.

PART 0D · SUBPART 66 · GUIDE 905

Subgroup Performance in AI Evaluation

Should AI Performance Evaluation Include Subgroup Performance?

High average AI accuracy can conceal poor performance for particular populations, languages, or task categories. Learn when subgroup analysis is necessary, how to measure performance differences, and how to interpret disparities without confusing statistical variation with evidence of discrimination.

PART 0D · SUBPART 66 · GUIDE 904

AI Response Stability and Reproducibility

Should AI Performance Evaluation Include Reproducibility or Response Stability?

An AI model may achieve high accuracy while producing inconsistent answers to identical questions. Learn how researchers can evaluate response stability, distinguish it from scientific reproducibility, measure variation across repeated trials, and determine when consistency matters for practical AI use.

PART 0D · SUBPART 66 · GUIDE 903

Hallucination Rates in AI Evaluation

Should AI Performance Evaluation Include a Hallucination Rate?

Hallucination rates can reveal factual reliability problems that overall accuracy scores may conceal. Learn how to define hallucinations, select an appropriate denominator, verify generated claims, and distinguish unsupported content from other AI errors.

PART 0D · SUBPART 66 · GUIDE 902

Evaluating AI Reasoning Without Internal Access

How Can Researchers Evaluate AI Reasoning Without Access to the Model’s Internal Reasoning?

Researchers can evaluate the quality of AI reasoning without observing a model's internal computations, but they must distinguish valid answers and explanations from claims about how the model actually arrived at them. Learn how to design reasoning tasks, evaluate observable justifications, and avoid overinterpreting generated chains of thought.

PART 0D · SUBPART 66 · GUIDE 901

Evaluating AI-Generated Codes and Classifications

How Should Researchers Evaluate AI-Generated Codes or Classifications?

AI-generated qualitative codes and classification labels require more than a simple accuracy calculation. Learn how to evaluate coding validity, agreement, category-specific errors, interpretive consistency, and the limitations of comparing AI judgments with human reference labels.

PART 0D · SUBPART 66 · GUIDE 900

Evaluating AI-Generated Summary Quality

How Should Researchers Evaluate the Quality of AI-Generated Summaries?

An AI-generated summary may be fluent and concise while misrepresenting the original research. Learn how to evaluate factual faithfulness, coverage of essential findings, consequential omissions, and practical usefulness using transparent rubrics and appropriate automated measures.

PART 0D · SUBPART 66 · GUIDE 899

Evaluating Partially Correct AI Answers

How Should Researchers Evaluate an AI Answer That Is Only Partly Correct?

Generative AI responses are not always entirely correct or incorrect. Learn how researchers can score partially correct answers, distinguish minor omissions from consequential errors, develop defensible rubrics, and report performance without allowing partial credit to conceal serious inaccuracies.

PART 0D · SUBPART 66 · GUIDE 898

Understanding Generative AI Accuracy

What Does “Accuracy” Mean for a Generative AI Research Task?

Accuracy means different things across generative AI research tasks. Learn when conventional accuracy calculations are appropriate, why open-ended responses require different scoring methods, and how to distinguish factual correctness from completeness, consistency, and usefulness.

PART 0D · SUBPART 66 · GUIDE 897

AI vs. Humans and Existing Software

Should an AI System Be Compared With Humans, Existing Software, or Both?

Comparing AI with humans or existing software answers different research questions. Learn when each comparator is appropriate, why both may be necessary, and how to design evaluations that establish whether AI offers a meaningful advantage over available alternatives.