Manuel B. Garcia

Manuel B. Garcia serves as the Senior Director for Educational Technology and Digital Learning at FEU Institute of Technology, Manila, Philippines. Read More

Contact Info

1607, FEU Tech Building,
P. Paredes St, Sampaloc,
Manila, Philippines
mbgarcia@feutech.edu.ph

Follow Me

Can Economic Data Collection Increase Participant Burden and Affect Study Quality?

Economic evaluations may require participants to report expenses, time use, healthcare utilization, and other resource consequences. These measurements can introduce additional burden, potentially affecting completion, retention, and data quality. Learn how to identify unnecessary demands and design proportionate economic data collection without overlooking important costs.

1328
Economic Data Collection and Participant Burden Guide 1328 of 1361
01 · The Question

Can Collecting More Economic Information Make a Research Study Less Reliable?

A research team evaluates an intervention and plans to conduct an economic evaluation alongside the effectiveness study. Participants already complete baseline assessments, outcome questionnaires, and follow-up visits. To estimate costs, the researchers add monthly expenditure diaries, resource-use questionnaires, travel records, and questions about employment and unpaid caregiving.

Each measurement appears relevant to the economic analysis. However, some participants find it difficult to remember minor expenses, others regard financial questions as intrusive, and several begin leaving sections incomplete.

The researchers face a methodological dilemma. Collecting less information may omit important costs, but collecting too much may increase burden and undermine the completeness or accuracy of the resulting data.

How should researchers balance the information requirements of an economic evaluation with participants' time, capabilities, and willingness to continue providing data?

02 · The Short Answer

Economic Data Collection Can Increase Burden, but More Measurement Is Not Always Better

In Brief

Yes. Economic data collection can increase participant burden through repeated questionnaires, expenditure diaries, recall tasks, and questions about personal finances or resource use. These demands may contribute to incomplete responses, measurement error, nonresponse, or attrition, potentially affecting the validity and precision of economic estimates. Researchers should collect information proportionate to the economic question and use appropriate measurement methods, schedules, and existing records where feasible.

However, shorter instruments and fewer assessments do not automatically produce better data. Reducing measurement may omit consequential resource categories or increase recall error. The appropriate approach balances coverage, accuracy, participant acceptability, feasibility, and the potential consequences of missing information.

03 · What You Need to Know

Understanding How Economic Measurement Affects Participants and Research Quality

What Is Participant Burden in Economic Data Collection?

Participant burden refers to the demands associated with providing information required by a research study. In economic evaluation, these demands may involve remembering expenditures, estimating time use, consulting financial records, completing diaries, or reporting service utilization.

Burden may be practical, cognitive, emotional, or financial. A participant might spend considerable time completing a questionnaire, struggle to distinguish routine expenses from intervention-related costs, or feel uncomfortable disclosing household financial circumstances.

These demands vary across populations. An instrument that is straightforward for one group may be difficult for participants with limited literacy, disabilities, language barriers, or competing responsibilities.

Researchers should therefore consider both the amount of information requested and the effort required to provide it accurately.

Why Can Economic Evaluation Require Additional Data Collection?

Effectiveness studies usually measure outcomes related to the intervention's intended benefits or harms. Economic evaluations additionally require information about resources consumed and, where relevant, costs borne by different stakeholders.

For example, a clinical intervention may influence healthcare visits, medication use, transportation expenses, and time away from employment. An educational intervention may require additional connectivity, equipment, instructor support, or participant time.

Some information can be obtained from organizational records. Other resource categories may be observable only to participants.

The question is not whether economic information should be collected at all, but which resource-use data are necessary for the intended evaluation.

Which Economic Measurement Methods May Create Burden?

Measurement approach Potential burden Important methodological consideration
Repeated resource-use questionnaires Time and repeated recall demands Assessment frequency and completeness
Daily expenditure diaries Continuous recording and recordkeeping Diary fatigue and declining adherence
Employment questionnaires Sensitive information and difficult productivity estimates Privacy, validity, and valuation assumptions
Travel and time-use logs Recording trips, waiting, and activities Attribution and recall accuracy
Administrative record linkage Potentially less direct reporting Consent, governance, coverage, and linkage quality
Interviews about expenses Scheduling and discussion of personal circumstances Interviewer effects and participant comfort

No method is universally least burdensome. A brief questionnaire may be easier than a daily diary for some participants, while frequent recording may be preferable when events are difficult to remember retrospectively.

How Can Repeated Economic Questionnaires Affect Completion?

Repeated assessments require participants to sustain attention and cooperation throughout the observation period.

When questionnaires repeatedly ask about numerous minor expenses or activities, participants may begin skipping questions, providing approximate answers, or discontinuing reporting.

However, these outcomes should not be assumed to occur uniformly. Completion may depend on questionnaire design, participant circumstances, study support, and the perceived relevance of the information.

Researchers should monitor response patterns rather than infer participant fatigue solely from the number of questionnaires administered.

The broader concern about repeated measurement and participant fatigue applies to economic data collection as well.

Can Economic Data Collection Increase Missing Data?

Yes. Participants may omit individual cost items, fail to return questionnaires, or withdraw from economic follow-up.

Missingness may be particularly consequential when it is related to resource consumption or participant circumstances.

For example, participants who experience substantial illness-related expenses may also find repeated questionnaires difficult to complete. If their resource use is systematically underrepresented, economic estimates may be biased.

Researchers should distinguish item nonresponse from complete questionnaire nonresponse and participant attrition.

They should also avoid assuming that a blank response means that no resource was used. An explicit zero and an unanswered question represent different information.

How Does Missing Economic Data Affect Cost-Effectiveness Analysis?

Incomplete cost information may reduce precision or introduce bias into estimates of mean costs and incremental economic consequences.

Cost data are often skewed because a relatively small number of participants may incur substantial expenditure. Missing observations among high-resource users can therefore be especially consequential.

Complete-case analysis may be inappropriate when participants with complete economic data differ systematically from those with missing observations.

Methods such as multiple imputation may be considered under suitable assumptions, but they cannot guarantee recovery of unobserved costs.

Researchers should investigate patterns of missingness, collect relevant auxiliary variables when feasible, and consider sensitivity analyses for assumptions that cannot be verified directly.

Can Longer Recall Periods Reduce Burden Without Reducing Accuracy?

Longer recall periods reduce the number of assessment contacts but may make certain resource-use events harder to remember.

For example, participants may recall a major hospital admission several months later but struggle to reconstruct numerous small transportation or connectivity purchases.

Shorter recall periods may improve reporting of some events while increasing the frequency of questionnaires.

The appropriate interval depends on event frequency, salience, the availability of records, and the measurement instrument.

Researchers should test recall periods rather than assume that either frequent or infrequent reporting is always superior.

Are Prospective Cost Diaries More Accurate Than Retrospective Questionnaires?

Prospective diaries can reduce reliance on long-term memory by allowing participants to record resource use close to the time it occurs.

However, diaries depend on sustained adherence. Participants may forget entries, record events retrospectively, or stop using the diary.

Retrospective questionnaires may be less demanding when events are infrequent or documented elsewhere.

Evidence comparing diary and recall methods should be interpreted in relation to the resource category and study population.

Researchers should not treat prospective recording as automatically error-free or retrospective reporting as inherently unreliable.

Can Economic Questions Create Emotional or Privacy-Related Burden?

Yes. Questions about income, employment, debt, household expenditure, or financial hardship may be sensitive.

Participants may be reluctant to disclose information they consider private, particularly when it appears unrelated to the intervention.

Researchers should explain why information is needed and avoid requesting more detail than the economic analysis requires.

Data collection procedures should include appropriate confidentiality, access controls, and applicable ethics and data protection safeguards.

Participants should not be pressured to disclose unnecessary financial information merely because the research team wishes to construct a comprehensive dataset.

Can Burden Differ Across Participant Groups?

Yes. The same instrument may require different levels of effort depending on literacy, language, disability, digital access, employment arrangements, and caregiving responsibilities.

For example, a digital expenditure diary may be convenient for participants with reliable internet access but difficult for those who depend on intermittent connectivity.

Similarly, a questionnaire based on formal employment may fail to capture unpaid work or irregular livelihood arrangements.

Researchers should consider how to make research materials accessible to participants with different literacy and disability needs.

Accessibility accommodations should preserve the intended measurement construct while reducing avoidable barriers.

Can Measurement Burden Threaten Representativeness?

Potentially. Participants with limited time, financial resources, digital access, or other constraints may be less likely to complete demanding economic assessments.

If these characteristics are associated with costs or intervention outcomes, the resulting economic sample may differ from the population the study intends to represent.

Researchers should examine whether economic data completeness varies across relevant participant characteristics.

However, differences in completion do not automatically establish selection bias. The implications depend on the analytical estimand, missing-data mechanism, and available information.

The relationship between participant burden, attrition, and research validity should be considered when designing economic follow-up.

Should Researchers Collect Every Potential Cost Category?

No. Economic measurement should follow the analytical perspective, comparator, time horizon, and decision problem.

Some resource categories may be unlikely to differ between alternatives or contribute materially to the economic conclusion.

However, omitting a category merely because it is difficult to measure can be problematic when it represents a substantial economic consequence.

Researchers should prioritize categories according to expected relevance, magnitude, variability, and availability of reliable data.

Exclusions should be justified and reported transparently.

Can Administrative Records Reduce Participant Burden?

Administrative and financial records may reduce the need for participants to report information already captured routinely.

For example, institutional records may provide software expenditure, service utilization, or reimbursement information.

However, these records may omit privately paid expenses, informal care, unpaid time, or resources not recorded by the organization.

Researchers should assess completeness, definitions, observation periods, and whether recorded charges represent the economic quantities required.

Record linkage may also require additional permissions and governance arrangements.

Replacing questionnaires with administrative records should improve the data collection process only when the records are appropriate for the intended analysis.

Can Digital Tools Make Economic Reporting Easier?

Digital tools may support convenient recording through mobile forms, reminders, and automated data capture.

For example, participants may record transportation expenses immediately after an intervention visit rather than reconstruct them months later.

However, digital tools can introduce barriers involving connectivity, device compatibility, accessibility, privacy, or technical confidence.

Researchers should evaluate whether the selected platform is suitable for the target population and whether alternative reporting methods are necessary.

Digitalization should not be confused with automatic burden reduction.

How Can Researchers Determine Which Economic Questions Are Essential?

A useful approach is to map each questionnaire item or data field to a resource category and an intended analytical use.

For example, a question about transportation expenditure may be necessary when participant travel costs differ between interventions. Detailed questions about unrelated household purchases may not be justified.

Researchers should identify whether each item contributes to the primary analysis, a prespecified secondary analysis, or an exploratory objective.

Items without a defensible analytical purpose should be reconsidered.

The aim is to maximize useful research information while minimizing unnecessary participant burden.

How Can Cognitive Interviewing Improve Economic Questionnaires?

Cognitive interviewing can help researchers investigate how participants understand questions, retrieve information, make judgments, and select responses.

For example, participants may interpret a question about travel costs as referring only to public transportation fares, while researchers intend to include parking and fuel expenditure.

Such misunderstandings may create measurement error even when questionnaires are completed fully.

Pretesting can reveal ambiguous wording, unsuitable recall periods, and categories that participants cannot reasonably estimate.

Researchers should revise instruments based on evidence about comprehension rather than assuming that a shorter questionnaire is necessarily clearer.

Should Economic Data Collection Be Piloted?

Pilot testing is particularly useful when economic measurement involves unfamiliar instruments, repeated diaries, complex recall tasks, or populations with diverse access needs.

A pilot may assess completion time, item nonresponse, reporting accuracy, acceptability, and the feasibility of obtaining administrative records.

Researchers should distinguish feasibility outcomes from definitive evidence about economic effectiveness.

A small pilot may reveal practical difficulties but may not provide precise estimates of resource-use variability or missingness in the main study.

Findings should inform proportionate revisions to the collection procedures.

Can Reducing Economic Measurement Compromise Study Quality?

Yes. Removing questions or lengthening recall periods may reduce burden while omitting consequential costs or increasing measurement error.

For example, eliminating questions about participant travel may be inappropriate when the interventions differ substantially in attendance requirements.

Researchers should distinguish redundant measurements from information necessary for valid economic comparison.

The relevant trade-off concerns the expected quality of the economic estimate, not questionnaire length alone.

Where a resource category is essential, researchers should consider less demanding ways of measuring it before deciding to exclude it.

How Should Economic Data Collection Be Scheduled?

Economic assessments should be scheduled according to when relevant resources are consumed and how reliably they can be recalled.

One-time implementation costs may be obtained from organizational records, while participant expenses may require reporting at selected follow-up intervals.

Researchers should avoid collecting information at every visit when less frequent measurement would provide sufficiently accurate data.

Conversely, important short-lived events may require more frequent recording.

The measurement schedule should be integrated with the broader protocol to avoid unnecessary additional contacts.

How Should Researchers Evaluate Whether Burden Reduction Worked?

Researchers can compare completion time, item nonresponse, questionnaire return rates, participant feedback, and the completeness of important resource categories before and after revising the measurement process.

Where feasible, alternative collection approaches may be compared directly.

However, higher completion rates do not necessarily establish greater measurement accuracy.

A revised instrument may be easier to complete but less comprehensive or less sensitive to relevant resource use.

Evaluation should therefore consider both participant experience and the quality of the economic information obtained.

How Should Burden and Missingness Be Reported?

Reports should describe the economic instruments, assessment schedule, data sources, and relevant completion patterns.

Researchers should identify substantial missing resource categories and explain how incomplete observations were handled.

Where collection procedures were changed during the study, consequential modifications should be documented.

The CHEERS 2022 reporting guidance supports transparent description of economic evaluation methods and uncertainty, while appropriate trial and feasibility reporting guidance may address participation and data completeness.

Researchers should avoid implying that economic estimates are complete merely because the effectiveness dataset has high retention.

Watch Out

Do not assume that more economic measurements automatically produce more reliable estimates. Excessive reporting demands may increase missingness and reduce participation, while indiscriminate shortening may omit important costs. The objective is to obtain sufficiently complete, accurate, and relevant economic information through proportionate procedures.

04 · A Practical Example

Reducing Economic Data Collection Burden in a University Intervention Study

Hypothetical Example

Revising a Student Resource-Use Questionnaire

A university evaluates a semester-long digital academic support intervention involving 300 students. The economic analysis considers institutional costs and selected participant expenses, particularly connectivity and transportation.

The initial protocol requires students to complete a 30-minute economic questionnaire every two weeks. During pilot testing, participants report difficulty recalling minor expenses and identifying which costs are attributable to the intervention.

The research team considers a shorter questionnaire administered monthly, supplemented by institutional records of software expenditure and technical support. All figures below are hypothetical.

Measurement feature Initial approach Revised approach
Questionnaire frequency Every two weeks, 8 assessments Monthly, 4 assessments
Estimated completion time 30 minutes per assessment 12 minutes per assessment
Total estimated reporting time 240 minutes 48 minutes
Institutional software costs Participant questionnaire Financial records
Participant travel and connectivity Detailed repeated reporting Focused questions with defined recall periods
Illustrative Burden Calculation
Total questionnaire time = Number of assessments × Minutes per assessment
This calculation measures estimated questionnaire completion time only. It does not include time retrieving records, responding to reminders, or other participation demands.
Initial: 8 × 30 = 240 minutes. Revised: 4 × 12 = 48 minutes. Estimated reduction: 192 minutes, or 80% of the initial questionnaire completion time.
Step 1: Define the economic question. The team identifies which institutional and participant resources are necessary for the planned analysis.
Step 2: Review item relevance. Questions unrelated to the intervention or comparator are removed, while consequential cost categories are retained.
Step 3: Identify alternative data sources. Institutional expenditure is obtained from appropriate financial records instead of asking students to report it.
Step 4: Test recall and comprehension. Cognitive interviews examine whether monthly reporting captures transportation and connectivity expenses adequately.
Step 5: Evaluate data completeness. Pilot completion rates, item nonresponse, and discrepancies with available records are examined.
Step 6: Finalize the measurement plan. The revised approach is adopted only if its lower burden does not materially compromise the information needed for the economic evaluation.

The hypothetical revision substantially reduces questionnaire completion time. However, the 80% reduction in time does not establish an 80% improvement in participant experience or data quality.

The team must still determine whether the longer recall period introduces important omissions and whether the revised instrument captures relevant resource use consistently.

The example illustrates why burden reduction should be evaluated against measurement quality rather than judged solely by the number of questions removed.

05 · What Researchers Often Get Wrong

Common Misconceptions About Economic Data Collection Burden

Misconception

Does Collecting More Cost Data Always Improve Economic Accuracy?

No. Additional measurements may improve coverage, but they can also increase reporting demands, nonresponse, and measurement error. The value of each measurement depends on its relevance and quality.

Misconception

Are Shorter Questionnaires Always More Accurate?

No. Shorter instruments may reduce completion demands but can omit important resources or use categories too broad for reliable reporting.

Misconception

Are Prospective Cost Diaries Free From Recall Problems?

No. Participants may delay entries, forget events, or discontinue reporting. Prospective recording reduces some recall demands but does not guarantee complete data.

Misconception

Does a Missing Cost Response Mean Zero Expenditure?

No. Missing information and an explicitly reported zero are distinct. Treating unanswered questions as zero may bias economic estimates.

Misconception

Can Administrative Records Replace Every Participant Questionnaire?

No. Administrative records may omit private expenses, informal care, or time costs. Their suitability must be assessed for each resource category.

Misconception

Does High Effectiveness-Outcome Retention Guarantee Complete Economic Data?

No. Participants may complete clinical or educational outcomes while leaving resource-use questionnaires incomplete. Economic data completeness requires separate assessment.

06 · What This Means for You

How to Design Economic Data Collection That Participants Can Complete Reliably

Begin with the economic analysis plan and identify which resource categories require direct participant reporting. Evaluate whether existing records can provide some information and whether the proposed questionnaire schedule is proportionate to the expected value of the data.

Pretest the instruments with people resembling the intended participants, then monitor completeness and burden during the study.

A simple decision framework

If economic information is available from reliable existing records
Consider using those records rather than requiring participants to report the same information.
If resource events are frequent and difficult to recall
Consider shorter recall intervals or proportionate prospective recording, subject to feasibility testing.
If financial questions are sensitive
Explain their purpose, minimize unnecessary detail, and apply appropriate confidentiality safeguards.
If questionnaire completion is declining
Investigate comprehension, burden, scheduling, and accessibility before modifying the instrument.
If reducing measurement would omit consequential costs
Explore alternative data sources or measurement approaches rather than excluding the category without justification.

The preferred design is one that produces sufficiently reliable economic evidence while respecting participants' time and circumstances.

07 · A Quick Checklist

Check Whether Economic Data Collection Is Proportionate and Reliable

Before finalizing economic measurement, check:
Map each economic questionnaire item to a defined resource category and analytical purpose.
Identify which resources can be obtained from suitable administrative or financial records.
Estimate questionnaire completion time and additional recordkeeping demands.
Test recall periods and item comprehension with representative participants.
Provide accessible and appropriate reporting arrangements for different participant needs.
Distinguish explicit zero resource use from missing responses.
Monitor economic item nonresponse, questionnaire completion, and participant attrition separately.
Prespecify appropriate missing-data and sensitivity analyses.
Document consequential measurement changes and evaluate their implications for data quality.
08 · Frequently Asked Questions

Frequently Asked Questions About Economic Measurement Burden

Can economic questionnaires increase participant attrition?

They may contribute to attrition when reporting demands become difficult to sustain. The relationship should be evaluated rather than assumed, because withdrawal can have multiple causes.

Are cost diaries better than retrospective questionnaires?

Neither method is universally superior. Diaries may reduce recall demands but require sustained adherence, while retrospective questionnaires may be practical for memorable or well-documented events.

How often should resource-use questionnaires be administered?

The interval should reflect event frequency, recall accuracy, data availability, and participant burden. Pilot testing may help determine an appropriate schedule.

Can missing cost data be treated as zero?

No. An unanswered question does not establish that no resource was consumed. Missingness should be identified and handled using appropriate methods.

Can administrative records reduce economic measurement burden?

Yes, when they provide suitable and sufficiently complete information. They may not capture privately incurred expenses or informal resource use.

Should researchers remove sensitive financial questions?

Questions should be retained only when justified by the research objective. Where sensitive information is necessary, appropriate explanations and confidentiality safeguards are important.

Does reducing questionnaire length always improve study quality?

No. Lower burden may improve feasibility, but removing consequential information can weaken economic estimates. Both measurement quality and completion should be evaluated.

Can economic data be incomplete even when trial retention is high?

Yes. Participants may remain in the effectiveness study while omitting individual resource-use questions or entire economic questionnaires.

09 · The Bottom Line

Economic Measurement Should Be Comprehensive Enough Without Becoming Unnecessarily Demanding

The Bottom Line

Economic data collection can increase participant burden and affect study quality when reporting demands contribute to incomplete, inaccurate, or systematically missing information. Researchers should prioritize economically relevant resources and select collection methods that balance accuracy, coverage, feasibility, and participant acceptability.

Reducing burden is valuable when it preserves the information required for credible economic conclusions. The goal is not the shortest possible questionnaire, but a proportionate measurement strategy that participants can complete reliably.

10 · Sources and Further Reading

Authoritative Sources on Economic Data Collection and Research Burden

11 · Cite this Guide

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.

Has the Field Guide helped your research?

If a guide helped clarify a question, inform a research decision, or move your work forward, I would love to hear about your experience. Your story may also help other researchers discover the Field Guide.

Share Your Experience
Takes only a few minutes