How to Plan Sample Size
Use rough MDE planning before fielding a conjoint or survey experiment.
The Short Version
Use rough MDE planning before fielding a conjoint or survey experiment. A practical way to begin is to estimate available respondents, then choose tasks per respondent. Finish by making sure you budget a pilot and main run separately. The full walkthrough below explains each part in order.
Is This Guide for You?
Researchers budgeting samples for student, online, or national studies.
Follow These Steps
Work through these 5 steps at your own pace. The wording is intentionally practical, and you can return to the checklist at the end when you are ready to review your work.
- Estimate available respondents.
- Choose tasks per respondent.
- Review approximate exposure per level.
- Check the MDE planner.
- Budget a pilot and main run separately.
What This Helps You Accomplish
Underpowered studies waste money and create ambiguous results. Planning helps set realistic expectations before launch.
What a Good Result Looks Like
A sample-size plan connects the smallest effect worth detecting to the design, expected exclusions, repeated tasks, and available recruitment budget. It should state assumptions openly instead of presenting a single calculator result as certainty.
Example: plan backward from usable completes
If the analysis needs 300 usable respondents and you expect 10% of starts to be incomplete or excluded under rules written in advance, recruiting exactly 300 starts is not enough. A planning target near 334 starts gives room for that expected loss. The estimate should be revisited after a pilot reveals actual completion and exclusion rates.
Decisions to Make Before You Begin
- Define the primary effect or comparison before calculating sample size.
- Choose a meaningful minimum detectable effect.
- Account for clustering from repeated tasks within respondents.
- Budget separately for starts, completed responses, and usable responses.
Common Mistakes to Avoid
- Choosing a sample solely because it matches the available budget.
- Ignoring attrition and preregistered exclusions.
- Claiming precise subgroup findings when the study was powered only for an overall effect.
Your Practical Next Step
Use the sample planner for a range of assumptions and retain the conservative scenario in the study plan. Treat the free pilot as information for refining those assumptions.
A Quick Confidence Check
- Plan for exclusions.
- Separate primary and exploratory outcomes.
- Be conservative with small subgroups.
Common Questions
Can I use this guide if I am new to this?
Yes. Researchers budgeting samples for student, online, or national studies. Follow the steps in order, start with a small test, and use the final checklist before you field or report anything important.
What is the simplest way to get started?
Begin by estimate available respondents. Next, choose tasks per respondent. You do not need to perfect every setting before running a small preview or pilot.
How do I know when I am ready?
Use the confidence check above. In particular: Plan for exclusions. Separate primary and exploratory outcomes. Be conservative with small subgroups. When the decision is consequential, keep your study documentation and ask a qualified colleague or methods reviewer to examine the design as well.
Related Guides
Research artifact: sample data, codebook, and scriptsResearch Pathways
Product names and fielding platforms mentioned in this guide belong to their respective owners. Conjoint Survey is an independent academic research tool and is not affiliated with, sponsored by, or endorsed by those services.