How to Add Manipulation Checks
Ask whether respondents noticed or understood the treatment.
The Short Version
Ask whether respondents noticed or understood the treatment. A practical way to begin is to add the treatment as a survey experiment, then add a question immediately after the treatment. Finish by making sure you report the check without replacing the main outcome. The full walkthrough below explains each part in order.
Is This Guide for You?
Researchers using message, video, image, or vignette treatments.
Follow These Steps
Work through these 5 steps at your own pace. The wording is intentionally practical, and you can return to the checklist at the end when you are ready to review your work.
- Add the treatment as a survey experiment.
- Add a question immediately after the treatment.
- Ask about the key treatment feature.
- Use Results to inspect pass rates by arm.
- Report the check without replacing the main outcome.
What This Helps You Accomplish
Manipulation checks help show whether respondents saw or understood the intended difference between conditions.
A Quick Confidence Check
- Do not reveal the hypothesis too early.
- Keep the check simple.
- Decide whether it is diagnostic or exclusionary before fielding.
Common Questions
Can I use this guide if I am new to this?
Yes. Researchers using message, video, image, or vignette treatments. Follow the steps in order, start with a small test, and use the final checklist before you field or report anything important.
What is the simplest way to get started?
Begin by add the treatment as a survey experiment. Next, add a question immediately after the treatment. You do not need to perfect every setting before running a small preview or pilot.
How do I know when I am ready?
Use the confidence check above. In particular: Do not reveal the hypothesis too early. Keep the check simple. Decide whether it is diagnostic or exclusionary before fielding. When the decision is consequential, keep your study documentation and ask a qualified colleague or methods reviewer to examine the design as well.
Related Guides
Research artifact: sample data, codebook, and scriptsResearch Pathways
Product names and fielding platforms mentioned in this guide belong to their respective owners. Conjoint Survey is an independent academic research tool and is not affiliated with, sponsored by, or endorsed by those services.