Defining Selection Criteria to Avoid Misrecruits in Usability Studies
Apply clear selection criteria to avoid misrecruits.
Apply clear selection criteria to your next study.
Summary
Selecting the right participants is critical to the validity of usability studies, and this article explains how to define inclusion, exclusion, and diversity criteria beyond simple demographics.
The author argues that misrecruitment leads to costly data noise, as illustrated by a fitness‑tool study where participants were not interested in fitness at all. By establishing clear criteria that align with research goals, researchers can avoid recruiting participants who will not engage with the product.
The article outlines a step‑by‑step process for creating a robust selection plan, including defining target user personas, setting behavioral thresholds, and documenting exclusion rules. It also stresses the importance of testing the criteria with a pilot group to catch any unintended biases. The piece concludes that thoughtful selection saves time, money, and improves the reliability of findings. Researchers are encouraged to adopt these practices for all future studies.
Key changes
- Define inclusion/exclusion criteria beyond demographics
- Align criteria with research goals
- Document exclusion rules
- Test criteria with a pilot group
- Reduce data noise
- Save time and money
- Improve reliability of findings