Self-selection Study Tips, Part 1: Recruiting for a Successful Study

The US Federal Drug Administration (FDA) defines a self-selection study as an evaluation that “assesses the ability of consumers to apply drug labeling information to their personal health situation to make correct decisions about whether or not it is appropriate for them to use a drug product.” Self-selection studies typically call for users to interact with a product and its labeling and determine their individual suitability to use the product (i.e., make a “self-selection” decision) based on demographic, health, and/or other factors. In this two-part blog series, we’ll share some key insights about running self-selection studies to set your study up for success.

The FDA guidance specifically refers to self-selection studies for non-prescription drug products, though we know that they have also requested self-selection data for other types of products, including prescription drug products, prescription and over-the-counter medical devices, and smartphone apps that include medical features. The tips below can be applied to conducting a self-selection study of any of these products.

The first step to any successful study is ensuring you have an appropriate number of unbiased participants with suitable characteristics for your study. Below, we share some of our top tips for recruiting participants for a self-selection study.

Determining sample size

The FDA guidance recommends that the self-selection study sample size be based on clinical rationale, results from pilot studies, and statistical significance, which can result in quite large sample sizes that might go into the hundreds. Although larger sample sizes might be ideal in some cases, the FDA has accepted self-selection data with sample sizes on a similar scale as a typical human factors (HF) validation study (i.e., 25-50 participants, including both appropriate and inappropriate users). Ultimately, the sample size should be justifiably sufficient to “provide a reliable answer to the primary objective;” that is, obtain a reliable sample of self-selection decisions.

Considering participant inclusion/exclusion criteria

Your participant sample should include both appropriate and inappropriate users of the product to enable the study data to demonstrate that (1) appropriate users can recognize that they can use the product and also that (2) inappropriate users – those who should not use the product – can recognize that they are not intended users. It’s important to consider the various demographic and other criteria that might make users appropriate versus inappropriate and evaluate those criteria by recruiting participants with a range of those characteristics. It might also be important to recruit individuals who meet one criteria but do not meet another. For example, if the product is appropriate for those who are ages 22 and older and have diabetes, recruit individuals over 22 with diabetes, under 22 with diabetes, over 22 without diabetes, and under 22 without diabetes. While recruiting for every permutation might not be realistic, include a representative and fairly broad range to allow for a thorough evaluation. Criteria that we often see include factors such as age, the presence or absence of a specific medical condition, whether someone has an implantable device such as a pacemaker, and whether someone is pregnant.

Avoiding bias in recruitment

It is also important to consider how the participant recruitment process might bias users to consider themselves an appropriate or an inappropriate user. For example, if participants know they were recruited because they’ve been diagnosed with a specific disease, they might make assumptions later during the study session about their eligibility to use the product that wouldn’t reflect actual use. Or, if a recruited inappropriate user sees the product’s intended use, they might assume they should pretend or falsely report that they are an appropriate user, which could compromise the data. To combat this potential bias in the recruitment phase, we share only limited, high-level information about the product (i.e., do not specify the exact indication or focus). Additionally, we ask potential participants distractor questions during the screening process. For example, we may ask the prospective participant if they have several different medical conditions/treatments, rather than solely the one of interest, so that respondents cannot determine the “right” answers. If we are looking to recruit people with growth hormone disorder, we might present a list of several medical conditions and simply ask “Which, if any, of the following conditions do you have?” rather than “Do you have growth hormone disorder?” which is arguably too specific and could bias respondents’ answers.

Juliana Amundson is Senior Human Factors Specialist at Emergo by UL’s Human Factors Research & Design division.