Self-selection Study Tips Part 2: Conducting Self-selection Studies

By Juliana Amundson and Michael Romero

As we mentioned in Part 1 of this series, a self-selection study assesses the ability of a participant to determine whether they are an appropriate or inappropriate use of a product. While the US Food and Drug Administration (FDA) has established guidelines for conducting self-selection studies for non-prescription drug products, FDA also requests these studies for other types of products, including prescription drug products, prescription and over-the-counter medical devices, and smartphone apps that include medical features.  

Part 1 of our 2-part series addressed recruiting participants for self-selection studies. But what about actually conducting the study session? On the surface, it seems like you could ask a basic “Yes” or “No” question, such as “Do you think you should use this product?” But our experiences have shown that it can be a bit trickier to elicit a realistic answer and avoid bias to this question in the self-selection process than you might expect.  

Below, we share two key considerations for how and when to pose the critical self-selection question to your participant during a study session. 

Eliciting the self-selection decision 

The specific question(s) you ask to elicit the participant’s self-selection decision during the session might depend on what the workflow looks like – is there a point in the product use workflow that prompts the user to confirm they are eligible via the software user interface, or is the intended use and user information simply stated on the packaging and the user must notice it on their own? If the latter, it might make sense to ask, “Do you think you should use this product?” to avoid drawing unrealistic attention to the intended user information. But if the product itself “sorts” the user, as might be the case with a mobile app or software where the user answers questions to determine whether they qualify, consider asking the question more directly “(e.g., “Do you consider yourself eligible to use this product?”) after the participant has input their self-selection information and been notified whether they are eligible. Either way, the question should get at the participant’s understanding of the product’s intended user eligibility criteria. 

Avoiding Bias in Self-Selection Studies 

Like traditional human factors (HF) validation testing, the goal is to simulate a realistic scenario, but certain limitations make self-selection studies slightly more difficult to simulate realistically. In real life, the user might make their self-selection decision subconsciously. Specifically, there would not be a third party explicitly asking them whether or not they consider themselves an appropriate user. Asking this very direct question during a self-selection study is necessary but inherently introduces some test artifacts in that, by asking the question, you are focusing the participant somewhat unrealistically on their eligibility to use a product, which might elicit bias in the participant’s self-selected response and product interactions. 

The best point during the session at which to ask the self-selection question depends highly on the product use workflow. It is best to ask the question precisely at the “turning point” of when someone should decide whether to proceed with using the product. If you ask this question too early in the product setup or preparation process, the participant might pay more attention to the self-selection information included in the labelling than they might have in actual use, predisposing the sample user to bias in the self-selection study and to explore the question unrealistically. If you ask the question too late, the participant might assume that they should set up the product for the sake of the test session, regardless of whether they are an appropriate user, which might confuse them further and compromise the data. It’s a fine balance that requires the moderator to have a nuanced understanding of the product’s use workflow and when users would naturally make the self-selection decision. 

Juliana Amundson and MIchael Rothstein are Senior Human Factors Specialists at Emergo by UL.