Creating a Realistic Simulated Medical Environment

Conducting a usability test of a medical device is usually not performed in the actual use environment. In fact, controlling the same conditions across all test sessions and replicating an unpredictable medical environment can be challenging. The US Food and Drug Administration (FDA) and European Notified Bodies require manufacturers to describe the intended use environment in their Human Factors Engineering (HFE) report or Usability Engineering (UE) file when testing medical devices.

This post describes steps you can follow to create a realistic simulated environment and immerse the participant in a simulation, while also focusing on evaluating activities in a controlled manner.

Defining the controlled setting

The first step is to think about how to best mimic the actual use environment’s conditions and characteristics in a controlled test setting. After all, regulators expect manufacturers to describe use environments but also to put this into “HFE practice.” Specifically, FDA guidance states that the HF validation test should be designed such that “The test conditions are sufficiently realistic to represent actual conditions of use.” As such, when developing your test protocol, you need to determine what conditions your test environment must meet to ensure you are effectively simulating use of your device in a natural manner.

When defining the test environment, it is important to use the characteristics that impact the user’s interactions with your device, as defined during earlier performed (user) research. For example, you might have determined what level of lighting the room should have to ensure good visibility of your device but also avoid glare on screens. And you might have considered available supplies and equipment that a user in a typical hospital environment has access to, which are also an added value to your test environment.

Another important consideration when designing a usability test (especially for HF validation tests) are distractions. Realistic usability test environments, particularly in some medical settings, should include distractions that are representative of the real world. For example, when working with healthcare professionals, consider that medical environments can be noisy environments, including constant beeping and alarm sounds caused by monitoring devices, or patients and clinical staff having conversations.

Once determined, these characteristics can be used as a guideline for the next step; creating the simulated test environment.

Get creative creating the environment

It can be challenging to represent the complexity of actual clinical situations and the real-world clinical environment in an empty room, but with some creativity, any office can be turned into a simulated operation room. For example:

• Make optimal use of a manikin. There are highly realistic manikins available that allow for actual procedures to be conducted.
• Investigate if you can use a medical training tool to improve your simulation. Those tools are typically used for hospital staff to practice similar procedures during their education but might also help immerse the participant into a realistic scenario.
• When simulating an operating procedure, consider using simulated blood to create a realistic scenario in which some equipment might get dirty.
• Distractions can be simulated by engaging in a conversation with the participant as they are performing a task, similar to how they would have a conversation with a colleague. Another technique that is often used is playing a background noise to help the participant feel immersed in the simulation.
• In some cases, there is also a role for the moderator to play. For example, the participant might ask certain questions they would normally ask a colleague during a procedure. As a moderator, ensure you are prepared to provide the participant with enough guidance so they can continue with their task as they normally would, while also making sure not to bias the participant and provide moderator assistance.

Behind the scenes

Organization is key. Creating a resetting checklist and performing a dry run helps to ensure the test team is prepared and can keep the room setting identical for each session. During a long session, it can also be useful to ask the participant to leave the room and take a break, giving the team the opportunity to prepare the room for the next use scenario without the participant being present. The less preparation the participant sees, the easier it is for them to be immersed in the environment.

Benefits of added realism

Eventually, a realistic testing environment benefits the participant’s experience and might also impact the findings in a positive way. A quite common root cause for usability test findings is test artifact related to the simulated environment. For example, a participant might commit a use error because they did not wash their hands before starting a procedure due to a lack of a sink. By ensuring that there is a sink in the room, similar to their typical work environment, there is limited chance for test artifact. Or, let’s say we are simulating an emergency situation during a surgery. If aspects of the test environment can mimic what would occur during a critical situation that requires immediate action (i.e., a patient’s blood pressure dropping during a surgery), this might impact how the participant responds to the emergency, even if using a manikin as a patient.

Aurelie Debeyre - managing human factors specialist
Lisanne DeVogel - managing human factors specialist