Procedures for Conducting In-Person Usability Testing During COVID-19 Pandemic

Like many other human factors professionals, we paused in-person research for a couple of months as COVID-19 gained an increasing foothold on the world. We pivoted to remote- and web-based research, performed various analysis and documentation to make sure that product development could progress during COVID-19.

As of June 1, our team has resumed conducting in-person usability testing our labs and other research facilities and in compliance with local requirements and corporate guidelines. We are doing so following a 14-page manual we developed that outlines how to conduct safe usability testing during the COVID-19 pandemic. With a couple of weeks under our belt, we’d like to highlight key changes we’ve made to our typical approach to protect the health and safety of test participants, personnel and clients.

During test planning and recruitment

• Notify test participants and visitors of COVID-related precautions and procedures. During participant recruitment, as well as during early conversations with our clients, we share details about our COVID-related precautions and procedures. This helps ensure that there are no surprises – and, hopefully, no opposition – when it comes to screening on arrival, donning protective gear, maintaining distance, and taking other measures.
   
• Excluding immunocompromised individuals. We are excluding individuals who are immunocompromised. Although these individuals might very well be the actual users of a given medical device, we will collaborate with our clients to consider proxy patients or delay in-person tests requiring those individuals, who are more susceptible to contracting COVID-19.
   
• Excluding individuals with known COVID exposure. We are excluding individuals who at the time test, treat, and/or diagnose COVID-19 patients – or individuals who live with someone who does. This exclusion serves two purposes – primarily, to protect those involved in testing from an increased potential to COVID-19 exposure and, secondarily, to not pull these first responders and frontline workers from their essential role in fighting this pandemic.
   
• Eliminating physical participant paperwork. Instead of having participants review paper copies of consent forms on-site, we email a version of the forms, answer questions via telephone, and collect signatures digitally.
   
• Completing a “go/no-go” checklist. We developed a “go/no-go” checklist to help us confirm it’s appropriate to proceed with conducting a given usability test during the COVID-19 pandemic. The checklist accounts for such factors as confirming local regulations allow such work, our client has reviewed and approved our in-person testing precautions, and an Investigational Review Board has reviewed and approved the protocol and precautions. The checklist must be signed by the project lead and one of our team’s Research Directors.

During test conduct

• Administering daily health screening to test participants, personnel, and visitors. All test participants, personnel, and any essential visitors (e.g., trainers) undergo a daily health screening that involves a temperature check and review of symptoms associated with COVID-19. Anyone with a fever (temperature over 100.4°F per the CDC) or one or more symptoms are not permitted to enter the test facility.
   
• Wearing protective equipment. All test participants, personnel, and visitors are required to don a new, Emergo-provided mask and gloves immediately upon arriving to, and before entering, the test facility. A mask may only be removed if needed to briefly use a device (e.g., breathe in from a single-use inhaler). Gloves may be worn throughout a test session, or may be removed and replaced with frequent hand sanitizer usage if ungloved device is most representative.
   
• Maintaining proper “social distance.” Everyone must maintain at least 6 feet (2 meters) of distance from others while on-site. If the test requires individuals to be closer for a period of time, participants and personnel will be informed prior to the test and additional precautions (e.g., plexiglass barriers, face shields) will be used to limit exposure.
   
• Moderating from the observation room. To enable us to maintain distance, when feasible, test personnel conduct the session from within the observation room. With this setup, test personnel sit at least 6 feet apart from each other and turn on the observation room lights to enable the participant – seated on the other side of a one-way mirror in the test room – to see the researchers. We use additional video cameras and/or microphones as needed to ensure a clear view of participant-device interactions and clear two-way audio exchange.
   
• Cleaning and disinfecting test spaces and supplies. At the start and end of the day, and after each test session, we clean and disinfect all spaces used for testing (work surfaces, chairs, door handles, etc.), as well as all supplies handled by test participants and personnel. We work with our client to determine the most effective and safe way to clean devices and other test items that must be reused versus discarded.

After test conduct

As you might expect, most of the precautions come into play during test planning, participant recruitment, and test conduct. However, to close the loop, we implemented a follow-up health screening to all test participants, personnel, and visitors. Everyone who attended an in-person usability test must undergo follow-up health screenings 1, 7, and 14 days after participating. The screening seeks to identify whether any individual has developed a fever or any other symptoms associated with COVID-19.

Conclusion

As the COVID-19 pandemic further evolves, we will revisit and revise our procedures, ensuring that we are implementing the right precautions at the right times, and prioritizing the health and safety of everyone involved. As a first step, though, the current procedure seems appropriately rigorous, if not conservative, which feels right for our risk-focused team.

It goes without saying (but I’ll say it!) that we all look forward to a time when we return to performing our work – and living our personal lives – as we did before COVID-19. But, in the meantime, we are confident in our procedures and excited that we have resumed conducting these important human factors engineering evaluations in-person during this “new normal.”

Allison Strochlic is Research Director at Emergo by UL's Human Factors Research & Design division.