What Does the Future Hold for Medical Device Usability Testing?

By Nye Canham and Dhanial Salim 

Usability testing is a mainstay in the application of human factors engineering (HFE) to medical device design. However, while regulatory requirements for usability testing keep developing, the application of usability testing in the medical industry has remained fundamentally unchanged since its adoption. 

With the release of generative artificial intelligence (AI) chatbots bringing AI to the forefront of public attention and continued fast-paced technical innovation in the medical industry, current approaches to usability testing may begin to appear antiquated. Furthermore, it brings into question whether traditional usability testing approaches can keep pace and remain effective tools to support the human factors research and design (HFR&D) and clinical evaluation of medical devices. 

While it should be expected that testing the safe use of medical devices is a perennial need, this may require usability testing to evolve or even be replaced by other forms of validation. 

The future of usability testing metrics  

Research into the scientific measurement of usability has identified numerous characteristics such as effectiveness, efficiency, satisfaction, ease of use, ease of learning, ease of understanding, fault-tolerance, attractiveness and safety. However, the ease of measuring the difficulties users have with products in comparison to other usability characteristics has led to a focus on identifying and classifying use errors. Usability testing typically involves the observation of a sample of a target population interacting with a product, with the difficulties they have counted and classified. This approach is buoyed by a narrow view of safety and an assumption that these measures will correlate with other unmeasured characteristics of usability. 

Future developments could open usability testing up to more dynamic approaches. Advancements in eye-tracking tools for usability testing, facial recognition and brain-computer interfaces could enable researchers to gain deeper insights into user behavior and cognitive processes. Imagine a scenario where a user interacts with a digital application, and AI algorithms capture subtle expressions, eye movements and brainwave patterns in real-time. This wealth of data would provide designers with invaluable feedback, allowing them to refine and optimize user interfaces for enhanced usability. By combining human validation with AI-powered analysis, usability testing will become more accurate, efficient and user-centric than ever before. 

Novel simulation technologies in usability testing 

Definitions of usability and requirements for testing tend to include considerations of context of use. In the medical industry, usability testing is often performed in simulated rather than actual use environments. In terms of ensuring the transferability of usability test findings to actual use, it is favorable to incorporate increased realism in a simulated medical environment, which can also enable the discovery of additional usability related findings. 

Simulation is a growing area in healthcare, particularly in the field of medical education, and medical simulation research has begun to explore the use of extended reality technologies. Extended reality can also be used in product evaluation, with applications particularly focused in the early phases of design using virtual prototyping. This use can allow designers to evaluate products in the virtual space before committing to manufacture. Extended reality also has applications to participatory design and collaboration between designers and users.  

In usability testing, developments in extended reality technologies could enable the augmentation of simulated environments with increasingly realistic experiences. Within these augmented environments, a participant’s immersive experience could allow testing with consideration of contextual factors such as work processes and clinical scenarios.  This could extend usability testing to include not only the identification of use errors, but also the evaluation of how a novel product fits into workflows and work environments. 

Automated usability testing 

Traditional usability testing approaches are resource intensive. Human researchers observe participant behavior, document findings, and perform analyses to classify use difficulties and attribute cause. Iterative processes of testing, identifying problems, and revising designs are confined by resource availability, deadlines and cost. 

Capabilities of AI in understanding speech and visuals and generating rules when sifting through data could automate aspects of data collection and analysis. AI and machine learning also have the potential to evaluate the usability of products. Machine learning models can be applied to the evaluation of user interfaces and the measurement of user experience.  

Artificial intelligence has the potential to revolutionize usability testing by automating data analysis and providing predictive design insights. Machine learning algorithms can sift through vast amounts of user data and identify patterns, uncovering usability issues and potential improvements that may have been overlooked by human analysts.  

AI could simulate user behavior, allowing designers to test various design iterations virtually. By employing AI-driven prototypes, designers can rapidly experiment with different interfaces, layouts, and interaction models, reducing the time and cost associated with traditional design cycles. This iterative approach, coupled with real-time analytics, will facilitate rapid innovation, enabling organizations to deliver intuitive and user-friendly products. 

The automation of data capture and analysis is foreseeable, but the enhancement of usability testing with predictive modeling is easier to comprehend than the complete automation of usability testing. This automation would require a substantial investment of resources, and empathy remains key in creating meaningful user experiences. Automation in some areas could free up human researchers and designers to focus their time elsewhere. Empathy allows designers and researchers to understand the emotional journey of users and design products that resonate with their feelings, aspirations, and pain points. By fostering a deep understanding of user perspectives, designers can create empathetic interfaces that establish an emotional connection with users. The future of product evaluation and usability testing could see an increased emphasis on empathetic design practices. Designers can employ qualitative research techniques such as ethnographic studies, interviews, and co-creation workshops to gain rich insights into user needs, desires, and frustrations. These insights will fuel the creation of products that go beyond functional utility, delivering delightful experiences that leave a lasting impression on users. 

Nye Canham is Senior Human Factors Specialist and Dhanial Salim is Human Factors Associate at Emergo by UL.