US FDA, Health Canada and UK MHRA Update on Transparency Guiding Principles for Machine Learning-Enabled Medical Devices

By Sade Sobande

The US Food and Drug Administration (FDA), Health Canada and UK MHRA (June 13) built on their 2021 guiding principles for good machine learning practice for medical device development by further identifying guiding principles for transparency for machine learning-enabled medical devices (MLMDs).

As the healthcare sector continues to experience rapid proliferation of artificial intelligence (AI) and machine learning (ML)-enabled devices, the need to implement guidance that will safeguard the safety, effectiveness and transparency of such devices becomes ever more critical. This is particularly true as concerns the opaque nature of some AI/ML models and the potential for continuous evolution of learning algorithms.

Additional guidance on ML medical devices

These principles build upon principles 7 and 9 of the initial guiding principles:

• Principle 7: Focus is placed on the performance of the human-AI team
• Principle 9: Users are provided with clear, essential information

The principles emphasize a human-centered design (HCD), ISO 9241-210:2019 Ergonomics of human-system interaction - Part 210: Human-centered design for interactive systems, an “approach to systems design and development that aims to make interactive systems more usable by focusing on the use of the system and applying human factors/ergonomics and usability knowledge and techniques”.

Guiding principles

A summary of the guiding principles and relevance to transparency is outlined in the figure below. Because MLMDs can require the user and other stakeholders in the healthcare ecosystem to understand complex information that is often context-dependent, information must be provided that allows the user to identify and evaluate the risk and benefits associated with the device. Information should be provided with the intent of enhancing the audiences’ understanding of the device, its intended use and how it fits into the clinical workflow.

Additionally, information should be provided in simple terms demonstrating how the MLMD processes input information and reaches its output. This allows healthcare professionals and other stakeholders to critically assess this information, its validity and how it is utilized.

Guiding principles, risk management and clinically relevant information

The guiding principles also mention disclosing risk management activities and clinically relevant information to foster trust about the safety and effectiveness of the device. While it is already good practice to disclose clinically relevant information and residual risks, clarity is perhaps needed on the level of detail that must be included. Too much information, particularly on risk management activities which can be considerable and are executed continually through the device lifetime may lead to information overload and distract from critical information that needs to be conveyed to the user.

The placement and timing of information shall also be carefully considered. The user interface shall be optimized to support a personalized, adaptive and reciprocal experience. Targeted information shall also be considered at specific points in the clinical workflow, for example, alerts or warnings during high-risk or time-critical steps.

Relevant audiences for transparency 

• Those who use and/or receive health care with the device
• Those who make decisions about the device to support patient outcomes

Motivation for transparency

• Clear intended use
• Patient-centered care
• Device's risks and benefits
• Safe and effective device use
• Health equity via identification of bias
• Maintenance and ongoing safety performance
• Increased trustworthiness, adoption and access to beneficial technologies

Sharing relevant information

• Clear and accurate description of the device
• Positioning of device within the healthcare workflow
• Impact of output information on healthcare decisions and/or the judgment of a healthcare professional
• Information on risk management activities e.g. management of bias
• Model 'logic'
• Clinically relevant information; limitations, gaps, contraindications
• Maintenance of device safety and effectiveness

Placement of information

• Optimization of the user interface (UI) so information conveyed is responsive to the user
• Adapting the UI to address user needs via a variety of modalities e.g. audio, video, alerts, safeguards, document libraries etc.

Timely communication

• Communication of relevant information throughout the total product lifecycle
• Timely notifications regarding for example, device updates, modifications, new information
• Provision of targeted information e.g. on-screen instructions, warnings

Methods to support transparency

• Employing human-centered design principles; responsive and iterative design, validation, monitoring and communications

Impact

Transparency is a fundamental principle of good machine learning practice (GMLP) and plays a pivotal role in supporting the safety of ML-enabled medical devices. The continued collaboration between US FDA, Health Canada and UK MHRA paves the way for more formal regulation of these devices, as well as global adoption by other regulators.

The guiding principles do more than support adequate medical device labeling. There is an emphasis on device design; usability, risk management, clinical information. These are principles that those in the medical device industry are familiar with but are worth mentioning in the context of MLMDs.

Human-centered design considerations

One of the key concepts is that of human-centered design; this goes beyond the consideration of users of the device to a consideration of other stakeholders involved in the patient healthcare ecosystem e.g. support staff, payers, governing bodies etc. It also requires manufacturers of ML-enabled devices to take a lifecycle approach to confirm users and stakeholders have been considered throughout the design and development process by addressing technical human factors and system interactions. This concept will not be new to medical device manufacturers but is an important one to explicitly be emphasized. Importantly, the guiding principles also require that audiences understand their relevance in design, how information is shared and how it is communicated.  

ISO 9241-210

Interestingly, when discussing the concept of human-centered design, ISO 9241-210:2019 is linked to the guiding principles document. This standard, which is not a medical device standard, focuses on human-centered design activities and principles for computer-based interactive systems. While it is not expected that the standard itself will be wholly adopted, it will be interesting to see whether modern aspects of this are integrated into already modern medical device standards such as IEC 62366, IEC 82304 and ISO 20417.

Concluding remarks

ML-enabled devices are capable of analyzing vast amounts of data to detect patterns and make predictions that would be impossible for humans to accomplish manually. These capabilities have led to the development of devices that can, for example, predict the outbreak of diseases, personalize treatment plans and improve diagnostic accuracy. With these benefits come significant challenges associated with the complexity, safety, effectiveness and trustworthiness of the devices. The guiding principles enumerated by US FDA, Health Canada and UK MHRA are essential to navigating these challenges and promoting the development of safe, trustworthy MLMDs throughout their lifecycle.

Any progress towards global harmonization is very much welcome news. With the challenges posed by MLMDs, global harmonization is increasingly one of the foundational ways to support the safety, performance and trustworthiness of these devices. It is hoped that more regulatory agencies will adopt these guiding principles and continue to engage in the creation of regulations, standards and guidance to support this rapidly evolving field.