Considerations for Post-HF Validation Design Changes

Imagine you have completed your human factors (HF) validation testing, assessed your study results, and are now developing your residual risk assessments based on the results. It’s at this time perhaps that you realize you might need to consider some design changes for your product. Recognizing that an HF validation test should be conducted with a production-equivalent product, design changes post-validation should not be the norm. However, we recognize for a variety of reasons that changes might be warranted. For example, perhaps you saw a pattern of safety-critical findings in your HF validation testing effort and determined that changes were needed to further mitigate critical risks. Or, maybe you discovered there are typos in the Instructions for Use (IFU) that require correction. Maybe the change is unrelated to your HF validation testing results and a manufacturing change necessitated a product modification. Regardless of the reason, there are a few considerations to make once you determine that a product design change is needed.

The idea of implementing design changes to your product is addressed generally in a few of FDA’s guidance documents, and perhaps most thoroughly in FDA’s “Deciding When to Submit a 510(k) for a Change to an Existing Device” guidance. The focus of this guidance relates mainly to changes made to an existing product (i.e., post-market), but the fundamental guidance and decision-making is similar to considering changes pre-submission. We consider there to be two key questions to ask yourself at this point:

• Does this change affect any critical tasks and/or create new critical tasks?
• Does this change significantly affect or alter the use of the product?

At face value, these might seem like straightforward questions to answer, and certainly align with FDA’s guidance to assess critical tasks in general. However, in practice these questions can lead to nuanced considerations, or a “gray area,” if you will.

To help navigate this gray area we have developed a framework that you can use to help consider any design changes. This framework, presented below, is intended to steer you in the right direction, rather than be a definitive guide of what next steps are needed. When in doubt, we always recommend reaching out to FDA directly to confirm your approach.

To utilize the framework, the first thing you want to ask yourself is if your change is related to a non-critical or critical task. From there, ask yourself about the impact of the change on the use of the product and the associated labeling. Is the impact low, moderate, or high? Once you can answer these questions, that will give you a sense for whether to consider retesting or not.

Consider the following case studies with our framework to help determine an appropriate course of action:

Case study 1 – Rewording a warning statement

• Proposed change: Rewording a warning statement in an insulin pump’s IFU to improve clarity.
• Critical: Yes; the warning is a mitigation related to a critical task.
• Impact: Low; the change affects one warning statement.
• Recommendation: Additional testing likely unnecessary – Although the change relates to a warning message, and therefore a critical task, an argument can likely be made that the modification was a clear and obvious improvement with low impact overall.

Case study 2 – Relocating a hardware button

• Proposed change: Relocating a video system control button on an endoscope due to a manufacturing issue.
• Critical: No; the use of this control is associated with only non-critical tasks.
• Impact: High; users interact with this button fairly frequently.
• Recommendation: Consider additional testing – When considering the impact of this change, think about how far you are moving the button and if this change might impact other key interactions with the endoscope. If you’re moving the button a substantial distance and other interactions might be impacted, then this is likely a high-impact change and evaluating the change with additional testing would likely be best. If other interactions are unaffected, then you can potentially rationalize that additional testing is unnecessary.

Case study 3 – Redesigning packaging

• Proposed change: Redesigning a reconstitution kit’s outer packaging.
• Critical: Yes; the packaging contains multiple mitigations related to several critical tasks.
• Impact: Moderate; all users will interact with the packaging, but only during initial use steps.
• Recommendation: Consider additional testing – If this change results in relocating critical information on the packaging (e.g., expiration date, storage information), then you might want to consider confirming that users can still locate this content with additional testing. If critical content is not impacted, then you could perhaps make an argument that additional testing is not necessary.

Case study 4 – Adding a Quick Reference Guide

• Proposed change: Adding a Quick Reference Guide (QRG) to mitigate use-related issues observed during an HF validation test for a syringe pump.
• Critical: Yes; the QRG includes several critical tasks.
• Impact: High; the QRG describes a key workflow.
• Recommendation: Additional testing likely needed – This change has high impact noting the high number of critical tasks affected, and the fact that this QRG is a new (untested) mitigation for these risks.

Practically speaking, if you want to make a design change without pursuing additional testing, then you need to feel confident making the argument that the design change is either insignificant and/or does not impact critical tasks, or that the change will only improve interactions and does not require additional testing to confirm this assertion. The argument that the change does not impact users’ interactions and/or will only improve their interactions becomes more challenging to make when 1) the change is more significant, 2) there are a substantial number of changes, and/or 3) the change is intended to address a failed mitigation (thereby suggesting this new mitigation might require assessment to verify success).

Laura Birmingham (Associate Research Director) and Stephanie Larsen (Managing Human Factors Specialist) are part of Emergo by UL’s Human Factors Research & Design division. This blog’s content was previously presented as part of their presentation at the 2021 International Symposium on Human Factors and Ergonomics in Health Care, titled Navigating the Gray Area of Post-HF Validation Design Changes: The Impact of Design Changes and How to Assess Them.

Additional human factors engineering (HFE) and usability resources:

• Human factors design and prototype development support
• HFE user research for medical devices, IVDs and combination products