Use Errors Part 2: Estimating the Severity of Potential Harms

In a previous blog post, I wrote about the challenge of estimating the likelihood of a use error. The main take-aways were that the estimates are not likely to be particularly accurate and that risk management efforts need to be driven largely by an assessment of the harm that can arise from a use error. I then previewed this blog, which speaks to the challenges of estimating severity and why it requires a systematic approach. I pointed out that a needlestick injury could result in harms ranging from pain, to minor bleeding, to infection, to amputation, to sepsis, and ultimately to death. So, where do you draw the line, so to speak, in a severity rating exercise?

To the best of my knowledge, there is no clear answer. The impression I have drawn from discussions held at human factors conferences and among fellow human factors engineering (HFE) practitioners is that the “rules of engagement” vary widely. This is not good news. It means that companies need to figure things out for themselves, wondering if they are being too liberal or conservative when assigning severity of harm ratings.

Varying approaches to severity of harm ratings

One of my clients that develops a medical device used to perform a lumbar puncture (i.e., spinal tap) treated practically every use error as catastrophic because it could introduce contaminants into the spinal fluid, which could then cause bacterial meningitis. Another client that develops a cardiac therapy device estimated low levels of harm due to use errors because they assumed medical interventions would arrest the propagation of harm. Here you can see diametrically opposed approaches, and neither is necessarily right or wrong. Although I advise a conservative approach, which tends to raise severity of harm ratings, I understand why being too conservative can create an obstacle to bringing safe and important products to market.

Distinguishing cascading events from initial use errors

Accordingly, you might steer toward a middle ground. This means, do not look at every use error as the start of a long cascade of events leading to death. Rather, consider that each cascading event probably has an even lower likelihood, and that the chance of a catastrophic outcome is virtually nil. Keep in mind that we are talking about the cascading events, not the initial use error. Here is an example of cascading events, related to the aforementioned needlestick injury, that could begin with a seemingly minor harm and subsequently lead to death.

1. Needlestick injury occurred
2. Needle was contaminated by bacteria
3. Wound became infected
4. Delayed medical attention to local infection
5. Local infection not responsive to antibiotics
6. Surgical procedure to address local infection ineffective
7. Infection became systemic (i.e., sepsis)
8. Treatment for sepsis was ineffective

You can see how a needlestick injury that might usually be inconsequential could, following a series of unfortunate events, lead to death.

Some companies choose to truncate the series of subsequent events at two, which might seem arbitrary. But, if you consider the joint probabilities of such events, the chances of things progressing to worse is extremely low. That is why you do not hear of people dying from needlestick injuries, except in the case of there being transmission of a blood-borne pathogen (e.g., TB, HIV). But, if a use scenario did expose a user to such pathogens, these events would likely occur only as a result of two or more subsequent failures.

Formal procedures for use error risk estimations

The key is for companies to have a procedure that dictates how far to go with your severity of harms analysis. If the procedure says to account for a series of two cascading failures starting with the use error, then so be it. This is a better approach than having development teams decide for themselves how far to go with the analysis and there being inconsistency. Accordingly, I suggest that having a use error risk estimation procedure that details how to determine severity of harm ratings is part of a good quality system.

---

Readers familiar with James Reason’s swiss cheese model of human error can apply the same general concept to a severity of harms analysis. One can imagine a series of unfortunate events that link a use error (needlestick) to death. The question becomes whether a company’s approach will consider the worst-case scenario (e.g., cascading failures resulting in death) or more likely scenarios that play out after the initial use error (e.g., patient receives appropriate medical attention).

Adapted graphic source.

Michael Wiklund, CHFP, P.E., is General Manager of Human Factors Research & Design at Emergo by UL.

Learn more about human factors research and design issues for medical products:

• Human factors engineering (HFE) research for medical devices, IVDs and combination products
• HFE design and prototype development support