How do you audit for compliance with ISO 14971:2012?

In Internal Auditing, ISO 14971, Risk Analysis, Risk Management on December 2, 2012 at 1:41 pm

Let’s say that you went ahead and purchased ISO 14971:2012, read Annex ZA, and you identified a couple of gaps in your procedure. After you revised your Risk Management Procedure to be compliant with the revised Standard, then what are you supposed to do?

For the next few weeks I plan to torture all of you with holiday music. If you don’t like it, buy a satellite radio for Christmas sake.

Most QA Managers struggle over whether they should purchase ISO 14971:2012 or not. I wrote a couple of blog postings about this, but my point was not to debate this question. My point was that companies need to be compliant with the MDD and the ISO 14971 Standard. The “changes” from the 2009 to the 2012 version are simply the European Commission reminding manufacturers that there are 7 aspects of the ISO 14791 Standard that  do not meet the requirements of the MDD. Therefore, if your company has already verified that your Risk Management Process is compliant with the MDD–then you have nothing to change. However, if your Risk Management Process is only compliant with ISO 14971:2009, then you need to revise your processes and procedures to address these 7 aspects.

Once you have made your revisions, how do you audit for compliance with ISO 14971:2012?

Step 1: Planning the Audit

This will be an internal audit and since you (the QA Manager) are the process owner for the Risk Management process, you cannot also audit this process. You need to assign someone that has the technical skill to perform the audit, but this person cannot be the process owner (you) or a direct report to the process owner (the rest of the QA department). Fortunately, the Director of Engineering is also trained as an internal auditor at your company. She is trained on ISO 14971:2009, but she is not trained on ISO 14971:2012. To address this gap, she must read the updated Standard to understand what’s new.

Clause 3.2 of ISO 14971 requires that top management review the Risk Management Process for Effectiveness.

Clause 3.2 of ISO 14971 requires that top management review the Risk Management Process for Effectiveness.

She has participated in risk management activities, but each product development engineer participates in risk management activities for their own design projects. Therefore, she has several projects she can sample risk management records from without auditing her own work. You have communicated that you need this audit finished sometime in December, because you want any CAPA’s resulting from the audit to be finalized before the next Management Review at the end of January. The timing of the Management Review is important, because the Risk Management Procedure requires that top management assess the effectiveness of the Risk Management Process during Management Review meetings.

There are no previous audit findings to close from the last audit of the Risk Management Process, but the Director of Engineering has 7 specific items to emphasize from the 2012 revision of the Standard and a revised procedure for Risk Management. Therefore, she will prepare for the audit by identifying some new interview questions to specifically address these changes–as well as some more generic, open-ended questions.

Specific Questions for 7 Items in ISO 14971:2012, Annex ZA:

1. How does the risk analysis evaluate the acceptability of risks in the lowest category? (This is a leading question, but it is specifically designed to determine if negligible risks are discarded.)

2. Please provide a few examples of how risks in the lowest category were reduced. (Sections 1 and 2 of the Annex I require all risks to be reduced as far as possible, and for all risks to be evaluated for acceptability. The wording of this question also allows auditors flexibility in their sampling.)

3.  How did the design team determine when they had implemented sufficient risk controls to minimize risks? (Many companies use a color-coded matrix as a quasi-objective method for determining when risks are adequately reduced. This process is often referred to as the ALARP concept. Annex ZA specifically prohibits using economic considerations as part of this determination.)

4. How did you conduct a risk-benefit analysis? (The Standard allows for performing a risk-benefit analysis when overall residual risks exceed the acceptability criteria as outlined in the risk management plan. However, the MDD requires an overall risk-benefit analysis in Section 1 of Annex I. Section 6 also requires that a risk-benefit analysis be performed for each individual risk.)

5. How were risk control options selected? (Section 2 of the MDD implies that the manufacturer shall review All the control options and pick the most appropriate ones. Therefore, the auditor should specifically look for evidence that the team systematically reviewed all possible control options to reduce risks–rather than stopping as soon as the risks were reduced to an acceptable level.)

6. What were your team’s priorities for implementation of risk control options? (It’s possible that the previous question will be sufficient to gather evidence that risk controls were implemented with the required prioritization as specified in the MDD. However, this question would be used as a follow-up question if it is not clear that the team prioritized the risk control options in accordance with Section 2 of Annex I.)

7. How was effect of labeling and warnings in the instructions for use incorporated into the estimation of residual risks? (Almost every company remembers to include residual risks in their IFU as a warning or caution statement. However, Section 2 of Annex I does not allow for including this information given to the users as a method of reducing risks. Therefore, in a Design FMEA you would not list labeling and IFUs in your column for current risk controls when you determine the risk. This should be identified as an action to be taken–with no impact on the score for residual risk.)

Auditor TipThe above questions are not examples of using the process approach, but each question is phrased in an open-ended manner to maximize the objective evidence gathered during the interview process. If you are doing a process audit, it’s still ok to include questions that use the element approach.

Generic Questions:

1. When was the ISO 14971:2012 version of the Standard added to the controlled list of external Standards?

2. Please provide examples of where you have updated the Essential Requirements Checklist (a Technical File document) to reference the newest revision of ISO 14971:2012, and please show at least one example of how the Risk Management Report was updated to reflect this revision.

3. How did you verify training effectiveness for the design team specific to the updated Risk Management Procedure prior to conducting a risk analysis?

Auditor TipThese generic questions do not require reading the ISO 14971:2012 Standard. Instead, each question forces the auditee to demonstrate their knowledge of the revised Standard by asking open-ended interview questions. Each of these questions is also designed to test linkages with other support processes. This is an example of how to use the process approach.

Step 2: Conducting the Audit

The next step of the auditing process is to conduct the audit. During the audit, the Director of Engineering will gather objective evidence of both conformity and nonconformity for the risk management process. The generic interview questions that were developed allow her to evaluate the effectiveness of linkages between the Risk Management Process and other processes such as: 1) document control, 2) creating technical documentation for regulatory submissions, and 3) the training process. The specific questions verify that each of the 7 elements identified in Annex ZA of ISO 14971:2012 are adequately addressed in the revised procedure. When the audit is completed, the auditor will have a closing meeting with the process owner (you) and the auditee(s) so that everyone is clear what the findings were, and if there were any nonconformities this is the time to clarify what needs to be done in order to prevent each nonconformity from recurring.

Step 3: Writing the Report & Taking Corrective Action(s)

This is no different from any other audit, but it is critical to have the report completed soon enough so that CAPA’s can be initiated (not necessarily completed) prior to the Management Review.

Step 4: Verifying Effectiveness of Corrective Action(s)

Many people struggle with verifying effectiveness of corrective actions–regardless of the process. My advice is to identify a process metric to measure the effectiveness. Then the effectiveness check is objective. For example, monitoring the frequently of updates to the list of external standards can help verify that the process for monitoring when Standards are updated is effective. Likewise, the frequency of updates to the Essential Requirements Checklist and the Risk Management records referenced in the Essential Requirements Checklist indicates if the Risk Management process is  being maintained. Finally, monitoring the lag between the time procedures are updated and when the associated training records are updated quickly identifies if there is a systemic problem with training or if a train gap is just an example of a single lapse.

  1. Hello There!
    First of all thank you for sharing such a valuable article, it is really a help for the ones interested in this subject.
    However, having published it over the www, well, some people may think voluntarily to comment it! 🙂 This can happen when you cook a free meal for strangers, so here is the thing! 🙂

    Let me mention only two things only (otherwise my universe is totally a happy bunny for the thoughts we had in this article) are still pinned in my mind having a read of your article, and I would be happy to know your or others thoughts about these.
    1., QA lead player
    For me the QA manager is mostly not the owner of the Risk management process, unless the QA manager is knowledgeable in medical science ( = having a PhD in medical science at least).
    I know … it took a long to time for the quality and regulatory fellows to confiscate this area, but seriously … is a couple of process stuff along with some lovely evaluation table really encourages us to do so? 🙂
    2., The seven curse to our most probable sins in Annex ZA
    I think there is a subtlety in these curses in the standards. I am recalling my memory only, but actually I think the actual interpretation is a wee more up to the discretion of the Notified body. With other words I think unless we know the perspective of the Notified Body about these discrepancies, we are more or less in darkness.

    Nevertheless, thank you for sharing your thoughts and expertise!

    • Thank you for reading and sharing your insightful comments. Your first comment is easy to address…
      1. I just happened to pick QA as the lead player, because I wanted to pick someone. I could easily have switched the roles, but by picking the QA Manager as the process owner, it created a hypothetical situation where someone other than QA had to do an audit. The roles could have been easily reversed for this hypothetical company. Every company is different and roles change over time. I worked in R&D, then operations, then engineering, and finally QA/RA was last. There was a full decade before I even started doing QA/RA. I actually learned the most about risk management from an automotive Product Engineer while I was a production supervisor–not in QA/RA or even in medical devices.
      2. I think “curse” and “sin” are words that seem harsh, but then again I killed QC. I have actually observed all 7 issues at more than one medical device company (thankfully not at the same company). Your argument that this is up for interpretation is the most popular response. However, this issue is no longer up for interpretation. The EU Commission has declared what the interpretation shall be and the days of each Notified Body doing things differently has almost fully eclipsed in the medical device industry. I think companies should be carefully reading the EU proposed regulations to see how little room the Commission has allowed for interpretation. The word “risk” appears 175 times in the proposal, and the language of Annex I’s section 1 and 2 have not softened. The Commission has actually added a new top priority for risk management and the four priorities are quite clear now:

      1. identify known or foreseeable hazards and estimate the associated risks arising from the intended use and foreseeable misuse;
      2. eliminate risks as far as possible through inherently safe design and manufacture;
      3. reduce as far as possible the remaining risks by taking adequate protection measures, including alarms; and
      4. provide training to users and/or inform users of any residual risks.

      If you haven’t already revised you Risk Management Procedure to address the requirements of the MDD, you might want to think about compliance with the EU proposed regulations with regard to Risk Management instead.

  2. Hey Rob,
    Is ISO 14971 a required step to complying with ISO 1345 if our product is stand alone software? Thanks!

    • Sorry, that was supposed to say ISO 13485!

    • In Clause 7.1 of ISO 13485, it references ISO 14971 and states that Risk Management shall be applied throughout the QMS. The Canadian Standards Association (www.csa.ca) published a guidance document called “14971 Plus” that I find helpful for implementing ISO 14971. I’m sure I’ve written something about this in the past too.

      ISO 14971 is considered “State of the Art”. Therefore, you are required to be compliant with ISO 14971 or the equivalent for CE Marking. The US FDA also recognizes ISO 14971. For software, there is a software standard that escapes my memory. I send an email to a friend of mine and see if they can leave a comment about this. That software standard will be more helpful to you than 14971, but you need both.

  3. My friend Leo Eisner (http://www.eisnersafety.com/industry_news/) was kind enough to email me the answer to my question above. ISO 62304 – Medical Device Software – Software Lifecycle Processes is the Standard. You can purchase a copy here: http://www.iso.org/iso/catalogue_detail.htm?csnumber=38421.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: