What if your Notified Body Auditor is Wrong?

My first certification audit ever didn’t go so well. The reason it didn’t go well is that the auditor wrote nonconformities that my boss and our regulatory consultant didn’t agree with. At the time, I was too inexperienced to know how to handle it. My boss and the consultant, however, totally lost it. I’ve never seen veins that big in someone’s forehead–even in cartoons.

I asked them both to leave the room, because I was afraid to “push back” on the auditor. Many Management Representatives feel the same way that I did during that initial certification audit. The best way to summarize our concerns is with the following picture:

Don’t poke the bear!

This week’s entertainment is for a friend of mine. Thank you for all your help. You’ve made my year.

Recently another LinkedIn group member emailed me to say that they have seen several auditors for registrars identifying nonconformities that represented their own personal opinions rather than specific requirements of the Standard. For example: there is a requirement to assign management responsibilities and document it, but there is no requirement to have an organization chart.

Another common mistake is when auditors insist that a company must create a turtle diagram for every single process. I support the use of turtle diagrams 100%, but the only requirement in the Standard is to use the process approach–not turtle diagrams specifically.

My favorite is my own personal mistake. I wrote a nonconformity for not having a process for implant registration cards for a company that was planning to ship a high-risk implant product to Canada. There is a requirement for implant registry cards, but I forgot that Canada defines “implants” in this case as only a very short list of implant devices–not implants in general.

Auditors are human. These are audit findings–not a jail sentence. Everyone needs to remember that the worst that can happen is that you receive a nonconformity. If the auditor finds a nonconformity, then you need to develop a CAPA plan. If the auditor finds nothing, you still need to do your own internal audits to identify nonconformities and to continuously improve processes.

The question is, what should you do when an auditor is wrong?

I recommend that you “push back”, but you need to know how. Many consultants suggest saying, “Can you show me in the Standard where it says I have to do that?” That’s just like poking a bear. If you do it once, it’s annoying. If you do it multiple times, an auditor might just eat you.

One Management Representative did that to me after I had taken the time to review the requirements with him. I responded by holding the ISO 13485 Standard in front of him and reciting clause 7.3.2. He responded by saying, “Well that’s up for interpretation.” I offered to recite the ISO 14969 guidance document for him, but his boss told him to  shut up.

This certainly wasn’t the only time a client pushed back during a registration audit, but other clients have had the sense to argue about things they actually understood.

One of the clients I audited, said that he would change the topic to the auditor’s favorite sports team. That’s one approach. I’m sure that more than one client has taken the approach of asking me to explain where they can learn about best practices. I’m sure that they were somewhat successful. Another approach is to slide the lunch menu in front of them; I have only met one auditor that would not be distracted by a lunch menu.

Here’s my step-by-step approach to pushing back when you disagree with an auditor:

1. shut-up and look it up (before you open your mouth, grab the applicable external standard and look up exactly what you are looking for)

2. If you are still convinced that your auditor is wrong, then tell that you are having trouble finding the requirement. Show them where you are looking, and then ask them to help you find the requirement.

3. If the auditor can’t show you where you are wrong, or it appears that the auditor is interpreting the Standard as they see fit, then focus on asking the auditor for guidance on what they will be looking for in your CAPA plan.

4. If the CAPA plan the auditor is looking for is something you think is a good idea, then shut up and implement the improvements. If the CAPA plan is not acceptable to you, then you should ask what the process is for resolution of disputes.

5. No matter what, don’t start an argument with the registrar. They actually enjoy it. They like a challenge, and they resent people with less experience critizing them.

6. If you still disagree with your auditor, then you should ask if the auditor can explain the process for appealing findings and follow that process.

Advertisement

Death by CAPA

I have no theme to relate this song with my posting, but you just can’t go wrong with blue jeans and a black t-shirt…

You might want to play this video twice…it’s a long posting.

I completed almost 100 audits in the past two years, and I review the Corrective Action and Preventive Action (CAPA) process during every single audit. Surprisingly, this seems to be a process with more variation from company to company than almost any other process I review. This also seems to be a major source of non-conformities. In the ISO 13485 Standard, clause 8.5.2 (Corrective Action) and clause 8.5.3 (Preventive Action) have almost identical requirements. Third-party auditors, however, emphasize that these are two separate clauses. We are purists. Although we acknowledge that companies may implement preventive actions as an extension to a corrective action, we also expect to see examples of actions that are strictly preventive in nature.

Many companies seem to be confused, but it doesn’t need to be. Just ask yourself one question. What is the source of this action?

If the answer is a complaint, audit nonconformity, or rejected components—then your actions are corrective.

If the answer is, a negative trend that is still within specifications or an “opportunity for improvement” (OFI) identified by an auditor—then your actions are preventive.

If you are investigating the root cause of a complaint, people will sample additional records to estimate the frequency of the quality issue. I describe this as investigating the depth of a problem. The FDA emphasizes the need to look to other product lines, or processes, to see if a similar problem exists. I describe this as investigating the breadth of a problem. Most companies describe actions taken on other product lines and/or processes as “preventive actions.” This is not always accurate. If a problem is found elsewhere, actions taken are corrective. If potential problems are found elsewhere, actions taken are preventive. You could have both types of actions, but most people incorrectly identify corrective actions as preventive actions.

Another common mistake is to characterize corrections as corrective actions.

The most striking difference between companies seems to be the number of CAPAs they initiate. There are many reasons, but the primary reason is failure to use a risk-based approach to CAPAs. Not every quality issue should result in the initiation of a formal CAPA. The first step is to investigate the root cause of a quality issue. The FDA requires that the root cause investigation is documented, but if you already have an open CAPA for the same root cause…

DO NOT OPEN A NEW CAPA!!!

If you do not have a CAPA open for the root cause that you identify, then what should you do?

I know this will shock everyone, but…it depends.

The image below gives you my basic philosophy.

 

 

 

 

 

 

 

 

Most investigations document the estimated probability of occurrence for a quality issue. This is only half of the necessary risk analysis I describe below. Another aspect of an investigation is to document the severity of potential harm resulting from the quality issue. If customer satisfaction, safety or efficacy are affected by a quality issue—the severity is big. Risk is the product of severity and probability of occurrence.

If the estimated risk is low and probability of occurrence is known, then alert limits and action limits can be statistically derived. These quality issues are candidates for continued trend analysis—although the alert limit or action limit may be modified in response to an investigation. If the trend analysis results in identifying events that require action, then that is the time when a formal CAPA should be opened. If the trend remains below your alert limit, then no formal CAPA is needed.

If the estimated risk is moderate or the probability of occurrence is unknown, then a formal CAPA should be considered. Ideally, you will be able to establish a base-line for occurrence and demonstrate that frequency decreases upon implementation of corrective actions. If you can demonstrate a significant drop in frequency, this verifies effectiveness of actions taken. If you need statistics to show a difference, then your actions are not effective.

If estimated risk is high or there are multiple causes that require multiple corrective actions, a quality improvement plan may be more appropriate. There are two clauses in the Standard that apply. Clause 5.4.2 addresses planning of changes to the Quality Management System. For example, if you correct problems with your incoming inspection process—this addresses 5.4.2. Clause 7.1 addresses planning of product realization. For example, if you correct problems with a component specification where the incoming inspection process is not effective—this addresses 7.1. Depending upon the number of contributing causes and the complexity of implementing solutions, the plan could be longer or shorter. If it will take more than 90 days to implement a corrective action, you might consider the following approach.

Step 1 – open a CAPA

Step 2 – identify the initiation of a quality plan as one of your corrective actions

Step 3 – close the CAPA when your quality plan is initiated (i.e. – documented and approved)

Step 4 –verify effectiveness by reviewing the progress of the quality plan in management reviews and other meeting forums…you can cross-reference the CAPA with the appropriate management review meeting minutes in your effectiveness section

If the corrective action required is installation of new equipment and validating that equipment, the CAPA can be closed as soon as a validation plan is created. The effectiveness of the CAPA is verified when the validation protocol is successfully implemented and a positive conclusion is reached. The same approach also works for implementing software solutions to better manage processes. The basic strategy is to get the long-term improvement projects started with the CAPA system, but monitor the status of these projects outside the CAPA system.

Best practices would be the implementation of Six-sigma projects with formal charters for each long-term improvement project.

NOTE: I believe in closing CAPAs when actions are implemented, and tracking the effectiveness checks for CAPAs as a separate quality system metric. If closure takes more than 90 days, the CAPA should probably be converted to a Quality Plan. This is NOT intended to be a “work around” to give companies a way to extend CAPAs that are not making progress in a timely manner.