A New Way to Grade Findings

Last November a new GHTF document was released on the topic of grading non-conformities: GHTF/SG3/N19:2012. This document is available on the new IMDRF website in the documents section. The 16-page document presents a new method for Certification Bodies to grade non-conformities and to communicate these findings to regulators such as the US FDA and Health Canada (e.g. – GD211 voluntary reports).

To download the guidance document, go to http://www.imdrf.org/.

N19 recommends the same three-part structure for writing nonconformities that is taught in Lead Auditor Classes, and there is even a table of examples provided with poorly written findings and well-written findings with more specific references to objective evidence.

Section 4.2 of the guidance document, however, introduces a new concept for grading of findings. The traditional grading of findings is: Major, Minor, and Observations. Opportunities for Improvement (OFI) are no longer allowed in regulatory reports to avoid the appearance of providing consulting advice to clients. For internal audits and supplier audits, OFIs are still used by most auditors.

The new grading process defined by the guidance document has a two-step process. The first step uses a grading matrix to quantitatively determine a grade for the finding based upon the impact upon the QMS and the frequency of occurrence.

The second step of the grading process is to review escalation rules that are defined in Section 4.2.2 of the guidance document. This section emphasizes the importance of using the word “absence” in the wording of findings if a required procedure is not present in the QMS. This type of finding should only happen during initiate certification audits where 100% of the required procedures are typically verified during the Stage 1 audit. If this occurs, then the grading is increased by 1 to a possible maximum of 5.

Another possible escalation event is the release of nonconforming devices outside the control of the manufacturer. If this occurs, then the grading is increased by 1 to a possible maximum of 5. If the required procedure is absent, and product is released that is nonconforming, the guidance states that the score should not be escalated above a 5.

In all of the Lead auditing courses I have taught, both of the above escalation events would be examples of a “Major Nonconformity.” Repeat occurrences of nonconformities would typically be escalated from a minor NC to a major NC, but in this new method the scores could be a “2” or a “4”—depending upon the impact upon the QMS.

I have had enough trouble in the past with training auditors to consistently grade findings during audits, and this is one of the most important sections of the exam for a Lead Auditing Course. Recently I suggested that a client consider using the risk analysis matrix that they were already using for process risk analysis and apply the matrix to grading of findings. An example of this type of matrix is shown below.

My client used semi-quantitative scores for severity (1-3) and occurrence (1-4). The two factors were multiplied to calculate a risk priority number (RPN) ranging from 1-12. The resulting matrix is also color coded to indicate the urgency of corrective action plans to be developed for the finding.

Has anyone implemented a grading system based upon this new guidance? If you have, please share your experiences here or on one of the LinkedIn Groups I have posted this question:

Medical Devices: QA/RA – http://bit.ly/SG3N19-QARA

ASQ – http://bit.ly/SG3N19-ASQ

Please share you own methods for grading findings?

Advertisement

Long-time No Post

My apologies for taking so long to get another blog posting out. I will be posting something big today, but follow the link below to see what I’ve been up to.

Medical Device Academy

What if your Notified Body Auditor is Wrong?

My first certification audit ever didn’t go so well. The reason it didn’t go well is that the auditor wrote nonconformities that my boss and our regulatory consultant didn’t agree with. At the time, I was too inexperienced to know how to handle it. My boss and the consultant, however, totally lost it. I’ve never seen veins that big in someone’s forehead–even in cartoons.

I asked them both to leave the room, because I was afraid to “push back” on the auditor. Many Management Representatives feel the same way that I did during that initial certification audit. The best way to summarize our concerns is with the following picture:

Don’t poke the bear!

This week’s entertainment is for a friend of mine. Thank you for all your help. You’ve made my year.

Recently another LinkedIn group member emailed me to say that they have seen several auditors for registrars identifying nonconformities that represented their own personal opinions rather than specific requirements of the Standard. For example: there is a requirement to assign management responsibilities and document it, but there is no requirement to have an organization chart.

Another common mistake is when auditors insist that a company must create a turtle diagram for every single process. I support the use of turtle diagrams 100%, but the only requirement in the Standard is to use the process approach–not turtle diagrams specifically.

My favorite is my own personal mistake. I wrote a nonconformity for not having a process for implant registration cards for a company that was planning to ship a high-risk implant product to Canada. There is a requirement for implant registry cards, but I forgot that Canada defines “implants” in this case as only a very short list of implant devices–not implants in general.

Auditors are human. These are audit findings–not a jail sentence. Everyone needs to remember that the worst that can happen is that you receive a nonconformity. If the auditor finds a nonconformity, then you need to develop a CAPA plan. If the auditor finds nothing, you still need to do your own internal audits to identify nonconformities and to continuously improve processes.

The question is, what should you do when an auditor is wrong?

I recommend that you “push back”, but you need to know how. Many consultants suggest saying, “Can you show me in the Standard where it says I have to do that?” That’s just like poking a bear. If you do it once, it’s annoying. If you do it multiple times, an auditor might just eat you.

One Management Representative did that to me after I had taken the time to review the requirements with him. I responded by holding the ISO 13485 Standard in front of him and reciting clause 7.3.2. He responded by saying, “Well that’s up for interpretation.” I offered to recite the ISO 14969 guidance document for him, but his boss told him to  shut up.

This certainly wasn’t the only time a client pushed back during a registration audit, but other clients have had the sense to argue about things they actually understood.

One of the clients I audited, said that he would change the topic to the auditor’s favorite sports team. That’s one approach. I’m sure that more than one client has taken the approach of asking me to explain where they can learn about best practices. I’m sure that they were somewhat successful. Another approach is to slide the lunch menu in front of them; I have only met one auditor that would not be distracted by a lunch menu.

Here’s my step-by-step approach to pushing back when you disagree with an auditor:

1. shut-up and look it up (before you open your mouth, grab the applicable external standard and look up exactly what you are looking for)

2. If you are still convinced that your auditor is wrong, then tell that you are having trouble finding the requirement. Show them where you are looking, and then ask them to help you find the requirement.

3. If the auditor can’t show you where you are wrong, or it appears that the auditor is interpreting the Standard as they see fit, then focus on asking the auditor for guidance on what they will be looking for in your CAPA plan.

4. If the CAPA plan the auditor is looking for is something you think is a good idea, then shut up and implement the improvements. If the CAPA plan is not acceptable to you, then you should ask what the process is for resolution of disputes.

5. No matter what, don’t start an argument with the registrar. They actually enjoy it. They like a challenge, and they resent people with less experience critizing them.

6. If you still disagree with your auditor, then you should ask if the auditor can explain the process for appealing findings and follow that process.

ISO/EN 14971:2012 and the MDD

Brigid beat me to it. She did a nice job of explaining her take each of the 7 issues that the EU Commission has with the ISO 14971 Standard.

QA Kiwi

Have you performed a gap analysis on your Risk Matrix against the MDD requirements?

No you don’t need to buy EN ISO14971:2012, but if you have an EU certificate and must comply with the EU Medical Devices Directive, then you must apply the EU requirements for risk management, spelled out in the new annexes. This version of 14971 was accepted as an EU harmonised standard in August 2012 and therefore applies now. The EU requirements aren’t new, but it has been common for companies to apply 14971 without noticing that there are some subtle differences in the Essential Requirements. The EN standard spells them out for us.

A copy of the annexes is available  here from the Danish Standards website. We are grateful to them for acknowledging that this is more about regulation than it is about standards.

While none of the requirements are new, they will impact on how your…

View original post 433 more words

Brigid beat me to it. She did a nice job of explaining her take each of the 7 issues that the EU Commission has with the ISO 14971 Standard.

QA Kiwi

Have you performed a gap analysis on your Risk Matrix against the MDD requirements?

No you don’t need to buy EN ISO14971:2012, but if you have an EU certificate and must comply with the EU Medical Devices Directive, then you must apply the EU requirements for risk management, spelled out in the new annexes. This version of 14971 was accepted as an EU harmonised standard in August 2012 and therefore applies now. The EU requirements aren’t new, but it has been common for companies to apply 14971 without noticing that there are some subtle differences in the Essential Requirements. The EN standard spells them out for us.

A copy of the annexes is available  here from the Danish Standards website. We are grateful to them for acknowledging that this is more about regulation than it is about standards.

While none of the requirements are new, they will impact on how your…

View original post 433 more words

Special Request for German-Speaking Supplier Quality Person

A friend of mine is looking for someone to work on a short-term project (i.e. – approx. 2 weeks). The person should be an experienced supplier quality engineer. The person must be fluent in German (read, write and speak). Work is in Austria. No budget for placement firm. Urgent immediate need for December. If you are not already connected with me, please request to connect with me on LinkedIn so I am only forwarding qualified candidates to my friend.

Just finished my second webinar!

If you missed it, I am performing another one LIVE with FXConferences next Wednesday, November 28 @ 11:30am EST. You can register for it by clicking here. I will not be reading my slides or walking you through changes one at a time. Instead, you’ll get some ideas for what to prepare in the next 12 months while politics argues over the wording of the final regulations. Hope to see you there.

The New EU Regulations – 21 Days and Counting!

The EU Directive will become Regulations…but when? September 26 the proposed regulations are scheduled to be released in draft form. Plans for implementation of the Interim Measures have already begun, but the regulations will not be finalized for 18-24 months while the politics takes over. My magic 8-ball tells me that there is a precedent on this side of the pond that can help us predict the future.

A few weeks ago I published a posting with a cheeky Brit–Lily Allen. Here’s one of Lily’s own favorites by a another talented British singer, song writer named Kate Nash.

Throughout the history of healthcare regulations worldwide there have been three rules that are never broken:

1. Regulations always get tougher.
2. Regulations are only partially effective.
3. Regulations cost everyone more money.

The PIP scandal lit a fire under the European Parliament, the Council and Notified Bodies. Now all three stakeholders are fighting to show the public that they are doing everything they can to ensure safety. Unfortunately, no matter what changes are made it is extremely difficult to prevent unethical behaviors.

Before I make predictions, we all need to remember that there is a larger news story–the European Economy.

The status of the European Economy will have the greatest impact on new regulations. My best evidence is the US FDA.

The FDA has been trying to improve the turnaround on submission reviews for my entire career. For a period of about 8 years, matching closely with the Presidential terms of George W. Bush, it seemed to get easier to get products through the FDA logjam. Then the global economy tanked and political winds changed in 2009.

Over the past three years, Republican’s have gained power and Congress is now pushing the FDA to actually improve the metrics for product approval. The FDA will now have 200 additional reviewers, and every plan for improving turnaround that has been tried is back on the table.  The FDA was given the funds to grow its army of inspectors first, and now the FDA is granted additional funds to hire additional reviewers and train them.

The European Union includes countries that are struggling to provide basic services, while other countries don’t want to bail their European neighbors out of debt. How is the European Parliament and the Council going to increase regulation of medical devices when everyone knows that regulations will cost more money?

The short answer is…they can’t.

One of two things must happen before true change can occur:

1. another healthcare scandal could trigger this change, or
2. the economy could improve.

Based upon the sluggish recovery of the US economy, I don’t see how #2 will happen in Europe during the next 18-24 months. I can’t predict #1, but historically scandals are years apart.

MAGIC 8-BALL TIME

I predict that the draft regulations will get watered down during the co-decision period. The most popular legislative tool is the “transition period”. For example, UDI legislation was passed in 2007 in the US but the proposed rule was not published by the FDA until 2012. The proposed rule includes a 7-year transition period for implementation of the new rule.

If the new EU regulation is finalized in 18-24 months, we can expect a long transition period during which various pieces of the regulations will be implemented. This transition period is essential for Notified Bodies to gradually increase their staff and for training new auditors. This will also give companies several years to organize their own plans for addressing the new regulations.

The one change I predict to happen quickly is consolidation. 60+ Notified Bodies are more expensive for the EU to support than a few large Notified Bodies. The FDA is a single, centralized regulatory body. The EU will not achieve the same degree of centralization, but I predict “a great consolidation”.

My final prediction is related to the vigilance process. In the US the MDR process has become highly automated and electronic submissions with a public database are the norm. This has allowed the FDA to rely on data analysis to identify problems and redirected the burden of data entry from the FDA to industry. We can expect Europe to follow this trend, by centralizing all vigilance reporting. The only remaining question about vigilance is how long will the transition period need to be for revision 8 (of MEDDEV 2.12/1).

On Vacation in Camden, Maine

I am on vacation in beautiful Camden, Maine this week.

It’s Heaven

This is where I grew up, and it’s always heavenly to visit your childhood home. Despite all the changes to this town over the past 40 years I have spent here, the harbor and Mount Battie have changed very little. Unfortunately, not all of my family is here, but I’m still enjoying a very Happy Father’s Day. Happy Father’s Day to you dad!

And here’s my gift to myself…

You’d never know we are related:-)

Ivy Finally Arrives!

This song just seemed appropriate.

Sorry about missing last week’s posting, but we had a baby!

Ivy Kathleen Kari Packard was born at 7:10am on April 7 (wow…I can’t even keep track of the dates). She was 7lbs & 11oz. She was 20 inches short, but I suspect she might grow.

Alex, Lillie and Gracie were really good and put up with spending each night away from Lisa. Hopefully Noah & Bailey will get to see Ivy soon.

One more video to make you laugh!…Ivy will not be drinking Evian anytime soon.