A New Way to Grade Findings

In GHTF, IMDRF, Internal Auditing, Uncategorized on March 24, 2013 at 7:36 pm

Grading Findings

Last November a new GHTF document was released on the topic of grading non-conformities: GHTF/SG3/N19:2012. This document is available on the new IMDRF website in the documents section. The 16-page document presents a new method for Certification Bodies to grade non-conformities and to communicate these findings to regulators such as the US FDA and Health Canada (e.g. – GD211 voluntary reports).

To download the guidance document, go to http://www.imdrf.org/.

To download the guidance document, go to http://www.imdrf.org/.

N19 recommends the same three-part structure for writing nonconformities that is taught in Lead Auditor Classes, and there is even a table of examples provided with poorly written findings and well-written findings with more specific references to objective evidence.

Section 4.2 of the guidance document, however, introduces a new concept for grading of findings. The traditional grading of findings is: Major, Minor, and Observations. Opportunities for Improvement (OFI) are no longer allowed in regulatory reports to avoid the appearance of providing consulting advice to clients. For internal audits and supplier audits, OFIs are still used by most auditors.

Figure 1 - Grading OverviewThe new grading process defined by the guidance document has a two-step process. The first step uses a grading matrix to quantitatively determine a grade for the finding based upon the impact upon the QMS and the frequency of occurrence.

The second step of the grading process is to review escalation rules that are defined in Section 4.2.2 of the guidance document. This section emphasizes the importance of using the word “absence” in the wording of findings if a required procedure is not present in the QMS. This type of finding should only happen during initiate certification audits where 100% of the required procedures are typically verified during the Stage 1 audit. If this occurs, then the grading is increased by 1 to a possible maximum of 5.Figure 2 - Grading Matrix

Another possible escalation event is the release of nonconforming devices outside the control of the manufacturer. If this occurs, then the grading is increased by 1 to a possible maximum of 5. If the required procedure is absent, and product is released that is nonconforming, the guidance states that the score should not be escalated above a 5.

In all of the Lead auditing courses I have taught, both of the above escalation events would be examples of a “Major Nonconformity.” Repeat occurrences of nonconformities would typically be escalated from a minor NC to a major NC, but in this new method the scores could be a “2” or a “4”—depending upon the impact upon the QMS.

Risk-Based MatrixI have had enough trouble in the past with training auditors to consistently grade findings during audits, and this is one of the most important sections of the exam for a Lead Auditing Course. Recently I suggested that a client consider using the risk analysis matrix that they were already using for process risk analysis and apply the matrix to grading of findings. An example of this type of matrix is shown below.

My client used semi-quantitative scores for severity (1-3) and occurrence (1-4). The two factors were multiplied to calculate a risk priority number (RPN) ranging from 1-12. The resulting matrix is also color coded to indicate the urgency of corrective action plans to be developed for the finding.

Has anyone implemented a grading system based upon this new guidance? If you have, please share your experiences here or on one of the LinkedIn Groups I have posted this question:

Medical Devices: QA/RA – http://bit.ly/SG3N19-QARA

ASQ – http://bit.ly/SG3N19-ASQ

Please share you own methods for grading findings?


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: