13485cert

Posts Tagged ‘Audit’

ISO 14971 – Buy the new 2012 version?…comment please

In CE Mark, CE Medical, International Standard, ISO, ISO 14971, Medical CE, Medical Device, Risk Analysis, Risk Management on August 2, 2012 at 8:38 pm

I’m sure that there are some that disagree with my determination that the latest revision of EN 14971, revision 2012, is unnecessary (the European Commission certainly does).

 You will have to go to my website to read my cheeky posting on this topic.

And here’s another cheeky attitude from the UK…(sorry, this is not a family channel).

Therefore, I would like to clarify why I feel this way by reviewing how risk is addressed in the MDD (93/42/EEC as modified by 2007/47/EC).

  1. The term risk is mentioned only 4 times in the Articles in the MDD
  2. The term risk is mentioned once in Annex II and III, twice in Annex VII, and three times in Annex VIII and X—for a total of 10 times.
  3. The other 41 times risk is mentioned are in the Essential Requirements (i.e. – Annex I).

When companies submit a Design Dossier for review by a Notified Body, an Essential Requirements Checklist is included. This references, in table format, how all the requirements of Annex I are being met—including those related to risks. Throughout Annex I, a similar phrase is repeated many times. For example, in the first Essential Requirement (ER1) it states: “…any risks which may be associated with [a device’s] intended use [shall] constitute acceptable risks when weighed against the benefits to the patient and are compatible with a high level of protection of health and safety.” In ER2 it states: “the manufacturer must…eliminate or reduce risks as far as possible…”. There is no room in the MDD for consideration of cost or economic impact when the manufacturer is designing a device with regard to risks and benefits.

If a company’s Risk Management Procedure has been found to be acceptable by a Notified Body, and the company has addressed all the Essential Requirements (ERs) with regard to risk, then there should be no impact from these 7 deviations identified in EN 14971:2012. However, if your company has not addressed each of these ERs, then you might want to consider each of these areas:

  1. Treatment of negligible risks
  2. Discretionary power of the manufacturer as to the acceptability of risks
  3. Risk reduction “as low as possible” (ALAP) verses “as low as reasonably possible” (ALARP)
  4. Discretion as to whether as risk benefit analysis needs to take place
  5. Discretion as to the risk control option/measures
  6. Deviation as to the first risk control method
  7. Information of the users influencing the residual risk

My final advice is to review Annex I and Annex X from the perspective of risk management. You may realize that you have some gaps that nobody noticed. After all, audits are just a sample.

PS – I think it’s ironic that the origins of the ALARP principle are UK case law (see link above).

Attention Auditors! – Have you read ISO 19011?

In Audit Schedule, Internal Auditing, International Standard, ISO, ISO 19011, PDCA, Procedures, Quality Management Systems on July 20, 2012 at 2:58 pm

If you have ever taken a lead auditor course, one of the critical handouts for the class should have been ISO 19011. The title is “Guidelines for Auditing Management Systems”. In November of last year, this standard was updated and the changes were not superficial.

The background entertainment for this week is one of my favorite modern rock songs, but it never seemed to get much air time. I hope you enjoyed the 90’s.

ISO 19011 covers the topic of quality management system auditing. This Standard provides guidance on managing audit programs, conducting both internal and external audits, and how to determine auditor competency. Improvements to the New 2011 Version of the Standard include:

  1. Broadening the scope to all management systems
  2. Clarifying the relationship between ISO 17021 and ISO 19011
  3. Introduction of the remote audit methods
  4. Introduction of risk as an auditing concept
  5. Confidentiality is a “new” principle
  6. Clause 5, Managing an audit program, was reorganized
  7. Clause 6, Performing an audit, was reorganized
  8. Clause 7, Competence and evaluation of auditors, was reorganized & strengthened
  9. Annex B is new and the contents of the help boxes was moved to this Annex
  10. Annex A now includes examples of discipline-specific knowledge and skills

One of the most common points of confusion in the lead auditor course is the difference between 1st, 2nd and 3rd party audits. In the previous revision of this Standard, this was just a note at the bottom of page 1 and the top of page two. The note was not very clear either. The new version of 19011, in Table 1 (reproduced below), the difference between these three types of auditing is crystal clear:

The above table is just an example of the improvements made to ISO 19011, and of course there is little value-add to clarifying a definition. Figure 1 from the new version, “Process flow for the management of an audit program, is a better example of a “value-add”. This vertical flow chart is reminiscent of Figure 1 from ISO 14971:2007. It categorizes the various stages of audit program management into the Plan-Do-Check-Act (PDCA) cycle. I highly recommend this style for presenting any process in your internal procedures as an example of best practices in writing an SOP. The flow chart even references each of the clauses in the Standard. Unfortunately Figure 2, “Typical audit activities”, does not categorize the stages of audit activities (Clauses 6.2 – 6.7 of the revised Standard) into the PDCA cycle. I guess they needed to leave some improvement for the next revision.

The new version retained the opening meeting checklist that was in the previous revision (Clause 6.4.2), and Clause 6.4.9 has a brief closing meeting checklist. Figure 3, “Overview of the process of collecting and verifying information”, is a poor example of a flow chart. Should I make a better one? (Send me an email if you think I should.)

The most valuable changes in this revision are Clause 5.3.2, “Competence of the person managing the audit program”, and all of Clause 7. Most of the audit procedures I read neglect to define the qualifications and method for determining competency of the audit program manager. Clause 5.3.2 tells you how. Put it in your own procedure. Most of the procedures I read include qualifications for a “Lead Auditor”, but I seldom see anything regarding competency. Unfortunately, this Standard only specifically addresses “Lead Auditor” competency in a two-sentence paragraph—Clause 7.2.5. When I teach people how to be a lead auditor, I spend more than an hour on this topic alone.

The Standard would be more effective by providing an example of how 3rd party auditors become qualified as a Lead Auditor. 3rd party accreditation requires the auditor to be an “acting lead” for audit preparation, opening meeting, conducting the audit, closing meeting, and final preparation/distribution of the audit report. This must be performed for 15 certification audits (i.e. – Stage 2 certification or recertification), and another qualified lead auditor must evaluate you and provide feedback.

The last big additions to this Standard were the Appendices. Annex A provides examples of discipline-specific knowledge and skills of auditors. This section is a little on the boring side. I prefer to tell a story about the internal auditor that was auditing incoming inspection—but they had no idea how to check for calibration or how to measure components. Appendix B, the finale, has a table (Table B.1) that provides some guidance on how to conduct remote audits (i.e. – desktop audits). I was pleased to see that conducting interviews is a major part of remote auditing in this table. Section B.7 provides some suggestions with regard to conducting interviews, but if you exhibit all 13 of the professional behavior traits found in Clause 7.2.2 then you really don’t need any advice on how to speak with people. For the rest of us mortals, we could use a five day course on interviewing alone.

Additional guidelines are available on the ISO website.

How to Train an Auditor on the Process Approach

In Audit Schedule, Internal Auditing on June 8, 2012 at 5:53 am

Country music fans are loyal blog readers too.

            I have been reviewing the trends for how people find my website, and a large number of you appear to be very interested in my auditing schedules and other audit-related topics. Therefore, this week’s blog is dedicated to training auditors on the process approach.

First, the process approach is just a different way of organizing audits. Instead of auditing by clause, or by procedure, instead you audit each process. Typical processes include:

  1. Design & Development
  2. Purchasing
  3. Incoming inspection
  4. Assembly
  5. Final Inspection
  6. Packaging
  7. Sterilization
  8. Customer Service
  9. Shipping
  10. Management Review
  11. CAPA
  12. Internal Auditing

There are two reasons why the process approach is recommended. First, the process approach identifies linkages between processes as inputs and outputs. Therefore, if there is a problem with communication between departments the process approach will catch it. If only a procedural audit is performed, the lack of communication to the next process is often overlooked. Second, the process approach is a more efficient way to cover all the clauses of the ISO Standard than auditing each clause (i.e. – the element approach).

My rationale for the claim of greater efficiency is simple: there are 19 required procedures in the ISO 13485 Standard, but there are only 12 processes identified above. The “missing” procedures are actually incorporated into each process audit. For example, each process audit requires a review of records as input and outputs. In addition, training records should be sampled for each employee interviewed during an audit. Finally, nonconforming materials can be identified and sampled at incoming inspection, in assembly processes, during final inspection, during packaging, and even during shipment.

The tool that BSI uses to teach the process approach is the “Turtle Diagram”. The following picture illustrates where the name came from.

Illustration of the Process Approach

Process Auditing – “Turtle Diagram”


The first skill to teach a new auditor is the interview. Each process audit should begin with an interview of the process owner. The process owner and the name of the process are typically documented in the center of the turtle diagram. Next most auditors will ask, “Do you have a procedure for ‘x process’?” This is a weak auditing technique, because it is an “closed-ended” or yes/no. This type of question does little to help the auditor gather objective evidence. Therefore I prefer to start with the question, “Could you please describe the process?” This should give you a general overview of the process if you are unfamiliar with it.

After getting a general overview of the process, I like to ask the question: “How do you know how to start the process.” For example, inspectors know that there is material for incoming inspection, because raw materials are in the quarantine area. I have seen visual systems, electronic and paper-based systems for notifying QC inspectors of product to inspect. If there is a record indicating that material needs to be inspected—that is the ideal scenario. A follow-up question is, “What are the outputs of the inspection process?” Once again, the auditor should be looking for paperwork. Sampling these records and other supporting records is how the process approach addresses Clause 4.2.4—control of records.

The next step of the process approach is to “determine what resources are used by incoming inspection.” This includes gages used for measurement, cleanliness of the work environment, etc. This portion of the process approach is where an auditor can review calibration, gowning procedures, and software validation. After “With What Resources,” the auditor then needs to identify all the incoming inspectors on all shifts. From this list the auditor should select people to interview and follow-up with a request for training records.

The sixth step of the process is to request procedures and forms. Many auditors believe that they need to read the procedure. However, if a company has long procedures this could potentially waste valuable time. Instead, I like to ask the inspector to show me where I can find various regulatory requirements in the procedures. This approach has the added benefit of forcing the inspector to demonstrate they are trained in the procedures—a more effective assessment of competency than reviewing a training record.

The seventh and final step of the turtle diagram seems to challenge process owners the most. This is where the auditor should be looking for department Quality Objectives and assessing if the department objectives are linked with company Quality Objectives. Manufacturing often measures first pass yield and reject rates, but every process can be measured. If the process owner doesn’t measure performance, how does the process owner know that all the required work is getting done? The seventh step also is where the auditor can sample and review monitoring and measurement of processes, and the trend analysis can be verified to be an input into the CAPA process.

In my brief description of the process approach I used the incoming inspection process. I typically choose this process for training new auditors, because it is a process that is quite similar in almost every company and it is easy to understand. More importantly, however, the incoming inspection process does a great job of covering more clauses of the Standard than most audits. Therefore, new auditors get a great appreciation for how almost all the clauses can be addressed in one process audit.

If you have questions, or you would like a copy of the turtle diagram I use for documentation of audits, please submit a request on my website contact us page.

How many hours does your company spend auditing?

In Audit Schedule, Internal Auditing, ISO 13485, Quality Management Systems on May 27, 2011 at 6:01 am

For this week’s video I picked a song played by Stevie Ray Vaughan and Jeff Healey–Look at Little Sister. They are two of the worlds greatest guitar players ever and both died way too soon.

Just in case you can’t connect…here’s another great Jeff Healy performance.

Each week I audit a different company or I teach a group of students how to audit. In the courses I teach, I use a slide that gives an example of an audit schedule (see my example above).

On the surface, this example seems like a good schedule. There are 12 auditors performing two audits each per year. If each auditor spends a day auditing and another day writing the report, the combined resources equal 48 days (~$20,000) allocated to auditing and each person spends less than 2% of their work year auditing.

Unfortunately, I have learned that the quality of auditing is directly related to how much time you spend auditing. Therefore, I recommend using fewer auditors. There is no perfect number, but “less is more”. My example also has another fundamental weakness. The audit schedule does not take full advantage of the process approach. Instead of performing an independent audit of document control and training, these two clauses/procedures should be incorporated into every audit. The same is true of maintenance and calibration. Wherever maintenance and calibration is relevant, these clauses should be investigated as part of auditing that area. For example, when the incoming inspection process is audited it only makes sense to look for evidence of calibration for any devices used to perform measurements in that area. For a second example…when the production area is being audited, it only makes sense to audit maintenance of production equipment too.

If the concept of process auditing is fully implemented, the following clauses can easily be audited in the regular course of reviewing other processes: 4.2.1) Quality System Documentation, 4.2.3) Document Control, 4.2.4) Record Control, 5.3) Quality Policy, 5.4.1) Quality Objectives, 6.2.2) Training, 6.3) Maintenance, 6.4) Work Environment, 7.1) Planning of Product Realization & Risk Management, 7.6) Calibration, 8.2.3) Monitoring & Measurement of Processes, 8.5.2) Corrective Action, and 8.5.3) Preventive Action. This strategy reduces the number of audits needed by more than half.

Another way to embrace the process approach to auditing is to assign auditors to processes that are upstream or downstream in the product realization process from their own area. For example, Manufacturing can audit Customer Service to better understand how customer requirements are confirmed during the order confirmation process. This is an example of auditing upstream, because Manufacturing receives the orders from Customer Service—often indirectly through an MRP system. Using this approach allows someone from manufacturing to identify opportunities for miscommunication between the two departments. If Regulatory Affairs audits the engineering process, this is an example of auditing downstream. Regulatory Affairs is often defining the requirements for the Technical Files and Design History Files that Engineering creates. If someone from Regulatory Affairs audits these processes, the auditor will realize what aspects of technical documentation are poorly understood by Engineering and quickly identify retraining opportunities.

One final aspect of the example audit schedule that I think can be improved is the practice of auditing the same process twice per year. This practice doesn’t seem to work very well for a few reasons. First, it requires that an auditor prepare for an audit twice per year and write two reports instead of one. This doubles the amount of time auditors spend in preparation and follow-up activities associated with an audit. Second, doubling the number of audits naturally shortens the duration of each audit. It is more difficult for auditors to cover all the applicable clauses in a shorter audit, because it takes time to locate records and pursue follow-up trails. Longer audits, covering more clauses, make it easier for the auditor to switch to a different clause while they are waiting for information. Third, if an area is audited every six months, it is often difficult to implement corrective actions and produce evidence of effectiveness before the area is due for auditing again.

It is not possible for me to provide a generic audit schedule that will work for every company or even to show how all the clauses will be addressed in one table. I can, however, provide an example of an improved schedule that illustrates the above concepts. This example (see below) uses four auditors instead of 12, and the number of days planned for each audit is two days instead of one. The preparation and reporting time is still one day per audit, and therefore the combined resources equal 24 days (~$10,000) allocated to auditing and each person spends 2.5 % of their work year auditing. I have provided a copy of this improved plan below. My intention is not to create the perfect plan, but to give audit program managers some new ideas for more efficient utilization of resources. I hope this helps, and please share your own ideas as comments to this posting.

%d bloggers like this: